Equity or Advantage?

1. Equity or Advantage? The effect of receiving access arrangements in university exams on students with specific learning difficulties Helen Duncan (Senior Neurodiversity Adviser) Disability Resource Centre

2. Background • Courses assessed by examinations • Assessing knowledge, ability and skills in subject • Construct irrelevant variance • information processing impairments • Marks reflect variability in processing speed, not just variability in skill • Exam arrangements for SpLD students: 25% extra time Use of word processor

3. LEARNING, KNOWLEDGE & CRITICAL THINKING assessed by TIMED EXAMS STUDENTS WITH SPLD irrelevant PERFORMANCE construct Barrier POTENTIAL Degree Classification Exam Arrangements skills Exam arrangements aim to permeate the construct-irrelevant skills barrier & link potential with performance.

4. THE DEBATE Tension between: • Ensuring equal opportunities: • levelling the playing field • Statutory duty to relieve a student of any substantial disadvantage that might arise as a result of their disability in the exam • Maintaining academic standards: • preserve the integrity of the examination for the full cohort • arrangements should not give one student an advantage over others

5. Extant research Contradictory findings • Only students with SpLD improved performance with extra time (Maximum impact thesis) • All students improve performance, but SpLD students show greatest gains (differential boost theory) • All students improve performance with extra time (over-inflation of scores)

6. Extant Research findings: patterns across studies Completely sufficient Is standard time adequate to complete the test? Moderately sufficient Insufficient Both SpLD & TD; but SpLD benefit more Who benefits from extra time? Both SpLD & TD; but TD benefit more SpLD only Differential Boost Hypothesis Over- inflation of scores Theoretical paradigm Maximum Potential Thesis

7. Test ceiling removed Test Ceiling TD TD TD SpLD SpLD SpLD Legend: Test Ceiling Score in standard time Maximum potential thesis Differential Boost hypothesis Over-inflation of scores Potential increase in score with extra time MODERATELY SUFFICIENT TIME SUFFICIENT TIME INSUFFICIENT TIME TD candidates reach test ceiling in standard time TD candidates are closer to the test ceiling in standard time No test ceiling

8. Critique of existing research Research limitations: • Research design • Ecological validity • heterogeneity

9. Critique of existing research ‘the body of research examined is not adequate to address many of the questions pertaining to the comparability of scores with and without [exam] accommodation…the literature is lacking in quantity of studies, restricted in types of design methodologies and under-representative of the diversity of individuals demonstrating the disorder’ (Gregg & Nelson, 2012, p142).

10. Rationale for current research study Current research study provoked by: • the limited extant empirical research • tension between duty to maintain academic standards and ensure equal opportunity for students with SpLD.

11. Methodology • 714 exam candidates from Summer 2016-18 exams (undergraduates) • 236 SpLD candidates with 25% extra time • 121 SpLD candidates with WP & 25% extra time • 357 Peers who sat same paper under standard conditions • Study > 95% power

12. Sample group: Breakdown Humanities 426 participants: • English (136) • History (186) • Law (104) Total STEM 288 participants: • Maths (152) • Medicine (136) 714

13. Methodology: Group comparisons Humanities only: • total word count on paper • words per minute All candidates: • Exam mark • Exam Classification • Dissertation mark • Dissertation classification • Difference between exam mark & dissertation mark

14. Statistical tests (SPSS) Comparisons: • Word count between groups • Exam mark between groups • Dissertation marks between groups • Dissertation mark & exam mark between groups Correlations: • Time & word count • Use of WP & word count • Word count & mark • Exam mark & dissertation mark

15. Comparison of total mean word count:Humanities subjects

16. Comparison of total mean word count: Humanities participants

17. Comparison of mean words per minute: Humanities candidates

18. Comparison of mean words per minute: All three Humanities subjects combined p< 0.001

19. Comparison of quantity of crossed out words

20. Comparison of mean marks: SpLD group with 25% extra time & TD group Humanities

21. Comparison of mean marks: SpLD group with 25% extra time & use of word-processor & TD group Humanities

22. Comparison of mean marks: SpLD group with 25% extra time & TD group Maths

23. Comparison of mean marks: SpLD group with 25% extra time & TD group Medicine Mean Mark (%)

24. Comparison of classifications Humanities

25. Comparison of classifications Maths

26. Correlation between word count and mark: Humanities

27. Correlation between word count and mark SpLD group

28. Correlation between word count and mark: TD group

29. RESPONSE WORD COUNT MARK Insufficient argument & excessive detail High word count Low Mark (2:2 or below) >5000 3,200 to 4,500 Cogent argument & optimal detail Optimal word count High Mark (2:1 or above) Insufficient argument & insufficient detail Low word count <2000 Low Mark (2:2 or below)

30. Outlier data: Highest word count • Highest word count • 5268 • SpLD candidate • Mark 63% • 14 candidates produced over 5000 • 7 SpLD (5 ET and 2 WP) • 7 TD • Mean mark 60%

31. Outlier data: Lowest word count • Lowest word count • 1316 • TD candidate • Mark 57% • 8 candidates produced word count below 2000 • 7 SpLD (5 WP and 2 ET) • 1 TD • Mean mark 62%

32. Dissertation / project marks Humanities • Mean SpLD mark • 69% • Mean TD mark • 68% No significant differences Maths • Mean SpLD Mark • 66% • Mean TD mark • 65% No significant differences

33. Comparison between exam mark and coursework mark Maths Humanities Mean Mark (%)

34. Correlation between exam mark and coursework mark TD SpLD

35. Validation of findings –data from university student statistics department 2017 (undergraduates)

36. Conclusions • SpLD candidates granted 25% extra time do not produce longer answers or achieve higher marks than their non-SpLD peers • SpLD candidates granted 25% extra time & use of a word processor do not produce longer answers or achieve higher marks than their non-SpLD peers • no correlation observed between word count and mark or degree classification across the participants as a whole • high word count fails to correlate with higher mark • low word count fails to correlate with low marks.

37. Conclusions • Data suggests that combination of use of a word processor with 25% extra time levels the playing field more than 25% extra time alone. • Use of word processor reduces the following barriers in exams • Spelling problems • Working memory • structure

38. Future Research & implications • Future research: • Effect on marks when SpLD identified part-way through course (candidate sat some exams under standard conditions and some with exam arrangements • Impact of study skills on exam results • Qualitative studies • Implications for practice – need to achieve parity • Universal design • Alternative modes of assessment