Accommodations in Testing: A National Perspective

1. Accommodations in Testing: A National Perspective Blue Ribbon Panel Meeting June, 2002

2. Overview • The Context: Participation of Students with Disabilities in Standards-Based Reform • What Other States are Doing Now • Predictions About the Future • Identify Key Resources for Committee While Addressing Three Topics Above

3. Standards-Based Reform • Important Resource: National Academy of Science/National Research Council Reports • Why pay special attention? • Participants • First two studies • Process • Report review • Role of sponsor

5. Access to Report • Read it online for free at: • http://www.nap.edu/catalog/5788.html • Order from National Academy Press (www.nap.edu)

6. Standards-Based Reform • Definition • Improve education by setting high content standards that define what should be taught and learned, and by holding educators and students accountable for attaining ambitious performance standards that define expected proficiency.

7. Reform • Four common elements: • 1. Student achievement is primary measure of school success. • 2. Adopt challenging content and performance standards. • 3. Extend standards to all students, including those for whom expectations have been low historically. • 4. Use assessment to spur reform and monitor its impact.

8. Reform • Impact on Students with Disabilities • Profound • Until recently, legal framework under which students with disabilities have been educated emphasized: • Individualized goals for instruction rather than common standards. • Accountability was for procedural compliance rather than student achievement outcomes.

9. Reform • Impact Has Been A Mixed Blessing • Positives • Raised level of expectations. • Accountability for outcomes extended to both regular and special educators. • Greater access to rich curriculum. • It’s where the action and dollars are.

10. Reform • Negatives • Existing standards limited to subject matter content to exclusion of vocational and independent living skills. • A diverse group, but in general, students with disabilities fare worse with respect to achievement, graduation, enrollment in postsecondary education, and employment. • Accountability consequences of failing to achieve are likely to fall disproportionately on students with disabilities and the educators who serve them.

11. A Clear Psychometric Definition • An accommodation represents an alteration to standard test conditions that neutralizes extraneous sources of difficulty that result from an interaction between standard administration and the student’s disability while preserving the measurement goals of the test.

12. The Bad News • “However, research on alterations of assessments for elementary and secondary students is extremely sparse and provides only limited guidance for policy makers and educators.” (Educating One and All, p. 171)

13. Effects of Extra Time on Timed Test • Studies of Scholastic Aptitude Test (SAT) and Graduate Record Exam (GRE) • Although some disabilities and accommodations slow down examinee, for these speeded tests, individuals with disabilities did not require more time on average. • Reason may be they ran into ceiling—items too difficult to answer.

14. Effects of Extra Time on Timed Test • Providing additional time on these speeded tests reduced validity to some degree, due to overprediction for relatively high-scoring students with learning disabilities. • Best solution is to provide an “accommodated test” to all by giving everyone a generous amount of time (switching from “speeded” to “power” tests).

15. NRC Report • Recommendations: • 1. Content standards, performance standards, and assessments should be designed to maximize participation of students with disabilities. • 2. Presumption should be that each student participates in common standards and assessments; alterations must have a compelling justification and be made on an individual basis.

16. NRC Report • Recommendations (cont.) • 3. States should revise policies that discourage maximum participation of students with disabilities in common accountability system, and should provide incentives to encourage maximum participation.

17. NRC Report • Recommendations (cont.) • 4. When standards or assessments are altered for a student with disabilities: • The alternate should be challenging yet potentially achievable. • It must reflect the full knowledge and skills necessary to live a productive life. • Parents should be involved in these decisions, and informed of any consequences of alterations.

18. NRC Report • Recommendations (cont.) • 5. Assessment accommodations should be used only to offset the impact of disabilities unrelated to the knowledge and skills being measured. • 6. Even if accommodations or alternate assessments are needed, all students should be counted in a universal, public accountability system.

19. What Other States Are Doing Now • Key resource: National Center on Educational Outcomes (NCEO) • http://education.umn.edu/NCEO/ • Charged with providing national leadership on the participation of students with disabilities in national and state assessments, standards-setting efforts, and graduation requirements.

20. Other States • Two useful publications: • 2001 State Special Education Outcomes: A Report on State Activities at the Beginning of a New Decade. • http://education.umn.edu/NCEO/OnlinePubs/2001StateReport.html • A Self-Study Guide to Implementation of Inclusive Assessment and Accountability Systems: A Best Practice Approach • http://education.umn.edu/NCEO/OnlinePubs/workbook.pdf

21. High Stakes Testing • Student Accountability – students are held responsible and consequences are assigned to them (e.g., must pass test to graduate or move to next grade) 20 States • System Accountability – educators, schools, or districts are held responsible and consequences are assigned to them (e.g., schools rated according to test scores, teachers receive rewards for student performance) 38 States

22. Florida is a leader in being one of 10 states that do not routinely exclude students from assessment participation. Most frequent reasons allowed for exclusion in other states are limited English proficiency, parent refusal, or judgment of student’s IEP team. Participation Requirements

23. 14 states disaggregated data on the participation of students with disabilities 17 states disaggregated data on the performance of students with disabilities

24. 2001 State Directors Told Us: All Students with Disabilities are Included in All Components of the Accountability System in 25 States

25. Accommodation Use • Is on the rise • About 50% of the LD students accommodated • Most common accommodations are: • small group administration • read-aloud • extended time

26. Alternate Assessment A substitute way of gathering information about the performance and progress of students who cannot not participate in typical state assessments even with accommodations.

27. Alternate Assessments • Are used in place of general state and district wide assessments • Serve as an index of student progress toward meeting standards held for all students

28. Alternate Assessments are Performance Based • Data are collected through: • Observation • Recollection (checklist/interview) • Record Review • Testing (Performance events)

29. Alternate Assessment Approaches Selected by States • Portfolio/body of evidence 28 states • Checklist/Rating scale 4 states • IEP analysis 5 states • Other 6 states • Uncertain or not reported 7 states

30. Out of Level Testing • Addresses problem that many current assessments do not provide reliable or valid measurement of gains for students who are well below high proficiency. • Stop-gap solution to problem of no data available on whether very low students are improving. • Use of out of level testing is increasing rapidly.

31. Predictions About Future • Common problems: • 1. Current assessments do not provide useful information about growth of very low performers.

32. Less Precision in the Tails

33. Predictions (cont.) • 2. Extensive research is not available on effects of accommodations on the validity of assessments, nor of the comparability of alternative assessments or of out-of-level assessments.

34. Predictions About Future • Solution to common problems requires investing in a new generation of assessments that reduces the need for accommodations. • Important advances in psychometrics (the science of measurement) and in the cognitive science of learning point the way to the next generation of assessment.

36. Access to Report • Read it online for free at: • http://www.nap.edu/catalog/10019.html • Order from National Academy Press (www.nap.edu)

37. Two Last Resources • Me with more time to prepare. • Florida Center for Reading Research.