Partnership for Accessible Reading Assessment (PARA) Research

1. Partnership for Accessible Reading Assessment (PARA) Research Martha Thurlow National Center on Educational Outcomes University of Minnesota

2. Research Program Assumptions Assumption #1:We do not know everything about what goes into accessible reading assessment yet. Assumption #2:Preliminary research must inform design of accessible assessment. Assumption #3:Both preliminary and experimental research will inform development of Principles and Guidelines for future assessments.

3. Three Major Research Question Areas • What characteristics of current assessment practices hinder accessibility? • What characteristics of students require more accessible assessment of reading? • What characteristics would accessible assessment have?

4. Two Research Phases Study Phases Current Assessment Characteristics Accessible Assessment Characteristics Student Characteristics Exploration Confirmation

5. Sample of Studies Study Phases Current Assessment Characteristics Accessible Assessment Characteristics Student Characteristics State Documents Exploration Confirmation State Documents Analysis – Identifying what is assessed in reading relative to state standards (via test specifications), analyzing accommodation policies for reading assessments, and examining current test forms for characteristics that promote and hinder accessibility.

6. Mean Percent of Items by Construct (across states)

7. Sample of Studies Study Phases Current Assessment Characteristics Accessible Assessment Characteristics Student Characteristics Literature+ Exploration Confirmation Literature Scans+ – Literature and Web site reviews, as well as input and feedback from organizations and experts to describe disabilities, general approaches to reading instruction, and assessment of reading for those disabilities that affect reading.

8. Sample of Studies Study Phases Current Assessment Characteristics Accessible Assessment Characteristics Student Characteristics Expert Review Exploration Confirmation Expert Review – Using a variety of procedures (focus group, MACB, Delphi), select accessible assessment practices based on likelihood of effectiveness (improve performance), validity (retain reading construct integrity), feasibility (practical to implement)

9. Sample of Studies Study Phases Current Assessment Characteristics Accessible Assessment Characteristics Student Characteristics Exploration “Pictures” Study Confirmation Pictures Study – “Pictures” or other interest items (decorative graphics, icons, etc.) that are not needed to answer test items are randomly assigned to control and experimental groups to confirm which practices differentially suppress performance.

10. Sample of Studies Study Phases Current Assessment Characteristics Accessible Assessment Characteristics Student Characteristics Exploration Nomination + Scrutiny Confirmation Nomination + Scrutiny Study – Information gathered from exploration phase used to solicit teacher nominations of students who are poorly assessed under current assessment practices. Students will be observed, individual assessment data examined, and think aloud procedures used to confirm the need for accessible assessment.

11. Reasons For Inadequate Measurement of Student’s Reading Knowledge and Skills • Limited reading fluency obscures comprehension capabilities. • Difficulty with sophisticated comprehension obscures basic reading skills. • Has strengths that typical tests do not cover. • Responds poorly to standardized testing circumstances or materials.

12. Sample of Studies Study Phases Current Assessment Characteristics Accessible Assessment Characteristics Student Characteristics Exploration Differential Boost Study Confirmation Differential Boost Study – Conduct experiments with students (those identified from Nomination Study as inaccurately assessed by current state tests) using random assignment to experimental and control groups to confirm which practices (e.g., no interest graphics, motivation) differentially enhance (boost) performance.