Partnership for Accessible Reading Assessment (PARA) Technical Advisory Committee Meeting

1. . Partnership for Accessible Reading Assessment (PARA) Technical Advisory Committee Meeting Tuesday, June 28, 2005 Washington, D.C.

2. Instructional Access and Assessment Obstacles for Students with Disabilities Sheryl Lazarus

3. Overview • Review of state reading standards. • How various disabilities may affect the way students access reading. • Analysis of accommodations that have an impact on the assessment of reading.

4. What “Reading” Means to States:Themes Found in State Reading Standards

5. Research Questions • What are the major themes in state literacy standards? • How might these standards be taught or assessed for students with disabilities?

6. Themes: State Academic Content Standards Define Reading As: • Acquisition of specific skills • Knowledge of elements or conventions in language • Interactive thinking activity • Problem-solving tool • Catalyst for personal growth

7. Common Descriptors Found in Themes

8. Research Questions • What is the format of reading assessments currently used in grades 4 and 8? • What standards are and are not assessed?

9. Methods • Review of reading standards in grades 4 and 8, for 5 states. • Review of test specifications in grades 4 and 8, for same 5 states.

10. Findings • States predominately assess student reading by having students read passages and answer questions. • Multiple choice tests require that students are able to decode print in order to comprehend passages. • Not all standards are assessed. Research, interactive, and personal growth standards are typically not assessed in statewide assessments.

11. . Issues that surround the instruction and assessment of reading for students with disabilities

12. Students Receiving Special Education Services Source: Education Week Quality Counts 2004

13. Reading and Students with Visual impairments • Most students with visual impairments are not blind. • Role of tactile (Braille) and auditory methods of accessing text. • Implications for reading.

14. Reading and Students who are Deaf or Hard of Hearing • Communication forms • American Sign Language • Manually Coded English • Lip reading • Cochlear implants • Implications for how students access text.

15. Reading and Students with Autism • Many students with Asperger Syndrome can decode words well, but may lack comprehension skills (Barnhill, 2004). • Students with autism may find it difficult to screen out distractions.

16. Reading and Students with Specific Learning Disabilities • 90% of students with learning disabilities identify reading as their primary difficulty (President’s Commission on Excellence, 2003). • Two instructional approaches most commonly used: • Remediation instruction • Compensatory (assistive) technology

17. Reading and Students with Mental Retardation • Traditionally, special educators have de-emphasized literacy in favor of functional, social or motor skills. • Today, now that students with MR have access to the general curriculum, students are achieving at much higher and more complex levels than expected. • 2 broad categories to teach reading: • Traditional or direct approach • Progressive or holistic approach

18. Reading and Students with Speech or Language Impairments • Since reading is a language based skill, students without strong language skills are at-risk. • Improve the learning environment by: • Seating away from distractions • Monitoring background noises • Establishing a consistent class structure • Speaking slowly and clearly • Using sequential words

19. Reading and Students with Emotional or Behavioral Disorders • No one learning strategy will be effective for every student. • 2 main instructional approaches: • Direct instruction • Peer tutoring • Setting in which instruction is provided will affect reading.

20. What do state accommodations policies say?

21. Accommodation: Read Aloud Directions

22. Accommodation: Read Aloud Directions Examples from State Policies • Allowed except on the reading test • Considered a non-standard accommodation on the reading test

23. Accommodation: Read Aloud Questions

24. Accommodation: Sign Interpret Questions

25. Questions • When we take into consideration the characteristics of students with disabilities, what are some of the limitations of current large-scale assessments? • What are the implications for this project?

26. Definition Panel and Focus Group Work Deborah Dillon

27. Goal 1 • Formulate a definition of reading proficiency, analyze the definition in relation to state standards, obtain input, and refine the definition. • Definition Panel Work: Formulate a working definition of key terms for use in collaborative efforts with other funded projects.

28. Definition Panel Leadership Team • Group co-leaders: Deborah Dillon, University of Minnesota and John Sabatini, ETS • Members: Ann Clapper, NCEO; Laurie Cutting, ETS and Kennedy Krieger Institute; Lee Galda and David O’Brien, University of Minnesota; other colleagues from the University of Minnesota

29. Membership of the Definition Panel (DP) • Peter Afflerbach, Professor, University of Maryland • Donna Alvermann, Professor, University of Georgia • Diane Browder, Professor, University of North Carolina – Charlotte • Donald Deshler, Professor and Director, University of Kansas Center for Research on Learning • Dave Edyburn, Associate Professor, University of Wisconsin – Milwaukee • Russell Gersten, Director, Instructional Research Group • Cay Holbrook, Associate Professor, University of British Columbia • Michael Kamil, Professor, Stanford University • Peggy McCardle, Associate Chief, Child Development and Behavior Branch, National Institute for Child Health and Human Development (NICHD) • Susan Rose, Associate Professor, University of Minnesota • Deborah Simmons, Professor, Texas A&M University • Dorothy Strickland, Professor, Rutgers University • Richard Wagner, Professor and Associate Director, Florida Center for Reading Research, Florida State University • Brenda Jane (BJ) Wiemer, Special Services Department Chair, Kirk Day School • Joanna Williams, Professor, Columbia Teachers College

30. DP Tasks • Prior to Face-to Face Meeting/s—request that panel members review selected reports and identify relevant research literature for a position paper/synthesis document developed to undergird a working definition of reading, reading proficiency, and other related terms. • Face-to-Face Panel Meeting/s—seek panel members’ consensus on a definition of “reading” and “reading proficiency;” draft descriptive statements or claims about the reading proficiencies of “typical” learners in grades 4-8.

31. DP Tasks—cont. • Help to draft a “Position Paper” or “Synthesis Document” —this paper is comprised of a summary of panel meeting minutes, a review of current literature and research reports (e.g., RAND; NAEP Reading Framework for 2009) used to ground and support a definition of “reading” and “reading proficiency,” and explanations of key terms. • Identify issues and points in the document that need clarification. • Finalize the definition based on external feedback from Focus Groups and other review processes.

32. Agenda for Definition Panel Meeting 1/29/05 • Session #1--Develop a robust definition of reading. • Session #2--Discuss associated reading proficiencies implied by the definition for learners in grades 4 & 8. • Session #3--Consider the definition of reading with respect to the full range of students. • Session #4—Identify key concepts and terms. • Session #5--Review and provide initial feedback on questions for focus groups.

33. An Overview of the DP Task Formulate a definition of the construct of “reading proficiency” that can be used as a basis for researchand development for accessible large-scale tests of reading proficiency (consistent with the requirements of the No Child Left Behind Act of 2001--NCLB) that provides:

34. An Overview of the DP Task—cont. a) a valid measure of proficiency against academicstandards; and b) individual interpretive, descriptive, and diagnostic reports for a full range of students with disabilities that affect reading.

35. Session #1—Guiding Questions Tasks: • Draft a definition of reading, possibly combining elements from the 5 report definitions (NRP, RAND, PISA, PIRLS, NAEP-2009) • Talk about key aspects that are critical for a robust definition of reading proficiency. • Identify significant issues or concerns of DP members • What do you feel strongly about that must be included in the definition and why? • If we include particular aspects in the definition, what are the implications? • If we exclude particular aspects in the definition, what are the implications?

36. Session #2: Discuss associated reading proficiencies implied by the definition for learners in grades 4 & 8 Tasks: In 3 small groups of 4-5 DP members share ideas and draft descriptive statements or claims about the reading proficiencies of “typical” learners. • Given the definition of reading that we generated, discuss and draft descriptive statements or claims about the reading proficiencies of “typical” learners in grades 4 & 8. What would these students be able to do as readers? • Identify significant issues/challenges that may require reworking of the initial definition of reading. • Identify significant issues and challenges for the PARA and DARA development teams—what innovations must occur with R&D?

37. Session #3: Consider the definition of reading with respect to the full range of students Tasks: In 3 small groups of 4-5 DP members, discuss the definition of reading and implications for students with disabilities. Reconvene as a whole group; discuss ideas and identify future work. • Given the definition of reading we developed, what are the components that are needed to be proficient in reading and what are the implications for individuals with different disabilities? --At this moment, is our definition inclusive? --Can the definition accommodate students with disabilities? --Where are the weak spots in the definition that need further work? --What might be the implications for assessments?

38. Results of January 2005 DP Meeting • Definition of Reading: Reading is decoding and understanding written text. Decoding requires translating the symbols of writing systems (including Braille) into the spoken words they represent. Understanding is determined by the purposes for reading, the context, the nature of the text, and the reader’s strategies and knowledge.

39. DP Small Group Work on the Definition • A small group was formed to examine the January DP meeting notes, consider the concerns and comments related to disabilities, and reconsider the definition of reading. Result—a revised definition of reading and a draft definition of reading proficiency. • Two other small groups were charged to work on 4th and 8th grade proficiency statements. Result—group members were not able to complete this work until the definition was finalized.

40. DP Large Group Work/Email Discussion re: the Definition • Some concerns were expressed over the revised definition of reading generated by the small group work (Definition B). • DP members voted to return to the definition drafted in January 2005 (Definition A). • The draft definition of reading proficiency (generated by the small group) was placed on hold. • Exec. Committee of NARAP discussed issues and offered a third draft definition of reading (Definition C).

41. Focus Group Goals • Main purpose – to obtain feedback on the definition of reading from members of the reading, disability, and educational measurement community. • Secondary purpose – to get the word out about the NARAP project.

42. Focus Group Process • Face-to-face (DARA): • “Piggyback” on large conferences • Broader constituency of educators • Cost effective, convenient, open to all • Web-based (PARA) • Not tied to specific conferences • Focus on specific disability groups • Will be more targeted by GAC members

43. Targeted Conferences • Learning Disabilities Association of America (LDA)[Cancelled, PARA will do as web-based] • Council for Exceptional Children (CEC)[April – Completed] • American Educational Research Association (AERA)[April – Completed] • International Reading Association (IRA)[May – Completed] • Council of Chief State School Officers (CCSSO)[June 20-21] • Society for the Scientific Study of Reading (SSSR)[June 25]

44. Sample for Initial Focus Group Results Total - 14 sessions, 76 people • CEC - 6 sessions, 35 people • AERA - 3 sessions, 17 people • IRA - 5 sessions, 24 people

45. Definition A • Reading is decoding and understanding written text. Decoding requires translating the symbols of writing systems (including Braille) into the spoken words they represent. Understanding is determined by the purposes for reading, the context, the nature of the text, and the readers’ strategies and knowledge.

46. Definition B • Reading is decoding and understanding text for particular reader purposes. Readers decode written text by translating text to speech, and translating directly to meaning. To understand written text, readers engage in constructive processes to make text meaningful, which is the end goal or product.

47. Definition C • Reading is the process of deriving meaning from text. For the majority of readers, this process involves decoding written text. Some individuals require adaptations such as Braille or auditorization to support the decoding process. Understanding text is determined by the purposes for reading, the context, the nature of the text, and the readers’ strategies and knowledge.

48. Questionnaire/ Focus Group Questions • How well does the definition define reading? • How well does the definition define reading for students with disabilities that affect reading? • How well will the definition support the development of a fair assessment for all students?

49. Focus Groups: General Reaction • Understanding should be the main focus of the definition. • Decoding over-emphasized. • not as important as understanding • one facet of larger process • References to “spoken words” and “speech” problematic for Deaf/Hearing Impaired. • Braille as “text” for Blind/Visually Impaired acceptable – concerns about auditorization.

50. Focus Groups: General Reaction- cont. • Assorted issues: • Understanding vs. Comprehension vs. Constructing/Deriving Meaning • Specific lists vs. generalities • Text-- how broadly defined • Reading vs. Literacy • Disconnect between process and product for many disabled students • ELL students not addressed