Assessment for Instruction in Reading in Grades 3-12 Joseph Torgesen and Yaacov Petscher

1. Assessment for Instruction in Reading in Grades 3-12 Joseph Torgesen and Yaacov Petscher Florida Center for Reading Research International Dyslexia Association, November, 2009

2. Purpose of this presentation: To describe Florida’s new 3-12 assessments for instruction in the context larger issues about formative assessments Three questions/issues will be discussed What are the purposes of each of the assessments within the FAIR system? What is the nature of each of the assessments? How does the Fair system fit within the larger context of a comprehensive formative assessment plan to improve outcomes in adolescent literacy?

3. Torgesen, J. K., & Miller, D. H. (2009). Assessments to guide adolescent literacy instruction. Portsmouth, NH: RMC Research Corporation, Center on Instruction. www. centeroninstruction.org Click on Reading Section

4. Three important purposes for assessment in reading with adolescents 1. How successful are schools in helping all students meet grade level standards in literacy? 2. Monitor student growth during the year to help guide instructional adjustments within classrooms or instructional groups 3. Identify students who may need special interventions to accelerate growth toward grade level standards

5. Three important purposes for assessment in reading with adolescents 1. How successful are schools in helping all students meet grade level standards in literacy? 2. Monitor student growth during the year to help guide instructional adjustments within classrooms or instructional groups 3. Identify students who may need special interventions to accelerate growth toward grade level standards

6. The FAIR system for grades 3-12 Broad Screen/Progress Monitoring Tool Reading Comprehension Task 3 times a year If Necessary Targeted Diagnostic Inventory Maze and Word Analysis Tasks Ongoing Progress Monitoring Diagnostic Toolkit As needed As needed

7. Brief Description of tests in the FAIR system Broad Screen/Progress monitoring tool. Is a computer-adaptive, multiple choice test of reading comprehension designed to assess grade level literacy skills in a manner similar to the FCAT. Maze test – a computer administered maze test to assess reading efficiency and low level comprehension Word analysis test – a computer-adaptive measure of student knowledge of the phonemic, orthographic, and morphological knowledge necessary for reading words accurately in text Informal assessment toolkit- Informal diagnostic instruments

8. Instructional questions the FAIR can answer Which students may require additional reading instruction (i.e. some type of intervention) because they currently have a low probability of success on the FCAT at their grade level? – Reading comprehension test Of students needing some type of intervention, which have relatively serious or pervasive problems with fundamental reading skills? Maze and WA test Of students needing some type of intervention, which might profit from relatively less intensive interventions that focus primarily on comprehension skills (strategies, knowledge, vocabulary) Maze and WA test

9. Instructional questions the FAIR can answer What specific aspects of basic reading skills or comprehension are students struggling with? Informal assessment toolkit Are students who receive intensive interventions in basic reading skills making acceptable progress in those skills in response to the instruction they are receiving? Maze and WA tests Are interventions that focus exclusively on improving higher level reading skills effective in increasing students’ reading skills and their probability of success on the FCAT at the end of the year? Reading comprehension test Are students who do not receive interventions making acceptable progress toward grade level standards in reading throughout the year? Reading comprehension test

10. If Necessary Targeted Diagnostic Inventory Maze and Word Analysis Tasks Ongoing Progress Monitoring Diagnostic Toolkit As needed As needed The FAIR system for grades 3-12 Broad Screen/Progress Monitoring Tool Reading Comprehension Task 3 times a year

11. The Broad Screen/Progress Monitoring Tool: The RC Task • 10-30 minute computer-adaptive task of reading comprehension • A student receives a minimum of one passage and a maximum of three passages-7 to 9 questions per • Predicts student’s FCAT success probability (FSP) • Administered to students performing below grade level on FCAT in previous year (levels 1 and 2) • Administered to 3rd grade students identified with a reading deficiency by their districts • Other students may take the assessment at the discretion of their districts • In addition to FSP, provides standard scores based on Florida norms

15. The Maze Task • Two 3-minute (grade-level) passages • Administered 3 times a year • Student responds to embedded cloze items within text • Assesses text reading efficiency (i.e., reading accuracy and speed, and gist - level comprehension) • Provides raw scores and standard scores based on Florida norms

19. TDI: The Word Analysis Task • A 5-15 minute computer-adaptive spelling task that assesses a student’s understanding of letter/sound correspondence, orthographic conventions, and morphology • A student receives a minimum of 5 words and a maximum of 30 words • Results indicate whether probability of success in reading is hampered by difficulties with word-level skills. • Provides raw score and standard scores based on Florida norms

22. All FAIR assessments are intended to be formative (help guide instructional decisions), rather than summative (evaluate instructional and learning success) A definition from Black and William (2007) “An assessment is formative to the extent that information from the assessment is fed back within the system and actually used to improve the performance of the system in some way.” (p. 31)

23. What is assessment for instruction? There is a lot of confusion in the literature about what types of assessments can be called formative, because there are many different kinds of instructional decisions to be made Does this student need to be in an intervention class? What kind of intervention class should be provided? Is this student’s current intervention placement strong enough to help him close the gap in reading? Does this student need more instruction/practice in using the prediction strategy to improve comprehension? What specific aspects of writing does this study need more instruction in?

24. What is assessment for instruction? There is a lot of confusion in the literature about what types of assessments can be called formative, because there are many different kinds of instructional decisions to be made Does this student need to be in an intervention class? What kind of intervention class should be provided? Is this student’s current intervention placement strong enough to help him close the gap in reading? Does this student need more instruction/practice in using the prediction strategy to improve comprehension? What specific aspects of writing does this study need more instruction in?

25. What types of formative assessments are currently in use? Screening tests– what are initial student abilities in critical areas related to the outcome of interest– intervention or no intervention, initial placement, etc. RC, Maze, WA General outcome progress monitoring/benchmark – is student making acceptable progress in critical outcomes? – change class, change teacher, more time, etc. – typically no details about why student is having problems, RC, Maze, WA Formal diagnostic assessments –in depth and reliable information about skill and knowledge levels in critical areas – similar to screening tests in types of decisions they inform

26. What types of formative assessments are currently in use? Informal diagnostic assessments – what specific phonics skills has the student mastered, what kind of questions can they answer from grade level text? – can be used for initial lesson planning. Informal assessment toolkit Classroom based formative assessments – what strategies is the student using? What are the problems with making correct inferences? What parts of current objectives has the student mastered? – information to guide ongoing lesson planning and instructional adjustments by the teacher Mastery assessments within CBM are a more formalized subset of classroom based assessments. Typically provided within well specified curriculum, and assess mastery of steps in complex tasks

27. Evidence/rationale for investment in formative assessments “The need for accurate assessments arises because of the enorrmous diversity in the rate of learning and level of literacy skills among adolescents.” (Torgesen & Miller, 2009) When there is great diversity among students in their rate of learning and level of literacy… “little variation in teaching will always result in great variation in student learning.” Formative assessments are necessary to help teachers provide instruction that is targeted and paced appropriately

28. Evidence/rationale for investment in formative assessments – how can you tell when you have a good system of formative assessment? Traditional ways of evaluating tests include examinations of their reliability and validity Can you have a reliable and valid test of reading comprehension that does not provide information useful for guiding instruction? It depends on the instructional question you want to answer–not all tests provide information relevant to all important instructional decisions

29. Evidence/rationale for investment in formative assessments The critical questions for “formative” tests are: 1) what kind of information do they provide, and 2) what is that information useful for? The most important outcome from formative assessments is improvement in student performance – thus, the ultimate test of their utility can only be made as they are integrated within a system of instructional response.

30. Evidence/rationale for investment in formative assessments “It is impossible to disentangle the impact of formative assessment from the instruction that follows it. When improved student outcomes are used as the criterion for effective practices, evaluations of classroom-based formative assessment are as much an evaluation of the instructional adjustments resulting from the assessments as they are of the assessments themselves.” (Torgesen & Miller, 2009)

31. Evidence/rationale for investment in formative assessments Two research summaries of interest Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Educational Assessment: Principals, Policy and Practice, 5, 7–74. A comprehensive review of classroom-based formative assessment. Effect sizes from .4 to .7 Fuchs, L. S., & Fuchs, D. (1986). Effects of systematic formative evaluation: A meta-analysis. Exceptional Children, 53, 199–208. A look at very frequent (2-5 times a week) progress monitoring. ES of .92 when teachers required to follow up in specific ways, .42 when follow-up actions left to teacher judgement

32. The FAIR system in this context: Screening/PM tool (reading comprehension test)– screening and general outcome progress monitoring no more frequent than once a month Maze test – screening and general outcome progress monitoring no more frequent than once a month Word analysis test – screening and general outcome progress monitoring no more frequent than three times a year– also can be used diagnostically through error analysis Informal assessment toolkit- Informal diagnostic instruments

33. Progress Assessments Standards based accountability measures End of year Throughout the year End of year “high stakes” test • Classroom Based Formative Assessment including CBA Mastery assessment • 2. General Outcome CBM Screening and informal diagnostics Beginning of year 1. General Screening 2. Targeted Screening 3. informal diagnostics At this time, the only evidence to support formative assessments with older students concern classroom based formative assessments and very frequent general outcome progress monitoring with special needs students.

34. Questions/Discussion