University of Memphis

1. Innovations in the Assessment of reading fluencyElizabeth Meisinger, Ph.D.Katherine Price, M.S.Tera Traylor, B.A. University of Memphis

2. What is Reading Fluency? • The ability to read quickly, accurately, and with appropriate expression (NRP, 2000). • Typically focus on large blocks of text , but you can define more narrowly at letter, word, or phrase level (Kuhn & Stahl, 2004). • Oral versus silent reading

3. Why is reading fluency important ? • Supports comprehension in early elementary school. • Automaticity theory (LaBerge & Samuels, 1974) • By fourth grade children shift from learning to readtoreading for the purpose of learning content area knowledge (Chall, 1996).

4. Relevance to School Psychology • Response to Intervention (RTI) • CBM Oral reading fluency probes • Conceptualized as a general indicator of reading competency (Fuchs et al., 2001). • Specific Learning Disability in Reading Fluency • Subtype added in reauthorization of Individuals with Disabilities Education Improvement Act (IDEIA, 2004).

5. Overview • Brief review of oral reading fluency measures • Review understudied aspects of fluency • Prosody • Miscues • Silent reading • Psychometric properties • Utility in practice of school psychology

6. Assessments of Oral Reading Fluency • Criterion-referenced tests • CBM (Localized, DIBELS, AIMSweb) • Informal Reading Inventories (ex. QRI-4) • Norm-referenced tests • Gray Oral Reading Test, 4th Ed. (GORT-4) • Text level • Test of Word Reading Efficiency (TOWRE) • Word level • Kaufman Test of Educational Achievement, 2nd Ed., (KTEA-2) • Word level • Wechsler Individual Achievement Test, 3rd Ed. (WIAT-III) • Text and prosody

7. Prosody • Appropriate intonation or expressiveness coupled with phrasing that allows for the maintenance of meaning (Miller &Schwanenflugel, 2006, 2008). • Ex. Jake’s going to the dance with Jenifer. • Often viewed by teachers as important for reading comprehension.

8. Prosodic Measures: Rating Scales • National Assessment of Educational Progress (NAEP) oral Reading Fluency Scale (Pinnell et al., 1995; Daane et al., 2005) • 4 point scale • primarily word-by-word with occasional 2- or 3-word phrases; • primarily 2-word phrases with some 3- or 4-word phrases; • Primarily 3- and 4-word phrase groups with little expression; • Expressive reading in larger, meaningful phrases with few regressions, repetitions, or deviations. • Inter-rater agreement 81% (exact) to 100% (adjacent) (Daane et al., 2005)

9. Rating Scales cont. • Multidimensional Fluency Scale (Raskinski et al., 2009; Zutell&Raskinski, 1991) • 3 subscales, 1-4 rating • Phrasing and expression • Smoothness and accuracy • Pacing • Inter-rater agreement 86%-99% • Test-retest reliability .90 • NAEP & MDFS correlate moderately with silent reading comprehension

10. Spectrographic Analysis(Kuhn, Schwanenflugel, &Meisinger, in press) • Digital recordings of readings • Comprehensive speech software programs • Allows for specific analysis of a variety of prosodic features: • Pitch, stress, duration, & pausing

11. Prosody’ s Role in Reading Development(Kuhn, Schwanenflugel, &Meisinger, in press) • Prosody is related to the development of fluent reading (fast, accurate text reading). • The relation between prosody and comprehension is less clear. • Does comprehension lead to prosody? • Does prosody support comprehension?? • Is the relation reciprocal???

12. Future Implications for Practice (Kuhn, Schwanenflugel, &Meisinger, in press) • Compare spectrographic measures and rating scales to determine reliable, valid assessments for classroom use. • Integrate prosody into assessments of reading fluency. • Psychoeducational reports • Progress monitoring • Address prosody in reading fluency interventions (if prosody contributes to comprehension).

13. Oral reading accuracy • Word reading accuracy is important component of reading fluency. • Readers who have difficulty pronouncing individual words experiencing difficulties deriving meaning from text (Harris & Hodges, 1995). • Fluent readers translate words quickly and easily, freeing attention for comprehension (LaBerge& Samuels, 1974). • Difficulties with word recognition is most pervasive cause of reading disability (Perfetti, 1985; Stanovich, 1986; Vellutino, Fletcher, Snowling, & Scanlon, 2004).

14. Miscue Analysis • The Reading Wars and Kenneth Goodman • Miscues are oral reading errors: a point in reading where the observed response does not equal the actual response (Goodman, 1973). • Errors provide valuable information about how the reader is engaging with the text. • Examines reading errors in context (whole passage) • Text: She put down her foot. • Child reads: “She put down her food.”

15. Miscue Analysis • Assumes that children use “ 3 cueing system” to help them construct meaning from the text: semantic, syntactic, and graphophonic cues • Goal of analyzing miscues is to determine if reader is using all 3 cues to help them identify words • Errors coded using a taxonomy • Reading Miscue Inventory (Goodman, Watson, & Burke, 2005) • Qualitative information can be collected and compared across time.

16. Miscue Analysis • Types of miscues commonly recorded and coded • Omissions • Insertions • Self-corrections • Repetitions • Substitutions • Grapho-phonically similar/dissimilar • Semantically similar/dissimilar • Syntactically similar/dissimilar

17. Excerpted from Reading Miscue Inventory (Goodman, Watson, & Burke, 2005)

18. Classroom Implications • Teachers exposed to miscue analysis in education programs • Some IRIs and tests of reading ability contain miscue analysis procedures (e.g., Qualitative Reading Inventory-4&GORT-4). • Running records use simplified miscue analysis procedures. • Instruction can be tailored to pattern of miscues • Reading Recovery

19. Research • NAEP Special Study of Oral Reading-- 4th grade students (Daane et al., 2005) • Students who committed fewest errors demonstrated better comprehension scores (NAEP reading assessment). • Oral reading errors, regardless of their effect on text meaning, were negatively related to comprehension. • A positive relationship exists between the proportion of errors self-corrected and comprehension score.

20. Research • Substitutions are the most common types of miscues, followed by omissions, then insertions (Goodman, 1976; D’Angelo&Mahlios, 1983). • The percentage of miscues acceptable within the context of the text increases as a function of grade level and reading ability (Christie, 1981). • Omissions and substitutions of phonologically similar words are best predictors of comprehension (Laing, 2002).

21. Research • Insertions and omissions cause few syntactic and semantic distortions, and do not interfere with comprehension (D’Angelo&Mahlios, 1983). • Specific miscue types are not significantly correlated with reading comprehension (Englert&Semmel, 1981).

22. Some Considerations • No empirical support exists for 3 cueing system • Procedure is complicated and time-consuming • Analysis is subjective: coding may vary depending on passage characteristics and judge • Little is known about how teachers actually use miscue analysis in the classroom • Suggestions for modified use (McKenna & Picard, 2006)

23. Future Research • More information needed regarding types of miscues related to comprehension in early grades • Does miscue analysis provide enough bang for the buck? • Teacher use of miscue analysis in the classroom

24. Implications for Practice • Qualitative writing samples may be used to supplement reading assessment • However, little empirical support for how specific miscues relate to comprehension • Familiarity with teacher assessment practices can facilitate better communication

25. Why silent reading fluency? • Oral reading fluency is often conceptualized as a proxy for general reading competence due to its close relation to comprehension in early elementary school. • i.e., can screen for general reading skill quickly with ORF probes • This relation decreases across development (Vellutino et al., 2004). • Further, children typically transition to primarily silent reading by fifth grade, suggesting that it may become important to utilize silent reading assessments in older children.

26. Silent Reading Assessment • Range of traditional and emerging assessment options: • paper-and-pencil • group administered sentence reading • computerized tasks • Merits and detractions of each type of assessment • Brief overview of areas in silent reading that currently lack strong empirical examination.

27. Paper-and-pencil Assessment • Most common and familiar • Typically involves the reader silently reading from a passage and marking or circling the last word read when the administrator calls time (often after 1 minute) (Fuchs et al., 2001) • Advantages • Familiar and ecologically valid • Inexpensive and easy • Group or individual administration • Flexible, brief • Can be curriculum-based • Disadvantages • Self-report--Is the kid actually reading? • Dearth of information

28. Paper-and-pencil Assessment: Psychometrics • Relation to Comprehension: • .38 (questions tied to passages) (Fuchs, et al., 2001) • .47 (ITBS reading comp) (Fuchs, et al., 2001) • .48 (GMRT) • .42 (Maze task) • Relation to Other Measures of Silent Reading: • Self-paced reading: .61 • Underlining: .58 • Obtained Reliability: • .86 (alternate form)

29. Test of Silent Reading Efficiency and Comprehension (TOSREC) • Recently released and available through Pro-Ed • (though it’s on backorder) • Assesses silent reading of connected text for comprehension • Student reads a series of sentences and circles the answers to comprehension questions following each sentence; the test is administered in 3 minutes • Advantages • Commercially available • Multiple forms • Nationally normed for Fall, Winter, and Spring or progress monitoring • Group or individually administered • Ecologically valid • Comprehension measure • Brief • Disadvantages • Comprehension is directly tied to the score • The test contains sentences instead of longer passages

30. Computerized Measures • Self-pacedreading methods • Based on the assumption that a participant will read at a rate that matches the comprehension process • Analysis of reading rate will reveal information about comprehension and the reading process • Three main categories: • Eye-tracking • Window Methods (not talking about today) • Underlining

31. Eye-Tracking • Text is exposed on a screen and the reader’s eye movements are tracked by computer software • Movements can be sampled up to 1000 times per second • Allows for an in-depth analysis of eye fixations • Where, How long (gaze length), Regressions, how many times the reader fixates on an individual word, etc. • No comparable psychometrics information available • Advantages • Large amount of in-depth information • Intra-word analyses • More analyses available—i.e., silent reading prosody, • Flexible (can be brief) • Disadvantages • Expensive to set up • Some set ups may be difficult to use with children especially in a school setting • Individually administered

32. Underlining • Combines elements of paper-and-pencil and eye-tracking • Text is read from the screen of a tablet PC, and the reader is asked to underline the text “on-line” with his or her reading • Underline each word as they read it, pause their underlining if they pause in their reading, go back and re-underline any regressions • Specialized software is used to track the location of the mouse on the screen • Allows for a deeper analysis than do paper-and-pencil assessments, without some of the disadvantages of eye-tracking • Advantages: • Less expensive than eye-tracking • Ecologically valid • Can be group administered • Greater amount of information • Brief, flexible • Disadvantages: • No intra-word information • No normative information • More expensive than paper-and-pencil methods

33. Example of an Adult with Regression

34. Example of 4th Grade Student’s Passage

35. Underlining:Psychometrics • Relation to comprehension: • 4th: GMRT: .41 Maze task: .53 • 6th: GMRT: .47 Maze task: .37 • Relation to other measures of silent reading: • 4th: SPR: .49 Paper-and-Pencil: .59 • 6th: SPR: .59 Paper-and-Pencil: .83 • Obtained Reliability: • .96 (alternate form)

37. Applications in Practice • Assessment of silent reading in older grades as opposed to oral reading • May make more sense after 4thgrade due todecreased ties to comprehension in older grades • Could be used in lieu of ORF as a brief measure of general reading competence for progress monitoring or screening • More choices available beyond paper-and-pencil assessment depending on what information you are seeking out of your assessment

38. Areas for Future Research • When does the shift occur? • How is this shift moderated by reading ability? • How do traditional and emerging silent reading measures compare to eye-tracking measures? • Seen in the literature as the gold standard for information about reading processes • How does the theoretical model of silent reading development differ from those of oral reading development?

39. Questions? Comments? bmsinger@memphis.edu