Review of RTI Literacy Assessment/Monitoring Tools

A Review of RTI Literacy Assessment/ Monitoring ToolsJim Wrightwww.interventioncentral.org

RTI Literacy: Assessment & Progress-Monitoring (Cont.) To measure student ‘response to instruction/intervention’ effectively, the RTI Literacy model measures students’ reading performance and progress on schedules matched to each student’s risk profile and intervention Tier membership. • Benchmarking/Universal Screening. All children in a grade level are assessed at least 3 times per year on a common collection of literacy assessments. • Strategic Monitoring. Students placed in Tier 2 (supplemental) reading groups are assessed 1-2 times per month to gauge their progress with this intervention. • Intensive Monitoring. Students who participate in an intensive, individualized Tier 3 reading intervention are assessed at least once per week. Source: Burns, M. K., & Gibbons, K. A. (2008). Implementing response-to-intervention in elementary and secondary schools: Procedures to assure scientific-based practices. New York: Routledge.

Curriculum-Based Measurement: Advantages as a Set of Tools to Monitor RTI/Academic Cases • Aligns with curriculum-goals and materials • Is reliable and valid (has ‘technical adequacy’) • Is criterion-referenced: sets specific performance levels for specific tasks • Uses standard procedures to prepare materials, administer, and score • Samples student performance to give objective, observable ‘low-inference’ information about student performance • Has decision rules to help educators to interpret student data and make appropriate instructional decisions • Is efficient to implement in schools (e.g., training can be done quickly; the measures are brief and feasible for classrooms, etc.) • Provides data that can be converted into visual displays for ease of communication Source: Hosp, M.K., Hosp, J. L., & Howell, K. W. (2007). The ABCs of CBM. New York: Guilford.

CBM Literacy Measures: Sources • DIBELS (https://dibels.uoregon.edu/) • AimsWeb (http://www.aimsweb.com) • Easy CBM (http://www.easycbm.com) • iSteep (http://www.isteep.com) • EdCheckup (http://www.edcheckup.com) • Intervention Central (http://www.interventioncentral.org)

Reading: 5 Big Ideas • Phonemic Awareness/Specific Subskill Mastery • Alphabetics • Fluency with Text • Vocabulary • Comprehension

Initial Sound Fluency (ISF) • “standardized, individually administered measure of phonological awareness that assesses a child’s ability to recognize and produce the initial sound in an orally presented word. The examiner presents four pictures to the child, names each picture, and then asks the child to identify (i.e., point to or say) the picture that begins with the sound produced orally by the examiner. • Time: About 3 minutes SOURCE: Good et al. (2002) DIBELS administration and scoring guide. https://dibels.uoregon.edu/measures/files/admin_and_scoring_6th_ed.pdf

Reading: 5 Big Ideas • Phonemic Awareness/Specific Subskill Mastery • Alphabetics • Fluency with Text • Vocabulary • Comprehension

Phoneme Segmentation Fluency (PSF) • “assesses a student’s ability to segment three- and four-phoneme words into their individual phonemes fluently. The PSF task is administered by the examiner orally presenting words of three to four phonemes. It requires the student to produce verbally the individual phonemes for each word.” • Time: 1 minute SOURCE: Good et al. (2002) DIBELS administration and scoring guide. https://dibels.uoregon.edu/measures/files/admin_and_scoring_6th_ed.pdf

Reading: 5 Big Ideas • Phonemic Awareness • Alphabetics/Specific Subskill Mastery • Fluency with Text • Vocabulary • Comprehension

Letter Naming Fluency (LNF) • “Students are presented with a page of upper- and lower-case letters arranged in a random order and are asked to name as many letters as they can.” • Time: 1 minute SOURCE: Good et al. (2002) DIBELS administration and scoring guide. https://dibels.uoregon.edu/measures/files/admin_and_scoring_6th_ed.pdf

Nonsense Word Fluency (NWF) • Tests the “alphabetic principle – including letter-sound correspondence and of the ability to blend letters into words in which letters represent their most common sounds. The student is presented a sheet of paper with randomly ordered VC and CVC nonsense words (e.g., sig, rav, ov) and asked to produce verbally the individual letter sound of each letter or verbally produce, or read, the whole nonsense word.” • Time: 1 minute SOURCE: Good et al. (2002) DIBELS administration and scoring guide. https://dibels.uoregon.edu/measures/files/admin_and_scoring_6th_ed.pdf

Reading: 5 Big Ideas • Phonemic Awareness • Alphabetics • Fluency with Text/General Outcome Measure • Vocabulary • Comprehension

Oral Reading Fluency (ORF) • “Student performance is measured by having students read a passage aloud for one minute. Words omitted, substituted, and hesitations of more than three seconds are scored as errors. Words self-corrected within three seconds are scored as accurate. The number of correct words per minute from the passage is the oral reading fluency rate.” • Time: 1 minute SOURCE: Good et al. (2002) DIBELS administration and scoring guide. https://dibels.uoregon.edu/measures/files/admin_and_scoring_6th_ed.pdf

Reading: 5 Big Ideas • Phonemic Awareness • Alphabetics • Fluency with Text • Vocabulary • Comprehension/General Outcome Measure

Comparison of RTI Assessment/Monitoring Systems DIBELS [Dynamic Indicators of Basic Early Literacy Skills] • Initial Sound Fluency: Preschool > Middle K • Letter Naming Fluency: Beginning K > Beginning Gr 1 • Phoneme Segmentation Fluency: Middle K > End Gr 1 • Nonsense Word Fluency: Middle K > Beginning Gr 2 • Oral Reading Fluency: Middle Gr 1 > Gr 6

Comparison of RTI Assessment/Monitoring Systems Easy CBM • Letter Naming Fluency: K > Gr 1 • Letter Sound Fluency: K > Gr 1 • Phoneme Segmentation Fluency: K > Gr 1 • Word Reading Fluency: K > Gr 3 • Oral Reading Fluency: Gr 1 > Gr 8

Comparison of RTI Assessment/Monitoring Systems AimsWeb • Letter Naming Fluency: Beginning K > Beginning Gr 1 • Letter Sound Fluency: Middle K > Beginning Gr 1 • Phoneme Segmentation Fluency: Middle K > Middle Gr 1 • Nonsense Word Fluency: Middle K > End Gr 1 • Oral Reading Fluency: Gr 1 > Gr 8 • Maze (Reading Comprehension Fluency): Gr 1 > Gr 8

Comparison of 2 RTI Assessment/Monitoring Systems • AimsWeb • Letter Naming Fluency: Beginning K > Beginning Gr 1 • Letter Sound Fluency: Middle K > Beginning Gr 1 • Phoneme Segmentation Fluency: Middle K > Middle Gr 1 • Nonsense Word Fluency: Middle K > End Gr 1 • Oral Reading Fluency: Gr 1 > Gr 8 • Maze (Reading Comprehension Fluency): Gr 1 > Gr 8 DIBELS • Initial Sound Fluency: Preschool > Middle K • Letter Naming Fluency:Beginning K > Beginning Gr 1 • Phoneme Segmentation Fluency: Middle K > End Gr 1 • Nonsense Word Fluency: Middle K > Beginning Gr 2 • Oral Reading Fluency: Middle Gr 1 > Gr 6

‘Elbow Group’ Activity: ‘RTI-Ready’ Literacy Measures: A Guide for Elementary Schools • In your ‘elbow groups’: • Review the set of CBM literacy assessment tools in the handout. • Select a ‘starter’ set of literacy measures by grade level that you would like your school to adopt. (If your school already has a standard set of CBM literacy/tools, discuss ways to optimize its use.)

CBM: Developing a Process to Collect Local NormsJim Wrightwww.interventioncentral.org

RTI Literacy: Assessment & Progress-Monitoring To measure student ‘response to instruction/intervention’ effectively, the RTI model measures students’ academic performance and progress on schedules matched to each student’s risk profile and intervention Tier membership. • Benchmarking/Universal Screening. All children in a grade level are assessed at least 3 times per year on a common collection of academic assessments. • Strategic Monitoring. Students placed in Tier 2 (supplemental) reading groups are assessed 1-2 times per month to gauge their progress with this intervention. • Intensive Monitoring. Students who participate in an intensive, individualized Tier 3 intervention are assessed at least once per week. Source: Burns, M. K., & Gibbons, K. A. (2008). Implementing response-to-intervention in elementary and secondary schools: Procedures to assure scientific-based practices. New York: Routledge.

Local Norms: Screening All Students (Stewart & Silberglit, 2008) Local norm data in basic academic skills are collected at least 3 times per year (fall, winter, spring). • Schools should consider using ‘curriculum-linked’ measures such as Curriculum-Based Measurement that will show generalized student growth in response to learning. • If possible, schools should consider avoiding ‘curriculum-locked’ measures that are tied to a single commercial instructional program. Source: Stewart, L. H. & Silberglit, B. (2008). Best practices in developing academic local norms. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp. 225-242). Bethesda, MD: National Association of School Psychologists.

Local Norms: Using a Wide Variety of Data (Stewart & Silberglit, 2008) Local norms can be compiled using: • Fluency measures such as Curriculum-Based Measurement. • Existing data, such as office disciplinary referrals. • Computer-delivered assessments, e.g., Measures of Academic Progress (MAP) from www.nwea.org Source: Stewart, L. H. & Silberglit, B. (2008). Best practices in developing academic local norms. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp. 225-242). Bethesda, MD: National Association of School Psychologists.

Measures of Academic Progress (MAP)www.nwea.org

Applications of Local Norm Data (Stewart & Silberglit, 2008) Local norm data can be used to: • Evaluate and improve the current core instructional program. • Allocate resources to classrooms, grades, and buildings where student academic needs are greatest. • Guide the creation of targeted Tier 2 (supplemental intervention) groups • Set academic goals for improvement for students on Tier 2 and Tier 3 interventions. • Move students across levels of intervention, based on performance relative to that of peers (local norms). Source: Stewart, L. H. & Silberglit, B. (2008). Best practices in developing academic local norms. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp. 225-242). Bethesda, MD: National Association of School Psychologists.

Local Norms: Supplement With Additional Academic Testing as Needed (Stewart & Silberglit, 2008) “At the individual student level, local norm data are just the first step toward determining why a student may be experiencing academic difficulty. Because local norms are collected on brief indicators of core academic skills, other sources of information and additional testing using the local norm measures or other tests are needed to validate the problem and determine why the student is having difficulty. … Percentage correct and rate information provide clues regarding automaticity and accuracy of skills. Error types, error patterns, and qualitative data provide clues about how a student approached the task. Patterns of strengths and weaknesses on subtests of an assessment can provide information about the concepts in which a student or group of students may need greater instructional support, provided these subtests are equated and reliable for these purposes.” p. 237 Source: Stewart, L. H. & Silberglit, B. (2008). Best practices in developing academic local norms. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp. 225-242). Bethesda, MD: National Association of School Psychologists.

Steps in Creating Process for Local Norming Using CBM Measures • Identify personnel to assist in collecting data. A range of staff and school stakeholders can assist in the school norming, including: • Administrators • Support staff (e.g., school psychologist, school social worker, specials teachers, paraprofessionals) • Parents and adult volunteers • Field placement students from graduate programs Source: Harn, B. (2000). Approaches and considerations of collecting schoolwide early literacy and reading performance data. University of Oregon: Retrieved from https://dibels.uoregon.edu/logistics/data_collection.pdf

Steps in Creating Process for Local Norming Using CBM Measures • Determine method for screening data collection. The school can have teachers collect data in the classroom or designate a team to conduct the screening: • In-Class: Teaching staff in the classroom collect the data over a calendar week. • Schoolwide/Single Day: A trained team of 6-10 sets up a testing area, cycles students through, and collects all data in one school day. • Schoolwide/Multiple Days: Trained team of 4-8 either goes to classrooms or creates a central testing location, completing the assessment over multiple days. • Within-Grade: Data collectors at a grade level norm the entire grade, with students kept busy with another activity (e.g., video) when not being screened. Source: Harn, B. (2000). Approaches and considerations of collecting schoolwide early literacy and reading performance data. University of Oregon: Retrieved from https://dibels.uoregon.edu/logistics/data_collection.pdf

Steps in Creating Process for Local Norming Using CBM Measures • Select dates for screening data collection. Data collection should occur at minimum three times per year in fall, winter, and spring. Consider: • Avoiding screening dates within two weeks of a major student break (e.g., summer or winter break). • Coordinate the screenings to avoid state testing periods and other major scheduling conflicts. Source: Harn, B. (2000). Approaches and considerations of collecting schoolwide early literacy and reading performance data. University of Oregon: Retrieved from https://dibels.uoregon.edu/logistics/data_collection.pdf

Steps in Creating Process for Local Norming Using CBM Measures • Create Preparation Checklist. Important preparation steps are carried out, including: • Selecting location of screening • Recruiting screening personnel • Ensure that training occurs for all data collectors • Line up data-entry personnel (e.g., for rapid computer data entry). Source: Harn, B. (2000). Approaches and considerations of collecting schoolwide early literacy and reading performance data. University of Oregon: Retrieved from https://dibels.uoregon.edu/logistics/data_collection.pdf

Team Activity: Draft a Plan to Conduct an Academic Screening in Your School or District Directions: • Develop a draft plan to screen your school using CBM Literacy measures 3 times per year.Use the Harn (2000) guidelines in your planning. • Record the main elements of the plan (‘preparation’, ‘initial implementation’, ‘institutionalization’, ‘ongoing development/updating’) using the RTI Rollout Planning document. • Be prepared to report out on the main elements of your plan to the large group.

Review of RTI Literacy Assessment/Monitoring Tools

Review of RTI Literacy Assessment/Monitoring Tools

Presentation Transcript