Benchmark Screening: What, Why and How

1. Benchmark Screening: What, Why and How A module for pre-service and in-service professional development MN RTI Center Author: Lisa H. Stewart, PhD Minnesota State University Moorhead www.scred.k12.mn.us click on RTI Center

2. MN RTI Center Training Modules • This module was developed with funding from the MN legislature • It is part of a series of modules available from the MN RTI Center for use in preservice and inservice training:

3. Overview • This module is Part 1 of 2 • Module 1: Benchmark Screening: What, Why and How • What is screening? • Why screen students? • Criteria for screeners/what tools? • Screening logistics • Module 2: Using Benchmark Screening Data

4. Assessment: One of the Key Components in RTI Curriculum and Instruction Assessment School Wide Organization & Problem Solving Systems (Teams, Process, etc) Adapted from Logan City School District, 2002

5. Assessment and Response to Intervention (RTI) A core feature of RTI is identifying a measurement system Screen large numbers of students Identify students in need of additional intervention Monitor students of concern more frequently 1 to 4x per month Typically weekly Diagnostic testing used for instructional planning to help target interventions as needed 5

6. Why Do Screening? • Activity • What does it mean to “screen” students? • Why is screening so important in a Response to Intervention system? (e.g., what assumptions of RTI require a good screening system?) • What happens if you do NOT have an efficient, systematic screening system in place in the school?

7. Screening is part of a problem-solving system Helps identify students at-risk in a PROACTIVE way Gives feedback to the system about how students progress throughout the year at a gross (3x per year) level If students are on track in the fall are they still on track in the winter? What is happening with students who started the year below target, are they catching up? Gives feedback to the system about changes from year to year Is our new reading curriculum having the impact we were expecting? 7

8. What Screening Looks Like in a Nutshell • School decides on brief tests to be given at each grade level and trains staff in the administration, scoring and use of the data • Students are given the tests 3x per year (Fall, Winter, Spring) • Person or team assigned in each building to organize data collection • All students are given the tests for their grade level within a short time frame (e.g., 1-2 weeks or less). Some tests may be group administered, others are individually administered. • Benchmark testing: about 5 minutes per student, desk to test (individually administered) • Administered by special ed, reading, or general ed teachers or paras • Entered into a computer/web based reporting system by clerical staff • Reports show the spread of student skills and lists student scores, etc. to use in instructional and resource planning

9. Example Screening Data:Spring Gr 1 Oral Reading Fluency 10/51 (20%) high risk 22/51 (43%) some risk 19/51 (37%) low risk: on or above target Class lists then identify specific students (and scores) in each category

10. Screening Data Gives an idea of what the range of student skills is like in your building and how much growth over time students are making

11. Screening Data can be linked to Progress Monitoring The goal is to have a cohesive system. If possible, use the same measures for both screening and progress monitoring (e.g, CBM). Screen ALL students 3x per year (F, W, S) Strategic Support and Monitoring Students at Some Risk Intensive Support & Monitoring for Students at Extreme Risk

12. A Smart System Structure Academic Systems Behavioral Systems • Intensive, Individual Interventions • Individual Students • Assessment-based • High Intensity • Of longer duration • Targeted Group Interventions • Some students (at-risk) • High efficiency • Rapid response • Targeted Group Interventions • Some students (at-risk) • High efficiency • Rapid response • Universal Interventions • All students • Preventive, proactive • Universal Interventions • All settings, all students • Preventive, proactive 75-85% 75-85% School-Wide Systems for Student Success • Intensive, Individual Interventions • Individual Students • Assessment-based • Intense, durable procedures 5-10% 5-10% 10-15% 10-15%

13. Terminology Check • Screening • Collecting data on all or a targeted group of students in a grade level or in the school • Universal Screening • Same as above but implies that all students are screened • Benchmarking • Often used synonymously with the terms above, but typically implies universal screening done 3x per year and data are interpreted using criterion target or “benchmark” scores

14. “Benchmark” Screening • Schools typically use cut off or criterion scores to decide if a student is at-risk or not. Those scores or targets are also referred to as “benchmarks”, thus the term “benchmarking” • Some states or published curriculum also use the term benchmarking but in a different way (e.g., to refer to the documentation of achieving a specific state standard) that has nothing to do with screening.

15. What to Measure for Screening?Create a “Measurement Net”:

16. How do you decide what Measures to Use for Screening? • Lots of ways to measure reading in the schools: • Measure of Academic Progress (MAP) • Guided Reading (Leveled Reading) • Statewide Accountability Tests • Published Curriculum Tests • Teacher Made Tests • General Outcome Measures (Curriculum-Based Measurement “family”) • STAR Reading • Etc • Not all of these are appropriate. Some are not reliable enough for screening, others are designed for another purpose and are not valid or practical for screening all students 3x per year

17. Characteristics of An Effective Measurement System for RTI valid reliable simple quick inexpensive easily understood can be given often sensitive to growth over short periods of time Credit: K Gibbons, M Shinn

18. Effective Screening Measures • Specific • Identifies at risk students who really are at risk • Sensitive • Students who “pass” really do go on to do well • Practical • Brief and simple (cheap is nice too) • Do no harm • If a student is identified as at risk will they get help or is it just a label? Reference: Hughes & Dexter, RTI Action Network

19. Buyer Beware! Many tools may make claims about being a good “screener”

20. Measurement and RTI: Screening Reliability coefficients of at least r =.80. Higher is better, especially for screening specificity. Well documented predictive validity Evidence the criterion (cut score) being used is reasonable and creates not too many false positives (students identified as at risk who aren’t) or false negatives (students who are at risk who aren’t identified as such) Brief, easy to use, affordable, and results/reports are accessible almost immediately

21. National Center for RTI Review of Screening Tools Note: Only reviews tests submitted, if it is not on the list it doesn’t mean it is bad, just that it wasn’t reviewed www.rti4success.org

22. RTI, General Outcome Measures and Curriculum Based Measurement Many schools use Curriculum Based Measurement (CBM) general outcome measures for screening and progress monitoring You don’t “have to “ use CBM, but many schools do Most common CBM tool in Grades 1- 8 is Oral Reading Fluency (ORF) Measure of reading rate (# of words correct per minute on a grade level passage) and a strong indicator of overall reading skill, including comprehension Early Literacy Measures are also available such as Nonsense Word Fluency (NWF), Phoneme Segmentation Fluency (PSF), Letter Name Fluency (LNF) and Letter Sound Fluency (LSF) 22

23. Why GOMs/CBM? Typically meet the criteria needed for RTI screening and progress monitoring Reliable, valid, specific, sensitive, practical Also, some utility for instructional planning (e.g., grouping) They are INDICATORS of whether there might be a problem, not diagnostic! Like taking your temperature or sticking a toothpick into a cake Oral reading fluency is a great INDICATOR of reading decoding, fluency and reading comprehension Fluency based because automaticity helps discriminate between students at different points of learning a skill 23

24. GOM…CBM… DIBELS… AIMSweb… DRAFT May 27, 2009 24

25. CBM Oral Reading Fluency Give 3 grade-level passages using standardized admin and scoring; use median (middle) score 3-second rule (Tell the student the word & point to next word) Discontinue rule (after 0 correct in first row, if <10 correct on 1st passage do not give other passages) 25

26. Fluency and Comprehension The purpose of reading is comprehension A good measures of overall reading proficiency is reading fluency because of its strong correlation to measures of comprehension.

27. Screening Logistics • What materials? • When to collect? • Who collects it? • How to enter and report the data?

28. What Materials? • Use computer or PDA-based testing system -OR- • Download reading passages, early literacy probes, etc. from the internet • Many sources of CBM materials available free or low cost: Aimsweb, DIBELS, edcheckup, etc. • Often organized as “booklets” for ease of use • Can use plastic cover and markers for scoring to save copy costs

29. Screening Materials in K and Gr 1 • Screening Measures will change from Fall to Winter to Spring slightly • Early literacy “subskill” measurement is dropped as reading develops • Downloaded materials and booklets

30. K and Gr 1 MeasuresAIMSweb Early Literacy and R-CBM(ORF) General Literacy Risk Factor= Black, Alphabetic Principle = Green Phonemic Awareness = Purple, Vocabulary = Blue Fluency with Connected Text & Comprehension= Red

31. Gr 2 to 12: AIMSweb Early Literacy and CBM Measures

32. Screening Logistics: Timing • Typically 3x per year: Fall, Winter, Spring • Have a district-wide testing window! (all grades and schools collect data within the same 2 week period) • In Fall K sometimes either test right away and again a month later or wait a little while to test • Benchmark testing: about 5 minutes per student (individually administered) • In the classroom • In stations in a commons area, lunchroom, etc.

33. Screening Logistics: People • Administered by trained staff • paras, special ed teachers, reading teachers, general ed teachers, school psychologists, speech language, etc. • Good training is essential! • Measurement person assigned in each building to organize data collection • Either collected electronically or entered into a web-based data management tool by clerical staff

34. Screening Logistics Math Quiz  • If you have a classroom with 25 students and to administer the screening measures takes approx. 5 min. per student (individual assessment time)… • How long would it take 5 people to “screen” the entire classroom?

35. Remember: Garbage IN…. Garbage OUT…. • Make sure your data are reliable and valid indicators or they won’t be good for nuthin… • Training • Assessment Integrity checks/refreshers • Well chosen tasks/indicators

36. Use Technology to Facilitate Screening

37. Using Technology to Capture Data • Collect the data using technology such as a PDA • Example:http://www.wirelessgeneration.com/ http://www.aimsweb.com • Students take the test on a computer • Example: STAR Reading http://www.renlearn.com/sr/

38. Using Technology to Organize and Report Data • Enter data into web-based data management system • Data gets back into the hands of the teachers and teams quickly and in meaningful reports for problem solving • Examples • http://dibels.uoregon.edu • http://www.aimsweb.com • http://www.edcheckup.com

39. Screening is just one part of an overall assessment system for making decisions

40. Remember: Screening is part of a problem-solving system • Helps identify students at-risk in a PROACTIVE way • Gives feedback to the system about how students progress throughout the year at a gross (3x per year) level • If students are on track in the fall are they still on track in the winter? • What is happening with students who started the year below target, are they catching up? • Gives feedback to the system about changes from year to year • Is our new reading curriculum having the impact we were expecting?

41. Build in Time to USE the Data! Schedule data “retreats” or grade level meeting times immediately after screening so you can look at and USE the data for planning.

42. Common Mistakes • Not enough professional development and communication about why these measures were picked, what the scores do and don’t mean, the rationale for screening, etc • Low or questionable quality of administration and scoring • Too much reliance on a small group of people for data collection • Teaching to the test • Limited sample of students tested (e.g., only Title students! ) • Slow turn around on reports • Data are not used

43. Using Screening Data: See Module 2!

44. Articles available with this module Stewart & Silberglitt. (2008). Best practices in developing academic local norms. In A. Thomas & J. Grimes (Eds.) Best Practices in School Psychology, V, NASP Publications.(pp. 225-242). NCRLD RTI Manual (2006). Chapter 1: School-wide screening Retrieved from http://www.nrcld.org/rti_manual/pages/RTIManualSection1.pdf 6/26/09 Jenkins & Johnson. Universal screening for reading problems: Why and how should we do this? Retrieved 6/23/09, from RTI Action Network site: http://www.rtinetwork.org/Essential/Assessment/Universal/ar/ReadingProblems Kovaleski & Pederson (2008) Best practices in data analysis teaming. In A. Thomas & J. Grimes (Eds.) Best Practices in School Psychology, V, NASP Ikeda, Neessen, & Witt. (2008). Best practices in universal screening. In A. Thomas & J. Grimes (Eds.) Best Practices in School Psychology, V, NASP Publications.(pp. 103-114). Gibbons, K (2008). Necessary Assessments in RTI. Retrieved from http://www.tqsource.org/forum/documents/GibbonsPaper.doc on 6/26/09

45. RTI Related Resources • National Center on RTI • http://www.rti4success.org/ • RTI Action Network – links for Assessment and Universal Screening • http://www.rtinetwork.org • MN RTI Center • http://www.scred.k12.mn.us/ and click on link • National Center on Student Progress Monitoring • http://www.studentprogress.org/ • Research Institute on Progress Monitoring • http://progressmonitoring.net/

46. RTI Related Resources (Cont’d) • National Association of School Psychologists • www.nasponline.org • National Association of State Directors of Special Education (NADSE) • www.nasdse.org • Council of Administrators of Special Education • www.casecec.org • Office of Special Education Programs (OSEP) toolkit and RTI materials • http://www.osepideasthatwork.org/toolkit/ta_responsiveness_intervention.asp

47. Key Sources for Reading Research, Assessment and Intervention… • University of Oregon IDEA (Institute for the Development of Educational Achievement) Big Ideas of Reading Site • http://reading.uoregon.edu/ • Florida Center for Reading Research • http://www.fcrr.org/ • Texas Vaughn Gross Center for Reading and Language Arts • http://www.texasreading.org/utcrla/ • American Federation of Teachers Reading resources (what works 1999 publications) • http://www.aft.org/teachers/pubs-reports/index.htm#reading • National Reading Panel • http://www.nationalreadingpanel.org/

48. Recommended Sites with Multiple Resources • Intervention Central- by Jim Wright (school psych from central NY) • http://www.interventioncentral.org • Center on Instruction • http://www.centeroninstruction.org • St. Croix River Education District • http://scred.k12.mn.us

49. Quiz • 1.) A core feature of RTI is identifying a(n) _________ system. • 2.) Collecting data on all or a targeted group of students in a grade level or in the school is called what? • A.) Curriculum • B.) Screening • C.) Intervention • D.) Review

50. Quiz (Cont’d) • 3.) What is a characteristic of an efficient measurement system for RTI? • A.) Valid • B.) Reliable • C.) Simple • D.) Quick • E.) All of the above