Progress Monitoring in Reading: Why, What, and How

1. Progress Monitoring in Reading:Why, What, and How A module for pre-service and in-service professional development MN RTI Center Authors: Lisa Habedank Stewart, PhD & Adam Christ, graduate student Minnesota State University Moorhead www.scred.k12.mn.us click on RTI Center

2. MN RTI Center Training Modules This module was developed with funding from the MN legislature It is part of a series of modules available from the MN RTI Center for use in preservice and inservice training: 2

3. Overview This module is Part 1 of 2 Part 1: Why, What, How to Progress Monitor Why do it? What do we mean by progress monitoring? How do you do it? Part 2: Using Progress Monitoring Data

4. Is this student making progress?

5. Assessment: One of the Key Components in RTI Curriculum and Instruction Assessment School Wide Organization & Problem Solving Systems (Teams, Process, etc) Adapted from Logan City School District, 2002

6. Assessment and Response to Intervention (RTI) A core feature of RTI is identifying a measurement system Screen large numbers of students Identify students in need of additional intervention Monitor students of concern more frequently 1 to 4x per month Typically weekly Diagnostic testing used for instructional planning to help target interventions as needed 6

7. Screening Data and Progress Monitoring can be linked The goal is to have a cohesive system. If possible, use the same measures for both screening and progress monitoring (e.g, CBM). Screen ALL students 3x per year (F, W, S) Strategic Support and Monitoring Students at Some Risk Intensive Support & Monitoring for Students at Extreme Risk

8. Why Monitor Progress? When teachers USE progress monitoring Students learn more! Teachers design better instructional programs Teacher decision making improves Students become more aware of their performance Safer & Fleishman, 2005

9. Why Monitor Progress? “In God we trust… All others must have data.” Dr. Stan Deno

10. Why Monitor Progress? We do NOT KNOW ahead of time whether an intervention will be successful for an individual student Do they ASSUME in the hospital that your heart is working just fine after your bypass surgery? After all… the surgery works well for MOST patients…..

11. To change what you are doing with a student if it is not working (formative assessment) so you are effective and efficient with your time and instruction To help make decisions about instructional goals, materials, levels, and groups To aid in communication with parents To document progress for special education students as required for periodic and annual reviews Why Progress Monitor Frequently? Credit: based on slide by Dr. Kim Gibbons, SCRED

12. Change (and Keep Changing) if Instruction isn’t Working… Credit: SCRED

13. You Can Also Use Monitoring Data to Celebrate Success!

14. What is Progress Monitoring? Standardized measures Reliable Valid Tied to important educational outcomes Given frequently (e.g., weekly) Simple, brief, efficient, and cheap  Sensitive to growth over short periods of time

15. Measurement and RTI: Progress Monitoring Reliability coefficients of r=.90+ Well documented treatment validity! Test and scores are very sensitive to increases or decreases in student skills over time Evidence of what slope of progress (how much growth in a day, week or a month) is typical under what conditions can greatly increase your ability to make decisions VERY brief, easy to use, affordable, alternate forms, and results/reports are accessible immediately

16. Standards for Scientifically Based Progress Monitoring National Center for Progress Monitoring www.studentprogress.org

17. Many tools that meet standards are members of the Curriculum-Based Measurement (CBM) “family.” www.studentprogress.org click on “Tools”

18. Buyer Beware! Many tools may make claims about monitoring progress but… Is it reliable and valid? Is it tied to important educational outcomes? Can it be given frequently (e.g., weekly)? Simple, brief, efficient, and cheap  Sensitive to growth over short periods of time

19. Short Term (Mastery) and Long Term Progress Monitoring Short Term Mastery Monitoring Test subskill mastery and individual lesson effectiveness Ex: Q&A, worksheets following directions unit tests, “hot” reads accuracy, skills “checks” CBE, cba Long Term General Outcome Measures Test retention, generalization and progress toward overall general outcome (reading) Ex: CBM, DIBELS

20. Both Mastery Monitoring and Long Term Progress Monitoring are Important Sometimes mastering subskills doesn’t generalize to the general outcome or students don’t retain the information over time For example: Melissa is very good at decoding letters and reading individual words, but is not generalizing these skills to reading text with automaticity and comprehension. Adam was really good at using his comprehension strategies and using those when they were working on these skills in class (showed mastery), but when they moved on to another unit he quit using the strategies.

21. How Often Do I Need to Monitor Progress? Informally we do this all the time! For small instructional adjustments (repeat the lesson, how much help to give, etc.) In a standardized way to make sure we are “on track” with this student?- depends on level of concern For students in reading who are behind already, monitor progress toward generalized outcome 1 to 4x per month, ideally weekly

22. How Do You Collect Frequent Progress Monitoring Data? Which students? What measures? What materials? How often? Who collects the data? Where? When?

23. Which students? Students of concern Below target Getting “extra” intervention or help Tier 2 or Tier 3 services

24. What measures? Web-based sources for information on measures as well as access to materials, web-based data management, etc. www.aimsweb.com dibels.uoregon.edu www.edcheckup.com Disclaimer: More data management systems are being developed and marketed all the time to provide information, resources, and data management. This is just a sample of the types of web-based resources available. Be a critical consumer!

25. What materials? When possible, students are monitored using grade level materials E.g., student reads a different grade level passage or “probe” each week If this is not possible due to frustration “test down” and use the highest grade level of materials possible Periodically “check” how the student is doing on grade level materials and move into grade level materials as soon as possible

26. Example of Testing Down: Reading “Survey Level” Data for Gus, Gr 4 Note: -Error rates high (5-15) in Gr 4, -slightly lower (4-8) in Gr 3 & 2, -and much lower (1-4)in Gr 1.

27. What Material Should We Use… To “instruct” Gus? To monitor Gus’ progress?

28. How Often Do We Monitor Progress? Depends on the sensitivity of the measure and the level of concern we have about the student, but 1 to 4x per month typically For CBM Oral Reading Fluency Weekly with 1 passage (this is most common) Every 3 weeks with 3 passages

29. Who does it? When? Where? Anyone trained in the procedures can collect progress monitoring data Classroom teachers, special education teachers, Title teachers, aides, related services staff, adult volunteers Be creative but careful When and Where? At a time and place that will provide valid information Use common sense

30. Sharing the Data Just having progress monitoring data is not enough. You need to USE it. Scheduled graph review dates Grade level meetings Problem solving meetings

31. Using Progress Monitoring Data: Is this intervention working? See MN RTI Center Module 2 for information on graphs, decision rules, etc 31

32. Remember: Garbage IN…. Garbage OUT…. Make sure your data have integrity or they won’t be good fer nuthin… Training Integrity checks/refreshers Well chosen measures and materials

33. Avoid Common Mistakes Don’t use the same passage/probe every week!  Have an organized system in place Progress monitoring schedule for students Preprinted passages/probes in a binder An easy way to graph and look at the data Scheduled time to share/look at the data

34. Remember: When teachers USE progress monitoring Students learn more! Teachers design better instructional programs Teacher decision making improves Students become more aware of their performance Safer & Fleishman, 2005

35. Web Resources • Research Institute on Progress Monitoring • http://progressmonitoring.net/ • Includes… • A Study Group Content Module with 15 sections on CBM including activities • http://progressmonitoring.org/pdf/cbmMOD1.pdf • Progress Monitoring Leadership Team Content Module with 6 sections (e.g. measureable goals, decision making) including activities • http://progressmonitoring.org/pdf/cbmMODldrshp.pdf • Handouts, videos, and power point presentations • Technical reports of CBM measures

36. Web Resources, Cont’d www.studentprogress.org http://www.studentprogress.org/chart/chart.asp http://www.rti4success.org/ click on Progress monitoring on right side www.interventioncentral.org look for information on CBM, graphing, etc. www.aimsweb.com, www.edcheckup.com, dibels.uoregon.edu Look for information about progress monitoring as well as access to materials and graphing for progress monitoring Research Matters: How Student Progress Monitoring Improves Instruction

37. Recommended Texts • Riley-Tillman & Burns. (2009). Evaluating Educational Interventions. Guilford Press.

38. Print Resources available with this module • Safer & Fleishman. (2005). How student progress monitoring improves instruction, Educational Leadership, 62(5), 81-83. • Fuchs. Progress monitoring within a multi-level prevention system. Retrieved June 5, 2009, from RTI Action Network Web site: http://www.rtinetwork.org/Essential/Assessment/Progress/ar/MultilevelPrevention • Fuchs & Fuchs What is scientifically-based research on progress monitoring? From the National Center on Student Progress Monitoring (studentprogress.org). Retrieved June 14, 2009, • Jenkins, Hudson, & Hee Lee. Using CBM-Reading assessments to monitor progress. Retrieved June 5, 2009, from RTI Action Network Web site: http://www.rtinetwork.org/Essential/Assessment/Progress/ar/Using CBM/1

39. Activity for Teachers or Practicum Students • Obtain progress monitoring probes and graphs • Passages and graphing materials self-created or downloaded • www.interventioncentral.org • dibels.uoregon.edu • Sign up for an account with AIMSweb (instructor accounts and student accounts available) • www.aimsweb.org • Practice administration and scoring • Progress Monitor a “real” kid (ideally 2-4 kids of varying risk levels monitored for at least 7-10 weeks) • Graph, analyze, and use data

40. Quiz • 1.) What are at least four reasons why teachers should monitor progress? • 2) What is the difference between mastery monitoring and frequency progress monitoring toward a general outcome? • 3) How often should you monitor progress for students receiving extra help ?

41. Quiz (Cont’d) • 4.) When possible, students are monitored using… • A.) above grade level materials. • B.) grade level materials. • C.) below grade level materials. • 5.) Who can collect progress monitoring data?

42. Quiz (Cont’d) • 6.) To have integrity, your data collection must include what? • A.) trained data collectors • B.) integrity checks/refreshers • C.) well chosen measures and materials • D.) all of the above

43. Quiz (Cont’d) • 7.) True or False? The most important use of frequent progress monitoring is to aid in communication with parents.

44. Note: The MN RTI Center does not endorse any particular product. Examples used are for instructional purposes only. • Special Thanks: • Thank you to Dr. Ann Casey, director of the MN RTI Center, for her leadership • Thank you to Aimee Hochstein, Kristen Bouwman, and Nathan Rowe, Minnesota State University Moorhead graduate students, for editing, writing quizzes, and enhancing the quality of these training materials