Evaluation of Response to Instruction and Intervention of Selected Statewide Schools

1. Evaluation of Response to Instruction and Intervention of Selected Statewide Schools Edward S. Shapiro, Ph.D. University Consultant to PDE RtII Initiative

2. Selected School Statewide EvaluationMethod PaTTAN surveyed all Intermediate Units, n=11 Rate districts as Beginning, In Process, or Experienced in RtII IU techs contacted those rated Experienced 15 districts invited to volunteer data sets 9 districts participated 5 additional pilot site schools added All school districts/schools sent request to describe components of RtII model Total of 37 schools participated

3. Selected Schools Wide variety of demographics Urban, rural, suburban Minority Free/Reduced Lunch Represented wide geographic region of state 3 Eastern region 7 Central region 4 Western region

4. 4 Key Outcome Evaluation Questions Risk levels? DIBELS/AIMSweb, 4Sight PSSA outcomes Movement between tiers? Movement within tiers (progress monitoring of students in tiered instruction)? Special education referrals for SLD consideration

5. 1. Levels of Risk If an RTII model is working, the pattern we would expect to see is low risk going up while at-risk goes down.

6. Did the high risk go down throughout the year for Kindergarten? Did the low risk go up throughout the year for kindergarten?

7. Risk Levels – DIBELS/AIMSweb – Gr 1

8. Risk Levels – DIBELS/AIMSweb- Gr 2

9. Risk Levels – DIBELS/AIMSweb & 4Sight – Gr 3

10. Risk Levels – DIBELS/AIMSweb & 4Sight – Gr 4

11. Risk Levels – DIBELS/AIMSweb & 4Sight – Gr 5

12. Risk Level in Spring Across Grades

13. Risk Levels Summary Strongest outcomes are at K and 1 Stabilized outcomes from grades 2 through 4 Somewhat improved at grade 5 over 2 – 4 4Sight outcomes stronger than DIBELS/AIMSweb Wide range of responses at all grades across schools PSSA outcomes 72% of schools at AYP Improvement or maintenance over 2007 in 70% of schools

14. Fewer students Moving from Tiers 1, to 2 to 3 More students Moving from Tiers 3, to 2 to 1 2. Tier Movement If RTI is working we should have more students moving from intensive to less intensive tiers than the other direction.

15. Tier Movement Change in tier assignment based on universal screening data (DIBELS or AIMSweb) Examined Beginning to Middle of year (BOY to MOY) Examined Middle to End of year (MOY to EOY) Partial reflection of when and where the most impact of the RtII implementation can be experienced Data averaged across schools and students

16. BOY to MOY- Students Starting at Intensive (Tier 3)

17. MOY to EOY- Students Starting at Benchmark (Tier 1)

18. Tier Movement Summary Largest amount of movement occurs in the earliest grades Largest amount of movement occurs from the BOY to MOY. Reinforce the importance of early intervention and the need to attend to improving literacy skills at the youngest grades Still change possible at higher grades as well as from MOY to EOY, the stabilization of student performance over grades and time is the predominant finding from these data

19. 3. Progress within the tiers If RTII is working, students who are receiving interventions will need to close the gap between themselves and their peers and move at a rate faster than that of their peers.

20. 3rd Grade Example • Typical 3rd grade student: fall benchmark 77 wcpm, ends in the spring at benchmark with 110 wcpm. • 110-77=33 words gained in a year. • 33 words divided by 36 weeks = .9 • .9 wcpm typical weekly gain for a 3rd grade student • Struggling 3rd grade student: fall reading 60 wcpm. • To reach the spring benchmark of 110 at the end of the year • 110-60 = 50 words needed to gain • 50 divided by 36 weeks = 1.39 • Struggling students have to accelerate their learning to reach the targeted rate of improvement

21. Movement within Tiers

22. Movement Within Tier Summary Kindergarten and grade 5 exceeded targeted ROI Kindergarten and grades 4 and 5 exceeded typical ROI Improvement evident in all grades, smallest in grade 3

23. 4. Special Education Referral Pattern If RTII is working, we should see a reduction in the number of referrals overall and we should see what we are calling an improvement in the efficiency of referral.

24. Special Ed Referral Patterns Limited data available 4 districts, 14 schools, 1 district containing 10 schools Data provided on students referred, evaluated, and eligibility decision Only those being considered for SLD were included in the analysis

25. Summary Special Ed Referral Patterns Between <1% and 2.5% of students across these schools were evaluated for special education as potential students with SLD Between 33% and 61% of students evaluated for SLD were found to be eligible Parent initiated referrals varied from approximately 33% to 67% of evaluations Discrepancy analysis found statistically significant difference between those found eligible and those not found eligible

26. Summary Largest changes occur in K and 1 While change is possible at higher grades, more difficult to get changes in those grades DIBELS/AIMSweb tend to overestimate risk at higher grades Preliminary evidence that a discrepancy analysis of end of year DIBELS/AIMSweb performance related to eligibility decisions for SLD