Session 4: Confronting Implementation Challenges When Providing Intensive Intervention

1. Strand I: Using Intensive Intervention to Meet the Academic and Behavior Needs of Struggling Learners • Session 4: Confronting Implementation Challenges When Providing Intensive Intervention • Lou Danielson, Allison Gandhi, Rebecca Zumeta, Chris Lemons

2. What Is Intensive Intervention? Intensive intervention addresses severe and persistent learning or behavior difficulties. Intensive intervention should be: • Driven by data • Characterized by increased intensity (e.g., smaller group, expanded time) and individualization of academic instruction and/or behavioral supports

3. Five Data-Based Individualization (DBI) Steps • Secondary intervention program, delivered with greater intensity • Progress monitoring • Informal diagnostic assessment • Adaptation • Continued progress monitoring, with adaptations occurring whenever needed to ensure adequate progress

4. A Bird’s Eye View of DBI

5. High-Performing Sites: What Have We Learned? Allison Gruner Gandhi American Institutes for Research

6. National Center on Intensive Intervention (NCII) Knowledge Development Sites • Purpose: to learn about strategies for implementing intensive intervention from sites that have demonstrated positive outcomes for students with disabilities (SWDs) • Five sites: • Alton, Illinois • Hancock, West Virginia • Jenison, Michigan • Okaloosa, Florida • Scituate, Massachusetts

7. NCII Knowledge Development Sites • Selection criteria and process: • Statistical analysis of state assessment data (Florida, Massachusetts, and West Virginia) • Sustained, strong, “better-than-predicted” academic performance for SWDs during the past five years • Telephone interviews • Clearly articulated vision and strategy for intensive intervention • Nominations from senior advisors • Positive outcomes in behavior • Represent diversity in size, geography, and demographics • Data collection: • Interviews with district and school administrators, interventionist staff, and parents

8. Okaloosa, Florida Scituate, Massachusetts

9. Hancock, West Virginia Jenison, Michigan

10. Alton, Illinois: Increased Implementation Fidelity (2009–11) and Reduction in Tier 3 Office Discipline Referrals (ODRs) (2006–11) Following Comprehensive Implementation of Positive Behavioral Interventions and Supports (PBIS)

11. Lesson 1 • Intensive intervention is most likely to be facilitated when implemented as a component of a multi-tiered system of support (MTSS). • In all sites, intensive intervention was defined as a component of an MTSS. These systems provided an infrastructure to support services for students with the most intensive needs, including those with disabilities, within the general education system.

12. Lesson 2 • Family engagement can be challenging, but is important to pursue to achieve successful outcomes for students with intensive needs. • Staff in all sites described family involvement as important for ensuring success in implementing intensive intervention.

13. Lesson 3 • Implementing intensive intervention in behavior brings a unique set of challenges, due largely to a lack of readily available tools. • The use of data to drive instructional decision making was pervasive in all sites; however, the use of behavioral progress monitoring data was less defined and consistent, largely due to a lack of tools or guidance on how to use such tools.

14. Lesson 4 • Lack of clarity regarding the distinction between Tiers 2 and 3 in a multi-tiered intervention system can make it challenging to appropriately design and plan for intensive intervention. • These five districts lacked a structured system for determining when and how adaptations to interventions should be made. This lack of structure made it difficult for districts to design and plan resources appropriately when implementing intensive intervention.

15. Lesson 5 • Schools and districts should identify and seek to avoid hidden inefficiencies in the ways in which they use staff, particularly skilled special education staff, within the tiered intervention system. • The five districts all found ways to be efficient when devoting resources to intensive intervention. However, our staff also observed subtle ways in which these sites may have been unknowingly and unnecessarily overtaxing their available resources. These included (a) adapting and individualizing secondary interventions too soon, and (b) separating teams and decision-making processes for special education from their tiered intervention system.

16. Confronting Challenges in NCII Implementation Sites Rebecca Zumeta American Institutes for Research

17. Intensive Technical Assistance (TA) Sites • Michigan • Covert • Grand Haven • Saline • Schwartz Creek • Minnesota • Minneapolis • Missouri • Columbia • Hazelwood • Southern Boone • Rhode Island • Bristol-Warren • Coventry • South Kingstown • West Warwick

18. Intensive TA Activities • On-site training on components of DBI • Implementation coaching to reinforce training content • Student-centered intervention planning • Coaching support for meetings, data review, and teacher observations • Collaboration with state and regional TA networks • Evaluation of implementation • Support for capacity building • Planning for implementation in secondary schools

20. Trends in Sites • Very few schools implemented tiered interventions consistently across reading, math, and behavior. • Most sites had difficulty differentiating Tiers 2 and 3 when work began. • Variable inclusion of SWDs in tiered interventions • Inconsistent implementation of progress monitoring and graphing of data • Frequent need for booster sessions related to lower (e.g., Tier 1 or Tier 2) intervention

21. What We Have Learned • Leadership matters. • Training is not enough—implementation support is crucial. • Interventionists are essential recipients of TA. • Personnel changes and competing initiatives impact implementation. • Implementation barriers occur for a variety of reasons—such as content knowledge, resources, time, and school or district policies.

22. Providing Responsive TA • Individualized TA plans • Response to specific training requests • Variety of coaching formats • On-site support for student planning meetings • Flexible formats and timelines for providing TA • Evaluation of implementation fidelity and barriers

23. Fidelity of Intervention in DBI Chris Lemons Vanderbilt University

24. Overview • What is it? • Why does it matter? • What might it look like? • Things to consider

25. What Is It? • Extent to which an intervention’s core components have been implemented as planned (Nelson et al., 2012) • For DBI, this includes the instructional platform, adapted iterations of intervention, ongoing progress monitoring, and decision-making procedures. • See IRIS module for additional information: http://iris.peabody.vanderbilt.edu/module/fid/

26. Why Does It Matter? • Necessary to document that DBI is being carried out as intended • Provides guidance on how to improve DBI implementation • Does the teacher need additional training or support? • Is an instructional adaptation needed? • Is there a systems-level problem (e.g., scheduling prevents sufficient intervention time, staff do not have access to evidence-based instructional platforms)?

27. What Might It Look Like? • Will likely include three levels: • Delivery of intervention (teacher log, observations) • DBI process (checklist and documentation paperwork used in frequent team meetings) • Systems-level rubric (Are essential components of DBI being implemented consistently? Are there systems-level problems that hinder DBI implementation?) • NCII is currently developing these measures.

28. What Might It Look Like? • Example from Lemons, Kearns, and Davidson, 2014. [Most recent version of Teaching Exceptional Children] • Mrs. Arnold, a special education teacher, is implementing DBI with one student, Rashan, in the area of reading. • She asks the reading specialist to observe her in the initial weeks of implementing the new intervention to provide feedback using a codeveloped checklist of essential intervention features.

29. What Might It Look Like? • Special education teacher used a daily log to document content of instruction, student attendance, and engagement. • Special education teacher scheduled bimonthly meetings to review data, adaptations, and DBI procedures with the principal and general education teacher. • In addition, a DBI team or external evaluator could use the NCII fidelity rubric to evaluate consistent use of DBI across a school or district if implementation had reached that level.

30. Things to Consider • Fidelity is like progress monitoring for the “responsiveness” of the system. • Can guide improvement efforts • Provides a feedback loop • At the student level, it is a type of ongoing formative assessment to guide adaptations.

31. Things to Consider • However, fidelity should focus on both “small” (individual teacher-student interactions) and “big” (systems-level features) components of DBI. • So, it is not just about whether teachers have the knowledge to implement DBI. • It also is about whether the bigger system is working as intended. • There is likely a need to build in contingency plans that outline responses when things are NOT going as planned.

32. Things to Consider • The primary focus of assessing fidelity in DBI should be to improve outcomes for students. • Problem-solving focus • Collaborative team efforts • Documenting and trying to understand challenges will help lead to solutions.

33. Summary and Conclusions Lou Danielson American Institutes for Research

34. Summary and Conclusions • Stakes are high—SWD performance is poor. • DBI holds great promise. • Implementing and sustaining innovations in education is difficult work. • NCII is providing high-quality training and coaching. • Implementation still may be variable.

35. Summary and Conclusions • NCII is assessing implementation fidelity (formative evaluation) at three levels: • System level • Staff level • Student level • Will follow up with sites based on implementation issues. • A summative evaluation to assess student outcomes also is in process.

36. Summary and Conclusions • Last session of strand—We hope you understand the potential of DBI to address the challenge before us. Please visit our website for more information. Also, see the current issue of Teaching Exceptional Children. Contact us at ncii@air.org, www.intensiveintervention.org 1050 Thomas Jefferson Street NW Washington, DC 20007-3835 866-577-5787

37. References • Lemons, C. J., Kearns, D. M., & Davidson, K. A. (2014). Data-based individualization in reading: Intensifying interventions for students with significant reading disabilities. Teaching Exceptional Children, 46(4), 20-29. • Nelson, M. C., Cordray, D. S., Hulleman, C. S., & Sommer, E. C. (2012). A procedure for assessing intervention fidelity in experiments testing educational and behavioral interventions. The Journal of Behavioral Health Services & Research, 39(4), 374-396.

38. National Center on Intensive Intervention This presentation was produced under the U.S. Department of Education, Office of Special Education Programs, Award No. H326Q110005. Celia Rosenquist serves as the project officer. The views expressed herein do not necessarily represent the positions or polices of the U.S. Department of Education. No official endorsement by the U.S. Department of Education of any product, commodity, service, or enterprise mentioned in this website is intended or should be inferred.