Scaling Up Evidence-based Practices

1. Scaling UpEvidence-based Practices Dean Fixsen, Karen Blase, Rob Horner, and George Sugai

2. Current Challenges Effective practices exist for literacy, social behavior and safety Traditional TA approaches (dissemination and compliance) are insufficient Methods are available to reliably produce benefits to students statewide © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

3. Currently EBPs now are boutique operations Now have convincing demonstrations that EBPs can work in the real world This is encouraging, but EBPs are not used on a sustainable scale sufficient to solve social problems © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

4. Future Start with the end in mind What will it take to: make statewide use of education innovations that produce increasingly effective outcomes for the next 50 years? © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

5. Problem • Most States already have attempted to implement a variety of these science-based practices • Some attempts achieve their purposes, but others fall short © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

6. Key Issues Four big issues have emerged: Select what to scale up Determine how to implement new education practices so they actually produce the intended benefits to students (effective) Define how to scale up effective practices so they are available to all students who could benefit from them (effective, accessible) Define how to align system structures and functions to fully support scale up efforts as part of “education as usual” (effective, accessible, sustainable) © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

7. When is a “practice” ready for TA Investment • Defining a “practice” • A “practice” is a procedure, or set of procedures, designed for use in a specific context, by individuals with certain skills/features, to produce specific outcomes for specific individuals. • Operationally defined procedures • Target population/ Context • Implementer Characteristics • Defined outcomes

8. The scope/size of a practice matters • Practices can be small, medium or large in scope • Time delay • Functional behavioral assessment • First Step to Success • Scale up practices only when they are bundled to a level where they have impact on a core social outcome. © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

9. When is a “practice” ready for TA Investment • Wisdom from: • Karin Guldbrandsson • H. Gotham • Dean Fixsen and Karen Blase • George Sugai • Bruce Joyce and Beverly Showers • Eight Features of a “practice” that make it “ready for implementation” investment

10. When is a “practice” ready for TA Investment • Substantive impact on highly valued outcome. • Practice is comprehensive enough that it addresses a highly valued outcome (not just a piece) • There is a perceived need for the outcome • Effect is dramatic (both in relative and absolute terms) • The practice is defined with precision • The practice can be described with operational precision and substance. • What is the “it” • Compatibility with existing practices

11. When is a “practice” ready for TA Investment • Evidence-based • Experimental documentation of relationship between practiced and impact published in peer-reviewed journal. (Effect Size of at least .50) • Emerging  Promising  Evidence-based • Efficiency • Minimal additional resources to perform new practice • Minimal additional resources to transition to new practice

12. When is a “practice” ready for TA Investment • Social Validity • The people required to behave differently are comfortable with the new behavior(s) they are being asked to perform • Documentation of effectiveness trials • Adaptability • The practice can be adapted to “fit” different contexts • The challenge of being precise and manualized versus being flexible and adaptive (be clear about core). • Generalizability • The practice is effective across the array of contexts normally encountered.

13. Priority Effectiveness Valued Outcomes Identifying & Modifying Practices Data- Based Prob. Solving Continuous Regeneration Capacity Building Continuous Measurement Practice Implementation Efficiency School Context

14. What “packaging features” are necessary for successful adoption? • Materials for exploration and awareness • What is the need, what is the practice, what is the documented link between the practice and valued outcome? • Materials for Installation and Initial Implementation • Materials for teaching people to master the new practice • Protocol for implementation

15. What “package features” are necessary for successful adoption? • Evaluation tools • Is practice being implemented with fidelity • Is practice producing valued outcome • Are training segments producing implementation of core elements of “practice”

16. © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

17. Evaluation Guided Action Planning © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

18. Key Issues Interventions that are and remain effective through several generations of teachers, principals, superintendents, and State and national leaders Implementation supports that are and remain effective through several generations of trainers, coaches, evaluators, administrators, and State leaders © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

19. Teaching–Family Replications Fixsen, Blase, Timbers, & Wolf (2001) 900 300 800 700 250 600 200 500 CumulativeHomes 150 400 300 100 CumulativeCouples 200 50 100 0 0 ≤1972 1974 1976 1978 1980 1982 © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

20. Follow Through Programs Figure 1: This figure shows the average effects of nine Follow Through models on measures of basic skills (word knowledge, spelling, language, and math computation), cognitive-conceptual skills (reading comprehension, math concepts, and math problem solving) and self-concept. This figure is adapted from Engelmann, S. and Carnine, D. (1982), Theory of Instruction: Principles and applications. New York: Irvington Press. © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

21. School Adopting School-wide Positive Behavior Support

22. Insufficient Methods Excellent experimental evidence for what does not work • Diffusion/dissemination of information by itself does not lead to successful implementation (research literature, mailings, promulgation of practice guidelines) • Training alone, no matter how well done, does not lead to successful implementation © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

23. Insufficient Methods Excellent experimental evidence for what does not work • Implementation by laws/ compliance by itself does not work • Implementation by “following themoney” by itself does not work • Implementation without changing supporting roles and functions does not work Paul Nutt (2002). Why Decisions Fail © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

24. Sobering Observation The Oregon Department of Education has released graduation rates for all public high schools. Nearly one-third of all high school students don't receive a diploma after four years of study. by Betsy Hammond, The Oregonian Monday June 29, 2009, Reduction in Incidence of Mental Retardation and Learning Disabilities "All organizations [and systems] are designed, intentionally or unwittingly, to achieve precisely the results they get."R. Spencer Darling Business Expert Rise in Incidence of Autism © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

25. Sobering Observations • Innovative practices do not fare well in old organizational structures and systems • Organizational and system changes are essential to successful implementation • Expect it • Plan for it © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

26. Benefits to focusing on Scaling Implementation Build on local strengths, context and accomplishments Implement 1-3 evidence-based practices at a sustainable scale of social importance. Document value of implementation to achieve valued student outcomes © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

27. Benefits to focusing on Scaling Implementation Develop capacity of your State to implement any evidence-based practice with fidelity and good outcomes De-silo implementation of evidence-based practices, innovations, legislative mandates in the State © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

28. Scale Up • To scale up interventions we must first scale up implementation capacity • Building implementation capacity is essential to maximizing the use of EBPs and other innovations • Large scale, real time change © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

29. Academic Systems Behavioral Systems • Intensive, Individual Interventions • Individual Students • Assessment-based • High Intensity • Of longer duration • Targeted Group Interventions • Some students (at-risk) • High efficiency • Rapid response • Targeted Group Interventions • Some students (at-risk) • High efficiency • Rapid response • Universal Interventions • All students • Preventive, proactive • Universal Interventions • All settings, all students • Preventive, proactive 80-90% 80-90% Multi-tier Model • Intensive, Individual Interventions • Individual Students • Assessment-based • Intense, durable procedures 1-5% 1-5% Attention, Effort, Precision 5-10% 5-10%

30. Capacity Development State Management Team State Transformation Team Regional Implementation Teams © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

31. State Management Group State Transformation Team SISEP Support & 2 FTE Regional Implementation Team N = 50 – 200 Schools Regional Implementation Team N = 50 – 200 Schools Regional Implementation Team N = 50 – 200 Schools Regional Implementation Team N = 50 – 200 Schools IMPLEMENTATION CAPACITY FOR SCALING UP EBPs

32. Policy Enabled Practice (PEP) System Change SISEP Support & 2 FTE Practice Informed Policy (PIP) First Regional Implementation Team N = 9 Staff State Management Group

33. Context • Many initiatives to improve education • Initiatives too often conflicting and/or competing Wraparound Literacy Applied Behavior Analysis Early Intervention Family Support Positive Behavior Support Response to Intervention Math

34. Using RTI to Align Systems Response to Intervention/Prevention Primary Prevention Universal Screening Multi-tiered Support Early Intervention Progress Monitoring Systems to support practices Early Intervention Literacy Wraparound ALIGNMENT Math Family Support Behavior Support Student Outcomes © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

35. Implementation Team • A group that knows the innovation very well (formal and practice knowledge) • A group that knows how to implement that innovation with fidelity and good effect • A group that accumulates data & experiential knowledge -- more effective and efficient over time (information economics, K. Arrow) © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

36. Implementation Team Prepare schools faculty, staff Prepare Communities Implementation Team Prepare Districts Work with Researchers Assure Implementation Assure Student Benefits © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

37. Capacity Development Start with too many overqualified people “Generation 1” RIT members become: Generation 2 expanded RIT members Generation 3 State Transformation Team leaders Generation 4 State and federal officials © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

38. State Management Group State Transformation Team 2 FTE & Others SISEP Support Regional Implementation Team N = 50 – 200 Schools Regional Implementation Team N = 50 – 200 Schools Regional Implementation Team N = 50 – 200 Schools Regional Implementation Team N = 50 – 200 Schools IMPLEMENTATION CAPACITY FOR SCALING UP EBPs [Phase 2]

39. Saturation 44/51 Intensive Development Saturation Intensive Development

40. Coaching Defined • Coaching is the active and iterative delivery of: • (a) prompts that increase successful behavior, and • (b) corrections that decrease unsuccessful behavior. • Coaching is done by someone with credibility and experience with the target skill(s) • Coaching is done on-site, in real time • Coaching is done after initial training • Coaching is done repeatedly (e.g. monthly) • Coaching intensity is adjusted to need

41. Outcomes of Coaching • Fluency with trained skills • Adaptation of trained concepts/skills to local contexts and challenges • And new challenges that arise • Rapid redirection from miss-applications • Increased fidelity of overall implementation • Improved sustainability • Most often due to ability to increase coaching intensity at critical points in time.

42. 10% 5% 0% 30% 20% 0% 60% 60% 5% 95% 95% 95% Joyce & Showers, 2002

43. Coaching within SWPBS Implementation • Context: • 9600 schools implementing SWPBS nationally • Defining the Role • Internal vs External • Selecting Coaches • Training and support for coaches • Assessing Impact

44. Rehearsal and Coaching • After initial training, a majority of participants (211 of 213) demonstrated poor implementation. Decision-makers should pair training prior to implementation with on-going rehearsal and performance feedback (coaching) • Test et al 2008 © Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

45. Example of the Impact of Coaching on Student Outcomes:Average Major Discipline Referrals per Day per Month Coach returns from leave

46. Visibility Funding Political Support Leadership Team Active Coordination Training Coaching Behavioral Expertise Evaluation Local Demonstration Schools

47. “Even if you’re on the right track, you’ll get run over if you just sit there” - Will Rogers