Scaling up and Sustaining Evidence-based Practices

1. Scaling up and Sustaining Evidence-based Practices Glen Dunlap, George Sugai, Tim Lewis, Steve Goodman, Rob Horner www.pbis.org

2. Goals • Define the features and procedures for moving evidence-based educational practices from demonstrations to large-scale adoptions. • Use School-wide Positive Behavior Support as one example of large-scale implementation

3. Main Themes • To take educational innovations to scale begin with Valued Outcomes • The outcomes need to be valued • The outcomes need to be comprehensive

4. Phases in Scaling of Evidence-based Practices Emergence Establishing Demonstrations/Capacity Elaboration Systems Adoptions

5. Phases of Implementation • Emergence • Define Innovation with precision • Define Supporting Systems • Define Implementation Process • Awareness dissemination • What is the innovation? • Is it evidence-based? Is it conceptually coherent? • Why is it effective? • How is it more efficient than what we currently do?

6. Phases of Implementation • Demonstration • Documentation that innovative can be implemented locally with (a) fidelity, and (b) effect on valued outcomes. • Provide demonstrations (1-50) • Repeated demonstrations in multiple contexts (parts of the state, urban centers, different grade levels) may be needed. • Demonstrations typically are done at greater expense than is sustainable or scalable, but are justified as examples that the innovation “can be done here” • Build infra-structure for scaling • State policy • State training and support capacity • Information systems

7. Phases of Implementation • Elaboration • Shift from demonstration to broad implementation • Use local trainers • Presentation by local demonstration sites • Many distributed (more cost effective) trainings • Training at multiple organizational levels • Administrators • School boards • Instructional staff • Specialists (e.g. behavior specialists, school psychologists, social workers, counselors) • Families • Disseminate outcome data • Conduct and disseminate comparative cost data

8. Phases of Implementation • System Adoption/Sustainability • Innovation is integrated into policy • Job descriptions • Hiring announcements • Annual personnel orientation • Regular reporting of data • Are we implementing evidence-based practices • Are we producing the effects we want for children • Investment in continuous regeneration • Implement  evaluate  adapt

9. Sustaining SWPBS Implementation Jennifer Doolittle University of Oregon 2006

10. Method • 285 schools who have been involved in implementing SWPBS for at least 3 years. • 71 not to criterion yet • 74 met 80%/80% criterion on SET but did NOT sustain for two years. • 140 met 80%/80% criterion on SET and did sustain. • School-wide Evaluation Tool • Logistic Regression

11. Doolittle (2006)

12. Results: Predictors of Implementation Effect sized used is d-statistic and interpreted as .2 = small effect, .5 = medium effect, .8 = large effect (Cohen, 1988). *

13. Results: Predictors of Sustained Implementation Effect sized used is d-statistic and interpreted as .2 = small effect, .5 = medium effect, .8 = large effect (Cohen, 1988). * *

14. Summary • The variables that were most relevant for initial implementation were DIFFERENT from the variables that affected sustainability.

15. Summary for Sustaining and Scaling • Begin with the Valued Outcomes • Innovations need to be more than effective: • Comprehensive • Efficient • Research-based • Dramatic improvement over what already exits. • The process of implementation changes as the scale increases • Increased efficiency • Increased emphasis on local capacity • Large scale implementation requires sustained effect • Continuous regeneration.