Sustainability Through the Looking Glass: Shifting Contingencies Across Levels of a System

1. Sustainability Through the Looking Glass: Shifting Contingencies Across Levels of a System Jack States Randy Keyworth Ronnie Detrich

2. Sustainable Practice A practice that has been adopted, implemented, and reliably maintained over time and generations of practitioners and decision makers. • average life of an education innovation is 18-48 months (Latham, 1988)

3. Why Initiatives Fail? Political support Funding Competing reforms Leadership Stability Insufficient training Faculty turnover Model specificity Faculty commitment Sustained professional development Schools’ past & current performance Positive student outcomes …no one risk is statistically significant…combinations of risk factors “Sustainability: Examining the Survival of Schools’ Comprehensive School Reform Efforts” American Institute for Research

4. What do we know about the science of “sustainable” implementation? • Maximize the variables that support implementation of the intervention • Minimize the variables that oppose implementation of the intervention • Align the contingencies at all levels of the system • Recognize and manage the different contingencies that exist at the different stages of implementation

5. What do we know about maximizing and minimizing the variables? Manage Performance Implementation is a social / cultural change process • As a social process, success often has very little to do with the details or merits of the actual innovation… • Which is why poor innovations are adopted more frequently than good innovations

6. How do we Manage Performance? OBM is Demonstrating That Managing the Contingencies for Groups is Much Like Managing Contingencies for Individuals Contingencies: a contingent relationship between an individual’s behavior and the outcome of that behavior that affects the practice’s subsequent probability. Example: • Behavior:Student A failing reading • Intervention: Individual tutoring after school • Outcome:Student A achieves grade level on reading test Meta-contingencies: a contingent relationship between a cultural practice and an outcome of that practice that affects the practice’s subsequent probability. Example: • Performance: High Stakes test results reveal students from School A are not meeting the minimum standards in reading • Intervention: Align the schools curriculum with the test standards • Outcome: Students enrolled in School A improve reading performance to minimum state standards

7. Achieving Sustainable Implementation for the Great Numbers Requires System Change “Build and manage an effective organization” “Hire and manage people who will be effective in doing jobs within the organization.”

8. CORE COMPONENTS OF IMPLEMENTATION Establish OUTCOMES, GOALS, AND MEASURES • Select the relevant goals (program, practice, fiscal, staffing) • Establish objective and measurable outcomes and align levels Employ and Align PERFORMANCE MANAGEMENT strategies • Build activities and systems • Recruitment and hiring • Performance expectations • Training • Consultation and coaching • Feedback and evaluation • Manage contingencies Conduct frequent and on-going MONITORING • Outcome and process • Assure program fidelity (program level) and treatment integrity (practice level) Be prepared to ADAPT AND INNOVATE • Evidence-based practices • Data-based decision making

9. Monitoring • Monitoring involves observing a behavior for any changes that my occur over time, or for effects an intervention may have on the observed behavior. • Evaluate success of the program or intervention against goals • Assess program fidelity and treatment integrity • Monitoring generates information that is essential when making data-based decisions. • Outcome measures • Process measures • Monitoring should occur at all levels of implementation: • Organizational (implementation) • Practitioner (intervention) • Consumer

10. Cultural Practices School A: Requires ten new vocabulary words to be introduced each week, to be used and spelled correctly in writing assignments. Individual Behavior Teacher A: Has parents review spelling test results with child Individual Behavior and Cultural Practices Contingency Alignment Desired Outcome School exceeds state expectations for student spelling

11. Federal State District School Teacher Levels of Implementation Contingency Alignment Student Education Outcomes Achieved

12. Federal State District School Teacher Level of RtIImplementationCalifornia IDIEA permits use of RtI California has no mandate or requirements for RtI A committee to study RtI has been formed - no policy Non-Alignment School special ed staff form team to review evidence based practices Does not believe that science is best means to judge effectiveness of practices Desired Outcomes Unlikely RtI Is implemented piece meal and sporadically across the state

13. Alignment of Core ImplementationComponents Across Levels

14. Why Do So Many Programs Fail? They do not plan for and address the different contingencies required of each stage

15. Adoption & Exploration Program Installation Initial Implementation Full Operation Long Term Operation Stages Of Implementation Over Time and Over Generations Desired Outcome Sustainability

16. Stages Of Implementation • Adoption & Exploration • Assess the fit • Utilize data based decision-making / consensus building • Achieve cultural integration • Program Installation (start-up) • Create new outcome expectations • Establish and/or redesign the infrastructure • Construct or redesign reporting systems (monitoring) • Ensure funding streams • Develop or modify policies • HR strategic development • Train and/or retrain personnel • Initial Implementation (performance change) • Initiate new practices and performance • Troubleshoot obstacles • Performance feedback systems operational • Adapt systems and apply to novel situations

17. Stages of Implementation • Full Operation (Integration) • HR functions operating (staffing and re-staffing) • Training and re-training • Treatment provided proficiently and with integrity • Managers supports and facilitates the new practice • Policies and procedures adapted to experience • Performance management systems support outcomes • Reporting systems functioning (monitoring) • Long-Term Operation • Practice refined and treatment integrity of practice maintained • Undesirable drift controlled • Innovations adopted and incorporated into the practice • Core practices and outcomes monitored and effectiveness sustained over time

18. Types of Change Planned and Formal • Adaptation: Changes to an intervention that effectively address issues unique to the operating environment including assuring a cultural fit. These changes do not alter the core components of the practice and do not modify targeted outcome(s).Adoption, Program Installation, and/or Initial Implementation. • Innovation: Changes that offer opportunities that improve and expand upon an intervention above what has been achieved by current practices and procedures. Innovations to a practice should not be attempted before treatment integrity is first attained. Full operation and/or Long-term operation Unplanned and Informal • Drift: Undesirable changes that are identified as threats to the treatment integrity of the practice as defined by the core practices and outcome(s). Full operation and/or Long-term operation

19. Core Components CORE COMPONENTS: The essential elements necessary to achieve the desired outcomes for the consumer. • Not knowing the core components leads to time and resources wasted on attempting to implement a variety of non-essential elements. • Knowing the core components are essential to answering critical questions required for adaptation. Examples • Differential reinforcement • Phonics instruction • Reading fluency • Systematic de-sensitization • Coaching • Use of evidence-based practices

20. Practices DEFINITION PRACTICES: Skills, techniques, and strategies, that can be used by practitioners for interventions that benefit consumers. Such practices reliably produce desirable effects and can be used individually or in combination to form more complex practices and/or programs. Examples • Direct Instruction (DI) • Positive Behavior Interventions (PBI) • Good Behavior Game • RtI • New Chance, (for young welfare mothers) • Teaching family model

21. Programs DEFINITION PROGRAMS: Collections of practices for a defined consumer base implemented through an organizational structure using a specific philosophy, value system, service delivery structure, particular funding sources, and core practice components. Programs represent an efficient method to translate practices into effective treatment outcomes. • Boys Town • Morningside Academy • Institute for Effective Education • Spectrum Center • Oakland Unified School District

22. Programs - Practices - Components

23. Sustainable Implementation RequiresSuccessful Aligning andManagement of the Contingencies • Across levels of the system (Fed, State, district, school, classroom) • Across the core components of implementation (goals, performance management systems, monitoring, and innovations) • Across the stages of implementation (adoption, installation, initial implementation, full operation, long-term) • Across the Intervention (program, practices and core components)