1 / 50

From Model Demonstration to Sustained Implementation of Evidence-Based Practices

From Model Demonstration to Sustained Implementation of Evidence-Based Practices. Pat Gonzalez, Ph.D. MDCC Project Officer, OSEP Debbie Shaver, Ph.D. MDCC Co-Principal Investigator, SRI International Ed Shapiro, Ph.D., Lehigh University Suzanne Robinson, Ph.D., University of Kansas

ramona
Download Presentation

From Model Demonstration to Sustained Implementation of Evidence-Based Practices

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. From Model Demonstration to Sustained Implementation of Evidence-Based Practices Pat Gonzalez, Ph.D. MDCC Project Officer, OSEP Debbie Shaver, Ph.D. MDCC Co-Principal Investigator, SRI International Ed Shapiro, Ph.D., Lehigh University Suzanne Robinson, Ph.D., University of Kansas Mary Wagner, Ph.D. MDCC Principal Investigator, SRI International

  2. Model Demonstration Coordination Center (MDCC) Debbie Shaver MDCC Co-Principal Investigator SRI International

  3. OSEP's model demonstrations and MDCC • Since 1970’s OSEP has funded model demonstrations projects (MDPs) to better understand how to improve outcomes for children and youth with disabilities. • Since 2005, OSEP has funded MDCC to document and learn from the model development and implementation process and outcomes of eight cohorts of MDPs.

  4. MDCC Roles MDCC works with cohorts of MDPs to: • Facilitate collaborative partnerships to learn and share. • Help establish consistent design elements to deepen evidence base for models. • Synthesize findings across MDPs to identify factors that lead to high-quality implementation, sustainability, and wider adoption of evidence-based practices • Identify key issues in translating research to practice to improve outcomes for children and youth

  5. Active Cohorts of MDP Grantees Cohort 4:Tiered writing approaches for secondary students Cohort 5:Tiered interventions for English language learners Cohort 6: Assistive technology for young children Cohort 7:Reentry of students from juvenile justice facilities Cohort 8:Stepping up technology implementation

  6. Previous Cohorts of MDP Grantees Cohort 1: Progress monitoring in elementary reading Cohort 2:Tertiary behavior interventions Cohort 3:Early childhood language development

  7. Framework for understanding model implementation and outcomes

  8. MDCC evaluation questions [1 of 3] Intervention Components • How do core intervention components change over time in response to implementation experiences, and how does this process differ across MDPs and cohorts? • How do variations in the core intervention components relate to fidelity of intervention, acceptance of the model, and establishment of conditions for sustaining the intervention? Implementation Components • How do variations in core implementation components relate to fidelity of implementation, acceptance of the model, and establishment of conditions for sustained implementation?

  9. MDCC evaluation questions [2 of 3] Destination Organizations • How do variations in the destination organizations (e.g., population served, organizational functioning, staff and leadership, climate/culture, resources, support for the model) relate to fidelity of implementation, acceptance of the model, and establishment of conditions supportive for sustained implementation? Context • How do variations in contextual factors relate to fidelity of implementation, acceptance of the model, and establishment of conditions for sustained implementation?

  10. MDCC evaluation questions [3 of 3] Outcomes • How do variations in core intervention and implementation components, destination organizations, and contexts relate to: • Implementation outcomes (changes in practice/functioning in the destination organizations, and changes in the skills, attitudes, and beliefs of staff) • Child/youth outcomes • Systems-level outcomes?

  11. Following up on implementation experiences MDCC: • Analyzes data provided by MDPs to address evaluation questions throughout model implementation. • Reports results of analyses and lessons learned to OSEP at implementation conclusion. Then OSEP asks, • What happens to the models after the MDPs exit their sites? • What promotes and hinders model sustainability there and spread to other sites?

  12. Follow-up studies • OSEP commissioned MDCC to facilitate early-cohort MDPs in returning to their sites to document: • The core intervention components of the models that were still in place in their original schools and districts • as originally implemented, • adapted, or • abandoned. • The extent to which the models, in whole or in part, had been implemented outside the original schools and districts.

  13. MDCC follow-up activities • Develop qualitative templates for MDPs to use in reporting what they found. • Document MDPs’ data sources and collection methods. • Compile follow-up data collected by MDPs. • Facilitate conversations about the factors MDP staff believed helped and hindered model sustainability and spread. • Synthesize the findings and report to OSEP.

  14. MDCC information and products • Follow-up reports from the first two cohorts can be found at: Model Demonstration Center Website's 'Products and Reports' page: http://mdcc.sri.com/prod_serv.html • Also available: • Descriptions of cohorts and projects • Cohort summary reports • “Briefs” on implementation- related topics • Contact information

  15. What Stays, What Goes, and Why?Sustainability of Model Demonstration Project MP3 Edward S. Shapiro, Ph.D. Director, Center for Promoting Research to Practice ProFessor, School Psychology Lehigh University

  16. Agenda • Brief description of Project MP3 • What remained two years later. • What changed (enhanced/diminished) • Key factors identified by the site in sustainability • Technical factors (e.g., instructional practices, financing systems), • Human factors (e.g., skill levels, communication patterns, staffing changes), • Contextual factors (e.g., policy shifts, demographics)

  17. MP3 Objectives • Project MP3 documents the implementation of a school-wide progress monitoring model in five elementary schools across Pennsylvania. • The model provides general and special education teachers (K- 4) with the necessary support and expertise to use progress monitoring data to determine students’ needs for reading intervention and to develop a Response-to-Intervention framework for special education referral.

  18. Objectives • Result in increased reading achievement for students in participating schools • Equip schools to prevent reading failure by • “Catching” students falling behind • Introducing intensive, differentiated, scientifically-based interventions that alter students’ trajectories • Document school adoption of new practices

  19. Project MP3 • Large district (by PA standards), 11,750 students • 13 elementary, 4 middle, 2 high schools • Funded in fall 2005, 9 month planning period • Implementation began fall 2006 in one school • Implementation in all 3 schools in 2007 • All project support ended in June 2009 • Revisited for sustainability in April 2011 • Continued interaction with district currently

  20. Participating Schools School A

  21. Project MP³ Model TIER 1: All students in core program (Everyone taught reading from H-M) Fall Benchmark Student benchmark score = BENCHMARK (90% will do fine) TIER 1 Enrichment 30 min 4-5x week PM 3x/year Student benchmark score = STRATEGIC (Might be at risk) TIER 2Intervention 30-45 min 4-5x week PM every other week Student benchmark score = INTENSIVE (Definitely at risk) TIER 3 Intervention 30-45 min 4-5x week PM 1x week TIER 1: All students in core program (Everyone taught reading from H-M) Winter Benchmark

  22. When we were finished…. • Benchmarking & progress monitoring processes • Data based decision making team processes at school (administrative) and grade (teacher) level • Multi-tiered system of supports (MTSS) • Tools for managing MTSS • Tools for analyzing MTSS • Fidelity of instruction and intervention assessments • Parent engagement

  23. Enhancements two years later: Technical factors • Fall benchmark for kindergarten push from September to October • Jump start program – interventions began before fall benchmark for students identified from previous spring in need of intensive intervention. • Grade level meetings in one school alternate between all grades meeting together at the same time one month, followed by focused and individual grade levels meeting the alternate month. • Data collection tools enhanced

  24. Enhancements two years later: Technical factors • Evaluation of fidelity continued, mostly by principal • Added period of intervention for intensive students • Expansion to all 13 elementary schools • Expansion to 1 middle school (selected as pilot site) • Two schools added PBIS model • Switch from DIBELS to AIMSweb • Continuation of parental engagement process (letters) • Desigated point of contact (teacher) for each grade level • All schools applied for and approved for using RTI for SLD determination (state approval process)

  25. Enhancements two years later: Human factors • Replication of model at all schools • Model embraced as part of school culture • Minimal teacher turnover • Schools examined data outcomes and focused on enhancing core instruction

  26. Challenges: Contextual factors • Resource reductions • Administrative staff changes, key leaders retired • Superintendent (twice) • Director of C & I- elementary • Director of Special Ed • Director of Pupil Services • Several supervisors • Principals at two of three MP3 schools

  27. General comments: Sustainability • Leadership and commitment • Culture shift, embedded perspectives • Approval at state level for SLD determination use • Maintenance of resources in light of budget cuts • Planned from the start • “What will be there when we leave?” • “Can the staff continue without our support?” • Fade and scaffold

  28. Two years later • Continuation despite staff changes • Maintenance and routine part of school cultures • Support at all levels of district from central to local • Hiring includes knowledge and experience with RTI

  29. Thanks • Dr. Ed Shapiro • Email: ed.shapiro@lehigh.edu

  30. RtI and Writing Instruction in High Schools Suzanne Robinson Associate Professor of Special Education University of Kansas

  31. Secondary school RtI (tiered support) context • Background for this “story” • Model demonstration project funded by OSEP 2010-2013 • RtI focus on 9th grade, expanded to 10th grade • Focus on writing instruction • Strategic writing program developed at KU-CRL for core instruction; Tier 2 adapted, focused instruction; Tier 3 intensive, personalized instruction • Multiple baseline design • Year 1 in one urban district (ended relationship). Year 2 in district with diverse population in different state. Year 3 added another high school. Year 4 focus on building sustained practices. Cohort 6 Model Demonstration Projects

  32. Defining the Tiered Support Model Tier 1 1. Strategic Writing as 50% of the 9th English Language Arts curriculum (supported in the other disciplines) Tier 2 • Small groups (some writing groups, some literature groups) • Co-teaching classes (2 teachers = 2+ groups) Tier 3 (trials still underway on what works) • Writing class/lab taught by ELA teachers • Writing class/lab taught by Special Education teachersresurrected • SPED Resource classes with students meeting IEP goals/Tier 3 needsresurrected • Self-contained special education classes and ELL classes • Co-teaching special education teacher (during resource period) sometimes yes/sometimes no • Peer tutors (yet to be implemented)

  33. Impact of program on student performance (2011-2012) Cohort 6 Model Demonstration Projects

  34. Impact of program on student performance (2011-2012) [1 of 7] Cohort 6 Model Demonstration Projects

  35. Impact of program on student performance (2011-2012) [2 of 7] Cohort 6 Model Demonstration Projects

  36. Impact of program on student performance (2011-2012) [3 of 7] Cohort 6 Model Demonstration Projects

  37. Impact of program on student performance (2011-2012) [4 of 7] Cohort 6 Model Demonstration Projects

  38. Impact of program on student performance [ 5 of 7] Cohort 6 Model Demonstration Projects

  39. Impact of program on student performance (2011-2012) [6 of 7] Cohort 6 Model Demonstration Projects

  40. Impact of program on student performance (2011-2012) [7 of 7] Cohort 6 Model Demonstration Projects

  41. Common Implementation Challenges in Secondary School Tiered Support Models • Staff capacity • Fidelity • Accountability • Scheduling • Resources • Beliefs about roles and instructional responsibility _____________ • Negotiated contract constraints • Other school reform initiatives

  42. Sustaining … threats and strategies • District leadership, building leadership, and school staff must want the change and must recommit as often as needed as their understanding of what change entails grows and reconsider what systemic adoption entails (scaling up). • The students in the school have to need what the reform offers in ways that matter. • The district, school, and teachers must be able to assist in implementation and set up structures to practice, maintain, and sustain the reform (model components, roles on leadership teams, PD/coaching, accountability management). • Solutions need to be systemic and they need to be as simple as possible.

  43. Lessons learned in tiered support systems in academic arenas with secondary schools. . . • Must come to consensus about what is absolutely necessary for accelerating achievement for all students in terms of: • Type of instruction needed • Necessity and possibility of mastery • How to facilitate mastery • Duration of instruction in the context of other demands • How to provide Tier 3 supports in the context of credit/scheduling demands • Who will provide Tier 3 support and how to facilitate timely communication between Tier 1 and 3 teachers • How to engage all teachers (across departments) and support or specialized service personnel (ELL, SpEd, remedial, etc.) Cohort 6 Model Demonstration Projects

  44. In conclusion • Technical considerations in sustaining effective practices • Contextual considerations in sustaining effective practices • Human considerations in sustaining effective practices Almost always, stakeholders in school-change projects have different, sometimes competing, as well as compatible interests in the outcomes of any reform effort. The interplay between compatible interests and different and/or personal interests can cause tension; these tensions compete for attention and can cause relational problems among partners in change. Cohort 6 Model Demonstration Projects

  45. What Promotes and Hinders Model Sustainability? Mary Wagner MDCC Principal Investigator SRI International

  46. Implement well Model demonstration projects (MDPs) with strong implementation had: • “Rolled” with the surprises, working with their sites where they found them. • Focused on changing: • The knowledge, attitudes, skills, and behaviors of key adults at all levels of the system. • Formal and informal organizational structures and cultures. • Relationships with stakeholders. • Built capacity for change at all levels. • Sold their sites on the importance of collecting and using data. • Produced evidence of impacts ASAP. • Actively learned from their experiences and adapted as they went.

  47. Supports for sustainability and spread • Multi-level leadership • Compatibility with ongoing systems; adapting to fit. • Improved practice and outcomes • They also improve administrator and practitioner efficacy. • Staff stayed committed over time. • Leaving behind useful, affordable tools (e.g., assessment materials, meeting protocols) • Increasing and relying on system-level supports • District policies and resources drove sustainability. “It’s just the way we do things now.” • State-level TA systems were critical for spread.

  48. Challenges to sustainability and spread • Staff turnover; capacity doesn’t stay built • The antidote: a culture of good practice. • Competing initiatives. • The antidote: use data to make the case; don’t forget the power of parents. • Low salience of the problem being solved • The antidote: use data to make the case; don’t forget the power of parents. • Budget cuts. • The antidote: creative repurposing of resources; “strong fingernails.”

  49. What’s next? • Cohort 3 (early childhood language promotion models) final report coming soon. • Cohort 3 submits data from its follow-up study to MDCC in August; report will follow. • MDCC is preparing a “brief” on issues involved in MDPs replicating and disseminating promising models. • Continued facilitation of cohorts 4 through 8. • Cohort 9 technology grantees starting up in the fall. • Learn more at http://mdcc.sri.com !!

  50. Questions??

More Related