1 / 15

Implementation Evaluation: Colorado Practice Model and MPCWIC*

Implementation Evaluation: Colorado Practice Model and MPCWIC*. Implementation Stages, Process and Cohort I and II survey results. Presentation to the Program Operations and Implementation Team (POIT) February 2013 Julie McCrae University of Denver, Graduate School of Social Work

luke
Download Presentation

Implementation Evaluation: Colorado Practice Model and MPCWIC*

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Implementation Evaluation: Colorado Practice Model and MPCWIC* Implementation Stages, Process and Cohort I and II survey results Presentation to the Program Operations and Implementation Team (POIT) February 2013 Julie McCrae University of Denver, Graduate School of Social Work *Mountains and Plains Child Welfare Implementation Center

  2. Outline of today’s presentation • Overview • Evaluation Questions • Overall project results – implementation • Cohort I and II Readiness Surveys • Results of interviews and focus groups • Survey incentives and the schedule for individual reports

  3. Overview Colorado’s goal, with MPCWIC support, was to: “Improve the experiences of children and families served, through expanding organizational capacity of DCWS and improving service capacity at the county level”. • Activities: • Project Management • Integrate report recommendations • Internal strategic planning & restructuring • Practice Model design & implementation • Communication Plan • Research & Evaluation Agenda • Performance-based Outcome Measures 2009 - 2012

  4. Implementation Drivers Stakeholder Engagement Performance appraisal Staff selection Data Facilitative Admin Shared vision Training & Coaching Leadership Systems intervention Buy-in Readiness Effective Change

  5. MPCWIC Evaluation

  6. Results: Implementation over Time

  7. Readiness Survey Results • 1,495 surveys sent to staff in 30 agencies • 873 completed the survey (58%) • In 15 agencies, 75 to 100% of staff completed the survey • Completion rates were higher in Cohort II agencies

  8. Results: Cohort I and II reports of Buy-in at Baseline p<.01; bivariate t-test

  9. Results: Readiness for Change All scores differed significantly using bivariate t-tests, indicating that Cohort II staff report higher levels of readiness to change compared with Cohort I staff.

  10. Other scores *Cohort II staff report more favorable views and less job stress Note (last column): Cohort I scores were all slightly higher (more positive) at Time 2 but the differences did not reach statistical significance

  11. Interviews and Focus Groups • 4 agencies were randomly selected • 22 total sessions were held including 52 staff: • 6 director interviews • 5 caseworker focus groups (n=30) • 10 supervisor individual interviews • 1 mid-level manager focus group (n=6) • Questions centered on appeal and understanding of the CPM, strengths and challenges to implementation, and advice to Cohort II

  12. Director Themes “When I came to state child welfare, it really seemed like a top-down model for organization and decision making and change - and that’s like the worst thing.” Inclusivity “I think it’s really great to have the tools (fishboning, brainstorming)…We ended up places I never guessed we would go using those tools.” Utility “I worry at this point as far as maintaining it - this first group of people is getting fairly burned out on it – ‘Am I done? Can I get my one afternoon a month back?’” Sustainability and Next Steps

  13. Worker Themes Inclusivity “It’s a never-ending battle of trying to improve the services that we provide….I think this one is different because it involves more workers rather than just being told that this is the change we’ll be making.” “Oh no, here we go with more stuff (laughter)…I understood the reasons why, it made sense to me as to why we were doing it. Trying to envision what that would look like and how intensive it would be. On workers and their workload.” Workload “Clear communication, …From day 1,….With concrete reasons why,…And the presumed impact,…We don’t care if it’s a work in progress…or if it changes… Just don’t send out the email that says this is effective immediately.” Communication

  14. Advice to Cohort II Agencies “Let it play out, it will become clear.” – Agency Director • (1) Trust the process “Everybody’s got a role” – Supervisor • (2) Include all areas of the agency

  15. Discussion • Questions or clarifications • What is surprising to you? • Report dissemination • Incentives MPCWIC is made possible by the Children’s Bureau Contact information Julie McCrae Julie.mccrae@du.edu 303-871-4533

More Related