200 likes | 296 Views
CMP Statewide Evaluation Measurement Plan Summary of HB1451 Outcomes Workgroup Efforts. Presentation to IOG Members. Goals of presentation. Provide background (rationale, legislative goals, objectives) for measurement plan Review the measurement plan and address questions Discuss next steps.
E N D
CMP Statewide Evaluation Measurement PlanSummary of HB1451 Outcomes Workgroup Efforts Presentation to IOG Members
Goals of presentation • Provide background (rationale, legislative goals, objectives) for measurement plan • Review the measurement plan and address questions • Discuss next steps
Measurement issues • CMPs vary widely in: • Outcomes • 144 indicators in 31 outcome areas in FY09-10 • Quality of indicators • Measurement processes • E.g., level of reporting (county vs. program vs. ISST) • Difficult to aggregate data across CMPs to demonstrate statewide impact on any of the legislative goals
Why we are doing this work • Make stronger statements about the Program’s efforts and successes • Improve the collection of outcome data • Improve service delivery and programs (ability to connect CMP practices with outcomes) • Provide evidence that collaborative management approaches lead to improved child and family outcomes • Promote greater likelihood of ongoing support at the state level
Outcomes Workgroup activities • Discussed ways to improve measurement processes • Identified CMP core components for measurement • Developed a measurement approach to implement across CMPs • Selected process and outcome indicators in child welfare and juvenile justice areas • Surfaced issues, challenges, solutions
Improving measurement: Alignment of interdependent relationships Population Program Model Outcomes
Population • The intended core target population of the CMP is children and families who are currently involved, or at risk for involvement, in multiple service agencies
Program Models • IOGs • Specified participating service agencies • Includes family advocacy organization • Focus is on establishing ISST • Developing shared processes • ISSTs • Development of a single plan that integrates services across multiple agencies • Family involvement in service planning
Outcomes • Key process outcomes • e.g., reduced duplication, cost shifting • Improvements on child and family outcomes • SSC outlined four domains • There are overarching goals • Ensure safety, stability, and well-being for children • Reduce further penetration into systems
Minimally, we need to be able to: • Accurately count number of CMP youth and families served by ISSTs • Show that CMP efforts are leading to positive outcomes across the state in at least a few key areas • Measurement plan was developed to meet these minimum requirements • Recognize that this does not capture all of CMPs’ efforts
Measurement Plan • Measures will focus on children and families referred to and served by ISSTs • A minimum set of defined process indicators will be uniformly measured across all CMPs • A small set of CW and JJ outcome indicators will be measured on relevant cases • Education and health/mental health outcome indicators will be defined next year
Overview of data collection • Process indicators will be collected through individual client-level database • ETO (web-based system) • Or modified local database (provide dataset) • Child welfare and juvenile justice outcomes will be collected from state data systems, where possible • Collected by OMNI at end of fiscal year • Annual reporting will continue in ETO
Process Indicators • Number of families referred to and served by ISST and date of ISST meeting • Type of outcome domain each family is enrolled under (CW, JJ, etc.) • Number and type of agencies or systems • Participating in the family’s service planning • Providing services to families • Number of families participating in service planning • Whether or not an integrated treatment plan was developed • Date the case was closed and/or end of service delivery • Outcomes (from selection of common outcomes) that are aligned with case
Process indicators: Data collection • Coordinators collect process data from whoever runs ISST meeting • Enter data into ETO sometime after the first ISST occurs • Coordinators will establish processes to: • Locate and record existing client IDs in Trails and Eclipse
Outcome Indicators - CW • Number (rate) of CMP youth with new open involvements in Trails after CMP services began • Number (rate) of CMP youth with no substantiated abuse finding after CMP services began • Number of moves that CMP youth experience when in out-of-home placement • Number (rate) of CMP youth discharged to a permanent home
Outcome Indicators - JJ • Number (rate) of CMP youth who successfully complete probation and/or parole • Number (rate) of revocations by technical violations where case resulted in unsuccessful termination among CMP youth • Number (rate) of CMP youth who recidivate, resulting in unsuccessful completion of probation and/or parole
Measurement of other legislative outcomes • Annual report and brief mid-year report in ETO (no more quarterly reports) • Strategies to reduce duplication and fragmentation, involve families, etc • Costs and cost savings • Local performance measures • Yearly collaboration surveys to IOGs
Next steps – short-term May-June • OMNI will build and train CMPs on ETO database July • CMPs will enter FY10-11 annual report data into ETO • OMNI will work with CMPs who choose to modify local databases • CMPs will begin tracking process indicators on ISST-referred families
Next steps – long-term FY11-12 • Develop plan to identify education and health/mental health outcome indicators • Identify steps to explore use of data in incentive formulas • Continue to streamline annual reporting
Questions? • What is needed for our discussion about MOU indicators for next year? • What kind of processes do we need to put in place so that we are collecting the data that needs to be collected?