1 / 46

Outcome Measurement System for CACs in Tennessee

Outcome Measurement System for CACs in Tennessee . Agenda. Background Administrative Guide- Survey Administration Administrative Guide- Score Sheets Data Analysis Final questions. Materials. Hard Copies Administrative Guide Disk Administrative Guide Surveys Score sheets

chessa
Download Presentation

Outcome Measurement System for CACs in Tennessee

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Outcome Measurement System for CACs in Tennessee

  2. Agenda • Background • Administrative Guide- Survey Administration • Administrative Guide- Score Sheets • Data Analysis • Final questions

  3. Materials • Hard Copies • Administrative Guide • Disk • Administrative Guide • Surveys • Score sheets • PowerPoint presentation

  4. Background • Membership Priority – Evaluation and Assessment CACTX maintains its commitment to an outcome based model and will initiate proactive efforts to conduct a comprehensive evaluation project designed to establish shared, meaningful outcomes for local CACs and for the statewide movement. 2006 2007 2008 2009 2010 2011

  5. What is out there? • NIJ Special Report • NCAC Cost-Benefit Analysis • UNH Multi-Site Evaluation of CACs • Various studies regarding individual components 2006 2007 2008 2009 2010 2011

  6. Policy Research Project 2006 2007 2008 2009 2010 2011

  7. Types of Evaluation • Program Monitoring Evaluation • Outcome Evaluation • Impact Evaluation

  8. Logic Model Zone 1 Zone 2 Zone 3 INPUTS ACTIVITIES OUTPUTS OUTCOMES IMPACT Accounting and Administration Performance Management Research *From Peter Frumkin, RGK Center Director Power Point Presentation – CACTX Annual Conference 2009

  9. Literature Review T H E M E S I N D I C A T O R S Focus Group Instrument Analysis Project Activities

  10. Project Activities • Orientation: • Introduction to CAC world • Tour of local center • Literature Review: • CAC Model and Outcome Measurement • Focus Groups: • 3 Focus Groups with CAC Directors • Instrument Analysis: • National Institute of Justice Study • Collected tools from CACs • Outcome Measurement System: • Develop and Review • Report/Final Product: • Final Deliverable to CACTX

  11. Literature Review – CAC Model • Importance of support networks, particularly non-offending caregiver for child recovery • Effectiveness of using MDT approach for handling child abuse cases • Implications of quality forensic interviews • Difficulty of comparing research on prosecution outcomes

  12. Literature Review-Outcome Measurement Definition Internal Uses External Uses

  13. Focus Groups – November 2008 Participants • Small CAC: 10 • Mid-Size CAC: 12 • Large CAC: 13

  14. Questions • 3- 5 important indicators that you collect. • Indicators you wish you could collect. • Indicators you are collecting for other funders. • Any concerns/recommendation that the team should be aware of.

  15. Assessment Instrument Analysis

  16. Example: Child Friendly Facility Instruments

  17. Example: Child Friendly Facility Instruments 6 instruments, 63 questions, 16 question categories

  18. Local CAC Instruments Total of 40 forms collected Modified cross tabulation

  19. Results of Analysis • Identification of key themes and concepts • Support for indicator phrasing • Development of outcome statements

  20. Indicator Categories • MDT • Child & Family Satisfaction • Justice • Community • CAC Organization

  21. Considerations • Developing an effective system for all CAC types • Developing a system that could be implemented with little added stress • Developing a system within the real world constraints: • Time, resources, evaluating children over a period of years, nature of subject matter

  22. Considerations • Keep the system simple, clear, uncluttered • Make administration as least taxing on resources as possible • Focus on programmatic level outcomes and the role of the CAC in the process • Trust in the aspects of the model verified by the Literature Review

  23. Outcome Measurement System OMS

  24. Children’s Advocacy Center Outcomes Primary Goals of CACs in Texas • Outcome #1: The Children’s Advocacy Center facilitates healing for the child and the caregivers.  Minimize re-victimization of child victims and their supportive family members throughout the investigative and prosecutorial stages of their cases and beyond • Outcome #2: The team approach results in more collaborative and efficient case investigations.  Facilitate prosecution of perpetrators through effective fact finding and strong case development.

  25. OMS Development Client Focused Indicators • Client’s needs are being met • Client is satisfied with the services received • Client has a positive CAC experience

  26. OMS Development MDT Focused Indicators • CAC supports the needs of MDT members • MDT team is committed to the mission • CAC facilitates effective collaboration between team members

  27. OMS Development

  28. Validity Validity is the strength of our conclusions, inferences or propositions. More formally, Cook and Campbell (1979) define it as the "best available approximation to the truth or falsity of a given inference, proposition or conclusion

  29. OMS Development Two Client Surveys • Initial Survey • Follow-up Survey Two Multi-Disciplinary Team Surveys • General Survey • Case Specific Survey One Needs Assessment Form • An output based form used for milestone analysis

  30. Sample Survey Questions Initial Survey General Survey

  31. Individual Client Needs Assessment

  32. OMS • Importance of “Scoring” • Need to turn our outcomes into data so we can evaluate performance • Scores allow for internal target setting • Gives us information we can communicate to the outside

  33. Does it work?

  34. The Pilot Test • Pilot Test Centers • Governed by two constraints: • Easily accessible from the Austin area • Number of clients served considered to ensure optimal level of data collection • Diverse centers to represent the differences between the 64 centers in Texas 2006 2007 2008 2009 2010 2011

  35. Reliability • Cronbach’s alpha • A statistical method to ensure the reliability of the instrument. • The data gathered from pilot test was used to ensure the reliability of the instrument.

  36. Pilot Rollout • OMS surveys in English and Spanish and score sheets on CD • Administrative guide • Detailed instructions • Phone Interview Script • Technical Support for implementation • Agreement

  37. FY10 (March - August 2010) 2006 2007 2008 2009 2010 2011

  38. Outcomes for CACs in Texas • The Children’s Advocacy Center facilitates healing for the child and caregivers. • 90% of caregivers felt that CAC’s facilitated healing for the child and themselves. • The multidisciplinary approach results in more collaborative and efficient case investigations. • 95% of MDT members felt their approach resulted in more collaborative and efficient case investigations.

  39. FY11 (Sept. 2010 - August 2011) 2006 2007 2008 2009 2010 2011

  40. Caregiver Initial Survey Results

  41. Caregiver Follow Up Survey Results

  42. MDT General Survey Results

  43. Individual Client Needs Assessment

  44. Implementation • November training • Sign agreement • Implementation – December 2011/January 2012 • Reporting- • July 15, 2012 • January 15, 2013 • July 15, 2013 • January 15, 2014

  45. Questions?

More Related