Achieving Our Mission

1. Achieving Our Mission Using Continuous Quality Improvement to Promote and Enhance Community Corrections Kimberly Gentry Sperber, Ph.D.

2. Efforts To Date • “What Works” Literature • Principles of Effective Interventions • Growing evidence based on individual program evaluations and meta-analyses • Continuing Gap Between Science and Practice • Few programs score as satisfactory on CPAI

3. Why Isn’t “It” Working?Latessa, Cullen, and Gendreau (2002) • Article notes 4 common failures of correctional programs: • Failure to use research in designing programs • Failure to follow appropriate assessment and classification practices • Failure to use effective treatment models • Failure to evaluate what we do

4. CPAI Data as Evidence of Fidelity • Lowenkamp and Latessa (2005) • Examined data from 38 residential correctional programs for adults • Looked at relationship between program fidelity and program effectiveness. • Program fidelity was assessed using the CPAI. • Found significant correlation between fidelity and effectiveness • CPAI scores correlated to reincarceration

5. Lowenkamp and Latessa FindingsContinued • Differences in recidivism rates based on CPAI scores: • Scores of 0-49% demonstrated 1.7% reduction compared to comparison group. • Scores of 50-59% demonstrated 8.1% reduction. • Scores of 60-69% demonstrated 22% reduction.

6. CPAI Data Continued • Holsinger (1999) • Examined data from Adolescent Community Correctional Facilities in Ohio • Looked at relationship between program fidelity and program effectiveness. • Program fidelity was assessed using the CPAI. • Outcome measures examined included any court contact, felony or misdemeanor, felony, personal offense, and commitment to a secure facility

7. CPAI Data Continued • Total composite score significantly correlated with all outcome measures. • Each individual domain of the CPAI also significantly correlated with all of the outcomes • Program Implementation • Client Assessment • Program Characteristics • Staff Quality • Evaluation

8. More Fidelity Research • Landenberger and Lipsey (2005) • Brand of CBT didn’t matter but quality of implementation did. • Implementation defined as low dropout rate, close monitoring of quality and fidelity, and adequate training for providers. • Schoenwald et al. (2003) • Therapist adherence to the model predicted post-treatment reductions in problem behaviors of the clients. • Henggeler et al. (2002) • Supervisors’ expertise in the model predicted therapist adherence to the model. • Sexton (2001) • Direct linear relationship between staff competence and recidivism reductions.

9. More Fidelity Research Cont’d. • Schoenwald and Chapman (2007) • A 1-unit increase in therapist adherence score predicted 38% lower rate of criminal charges 2 years post-treatment • A 1-unit increase in supervisor adherence score predicted 53% lower rate of criminal charges 2 years post-treatment. • Schoenwald et al. (2007) • When therapist adherence was low, criminal outcomes for substance abusing youth were worse relative to the outcomes of the non-substance abusing youth.

10. Washington State Example(Barnowski, 2004) • For each program (FFT and ART), an equivalent comparison/control group was created • Felony recidivism rates were calculated for each of three groups, for each of the programs • Youth who received services from therapists deemed ‘competent’ • Youth who received services from therapists deemed ‘not competent’ • Youth who did not receive any services (control group)

11. Family Functional Therapy Results: % New Felony Results calculated using multivariate models in order to control for potential differences between groups

12. Project Greenlight • Short-term prison-based reentry program in New York • CBT Skills Training • Employment Services • Housing Services • Drug Education and Awareness • Family Counseling • Practical Skills Training • Community-Based Networks • Familiarity With Parole • Individualized Release Plans

13. Project Greenlight Benefits • Participants received more service referrals • Participants reported more contacts with community services after release • Participants demonstrated significantly more familiarity with parole conditions • Participants were more positive about parole

14. But Did It Work?

15. What Went Wrong? • Violation of the risk principle • Ceased use of risk assessment instrument when staff deemed process too cumbersome • Violation of the need principle • All offenders received same services whether needed or not • Violation of the fidelity principle • Staff modified delivery of the CBT curriculum (shortened the duration, increased frequency, increased class size) • Differential staff competence • Certain case managers produced worse outcomes

16. UC Halfway House/CBCF Study in Ohio: A Look at Fidelity Statewide • Average Treatment Effect was 4% reduction in recidivism • Lowest was a 41% Increase in recidivism • Highest was a 43% reduction in recidivism • Programs that had acceptable termination rates, had been in operation for 3 years or more, had a cognitive behavioral program, targeted criminogenic needs, used role playing in almost every session, and varied treatment and length of supervision by risk had a 39% reduction in recidivism

17. What Do We Know About Fidelity? • Fidelity is related to successful outcomes (i.e., recidivism reductions). • Poor fidelity can lead to null effects or even iatrogenic effects. • Fidelity can be measured and monitored. • Fidelity cannot be assumed.

18. QA versus CQI CQI – What Is It? Infrastructure Peer Review Indicators Client Satisfaction Action Planning Process Evaluation Outcome Evaluation Benefits Monitoring Fidelity Through a CQI Process

19. QA – The Old Way • Retrospective review process • Emphasis on regulatory and contract compliance • Catching people being bad leads to hide and seek behavior

20. CQI – The New Way • CQI is a prospective process • Holds quality as a central priority within the organization • Focus on customer needs; relies on feedback from internal and external customers • Emphasizes systematic use of data • Not blame-seeking • Trust, respect, and communication • Move toward staff responsibility for quality , problem solving and ownership of services

21. Objectives of CQI • To facilitate the Agency’s mission • To ensure appropriateness of services • To improve efficiency of services/processes • To improve effectiveness of directing services to client needs • To foster a culture of learning • To ensure compliance with funding and regulatory standards

22. Building a CQI Process • Formal infrastructure • Core Elements • Documentation Review • Indicators • Process Versus Outcome • Performance Goals • Action Planning • Customer Satisfaction • Clients, Staff, Stakeholders • Program Evaluation

23. Process Evaluation Sample Projects

24. Process Evaluation • Are we serving our target population? • Are the services being delivered? • Did we implement the program as designed (tx fidelity)? • Are there areas that need improvement?

25. Example 1Review of LSI Scores • Reviewed all open cases at Facility A • Recorded LSI risk category, UC Risk category, and name of interviewer • 77.5% of cases reviewed did not have a match between staff rating and UC rating

26. LSI Scores Post-Training • First 2 weeks after training – 0 matches • 3-6 weeks after training – 46.2% matched • First 2 weeks after training – 50% were off by 2 risk categories • 3-6 weeks after the training – 0% were off by 2 risk categories

27. Individual LSI Reviews • Schedule of videotaped interviews • Submitted for review • Use of standardized audit sheet • Feedback loop for staff development • Aggregate results to inform training efforts

28. Sample LSI Audit Items • Explained purpose of interview • Adequate use of open-ended questions • Avoided double-barreled questions • Adequate use of follow-up questions • Overcame problems such as silence or excessive talking • Used interview guide • Scored correctly • Tx plan clearly relates to information captured in LSI

29. Example2CBIT Site Assessments • Cognitive Behavioral Implementation Team • Site visits for observation and rating • Standardized assessment process • Standardized reports back to sites • Combination of quantitative data and qualitative data

30. Example 3Focus Review at an Adolescent Residential Program • Examined changes in client characteristics over time • Examined successful completion over time • Identified factors predictive of AWOL’s, incidents, and completion • Examined use of role-plays in groups • Primary predictors of intermediate outcomes: • Overall Risk (education and peers specifically also important) • Criminal History • Treatment Dosage • Involvement in incidents

31. Example 4Focus Review at a Male Halfway House - Rural • Clients with any/more incidents were less likely to graduate successfully. • Clients who lost a job while in the program were significantly less likely to graduate successfully. • Clients with higher total LSI-R scores were also less likely to successfully complete the program. • Clients with higher total LSI-R scores were more likely to engage in program rule infractions. • Clients with higher intake HIT scores were more likely to engage in program rule infractions at TCC. Thus, individuals categorized as having stronger anti-social thinking patterns were more likely to be involved in incidents. • Age was also found to be predictive of rule infractions, with younger clients exhibiting greater likelihood of engaging in more incidents than older clients. • Employments status was found to be a significant predictor of program incidents. Specifically, clients who lost a job while in the program were more likely to violate program rules. • Dosage levels were found to be predictive of raw HIT score improvement.

32. Example 5Assessing Best Practices at 17 Sites • Use of ICCA Treatment Survey to establish baseline • Complete again based on best practice • Perform Gap Analysis • Action Plan • Reassess

33. Outcome Evaluation Sample Projects

34. Outcome Evaluation • Are our services effective? • Do clients benefit (change) from the services? • Intermediate outcomes • Reduction in risk • Reduction in antisocial values • Long-term outcomes • Recidivism • Sobriety

35. Example 1Off-Site Non-Emergency Medical Visits for 6 Months • Goals of the project: • Reduce the overall number of off-site visits for non-emergency medical care. • Reduce the number of staff-escorted visits (impacts dollars and coverage) • Reduce the amount of behavioral treatment missed • Increase the number of referrals for primary care upon discharge

36. Example 1 ContinuedOutcomes Data

37. ER VISIT COSTS $ 400 Average Visit $ 34 Average Staff Cost $$$ Hospital Pharmacy TOTAL $ 434 not including pharmacy savings TALBERT HOUSE $ 62.50 Average Visit $ 0 off-site staff cost Medication savings: samples, patient assistance programs TOTAL $62.50 average cost/visit Savings of $371.50/visit Sample Cost ComparisonER versus On-Site Care

38. Example 2Relationship Between Intermediate Outcomes and Recidivism • Female adolescent program’s intermediate outcome measures: • Antisocial attitudes • Self-esteem • Self-efficacy • Family functioning • Determine whether improvement on intermediate measures results in lower recidivism.

39. Example 2Relationship Between Intermediate Outcomes and Recidivism • Preliminary Results • Increased self-esteem = 71% • Increased self-efficacy = 61.3% • Reduced antisocial attitudes = 82.7% • All statistically significant • Statistically significant improvement in family functioning: • Cohesion • Conflict • Organization • Intellectual-Cultural Orientation • Moral-Religious Emphasis

40. Example 3Outcomes Pre/Post TFM ImplementationPassages Program for Girls

41. Example 4Outcome Evaluation of New Dosage Protocol • Practical application of the risk principle • Seeking to quantify how much dosage is required to reduce recidivism • Will compare clients discharged from the program pre-implementation to clients discharged from the program post-implementation.

42. Benefits of Program Evaluation • Proof of effective services • Maintain or secure funding • Improve staff morale and retention • Educate key stakeholders about services • Highlights opportunities for improvement • Data to inform quality improvement initiatives • Establish/enhance best practices • Monitor/ensure treatment fidelity

43. The Role of QA/QI in Community Corrections(based on UC Halfway House and CBCF study)

44. NPC Research on Drug Courts

45. Conclusions • Many programs are not implementing the principles of effective intervention with strong fidelity. • Result is an ongoing gap between science and practice. • This gap often results in null or even iatrogenic effects. • Correctional organizations have a responsibility to ensure effective services. • Monitoring fidelity is key to success. • Responsibility for EBP needs to be aligned at all levels – administration, management, line staff. • Need to focus on creating formal infrastructure to support and sustain evidence-based practices.