Los Angeles County Evaluation System

1. Los Angeles County Evaluation System Accomplishments and Next Steps toward Outcomes and Performance Measurement of Treatment Services in LA County 2008

2. Why was LACES Implemented? • LA County Commissions on Alcoholism and Narcotics and Dangerous Drugs recommended to ADPA that a system for measuring the effectiveness of LA County AOD Treatment services should be developed. • This sentiment was reinforced by the LA County Board of Supervisors and LACES was funded and initiated. • SAMHSA instituted the NOMS • California instituted CalOMS • Data required to support SACPA and NORA • LA Board of Supervisors Resolution to require health services in LA County to move provider contracts to a performance based platform.

3. Overview of LA County Evaluation System: LACES • LACES is an eight year collaborative partnership between ADPA, contracted providers, UCLA/ISAP and other county and state representatives. • The purpose of LACES is to design, implement, and conduct an evaluation of the adult alcohol and other drug treatment/recovery system in Los Angeles County.

4. Goals of LACES • Gather information on the alcohol and other drug treatment system • Obtain additional insight into the clients who receive treatment at ADPA contracted sites. • Develop a system-wide data collection system to evaluate the outcomes for individuals in treatment in AOD LA County and to assess performance of LA AOD treatment service agencies. • Use data to influence the future development of the AOD system.

5. Accomplishments of LACES To date LACES has: • Laid the foundation and developed many of the fundamentals of the ongoing evaluation system. • Designed and implemented evidence-based, standardized evaluation procedures. • Collects client data at admission and discharge. • Demonstrates treatment benefits to individuals who receive treatment. • Reported findings in LACES Status Report.

6. LACES Accomplishments, cont’d. • Led the efforts to review and improve existing treatment forms and instrument. • Revisions were required to bring LA County compliant with the federal mandates (NOMS/PPG). • Revisions adopted by California (CalOMS). • Conducted extensive training of treatment/recovery staff. • Provided instruction to staff to improve validity and reliability of LACPRS information. • Instructed over 1500 staff in various trainings.

7. Additional Accomplishments • Designed and implemented LACES reports. • LACES Clinical Narrative Report. • Encourages and builds off of accurate completion of admission information by treatment/recovery staff. • A summary of clinical related information. • LACES Discharge Outcome Report • Admission & discharge information on a single individual. • Useful report treatment outcomes to others. • LACES Site Report.

8. What is a Site Report? • Brief – Six pages • Provides demographic, client outcome, and program performance measures. • Information is gathered from the admission & discharge forms. • Includes site specific information as well as information from other sites of the same type. • Reports provided to all programs on an annual basis and to those programs with high admissions every quarter.

9. What is the Purpose of the Site Report? • To provide information concerning how participants are benefiting from treatment. • To provide information concerning areas of program functioning and short-term (admission to discharge) treatment outcome. • To identify provider performance strengths and areas needing improvement.

10. Program Functioning & Treatment Outcomes Areas included in the Site Reports include: • Program Performance • Number of admissions and discharges. • Percent of cases with complete discharge information collected. • Participants who completed treatment. • Treatment Outcomes • Retention in treatment (average days). • Changes in alcohol and other drug use. • Changes in other areas including health problems, employment, and social support.

11. Performance Measurement and Outcomes Monitoring in the US A Two Day Meeting Sacramento, California November 15, 16 2007

12. Objective 1: Major Activities • Develop a set of performance and outcome measures using CalOMS data to improve accountability and treatment quality and meet national standards. • Review and synthesize literature on performance measurement and outcomes monitoring. • Bottom line: Develop some models to define and measure program performance.

13. What do we want to accomplish? • Some specific issues for discussion • Which CalOMS data elements, alone and in combination, provide meaningful performance and outcomes measures? • Are there data elements that need to be added to CalOMS data set to optimally measure performance and outcomes? • Is it possible to operationally define “treatment success” and how can this issue be communicated to the public and policy makers?

14. What do we want to accomplish? • Determine how to adequately measure access to care? • Determine how feedback from clients can be used to improve program performance and treatment outcomes (e.g., satisfaction surveys)? • What is the role of “Evidence-based Practices” in improving performance and outcomes? • How are these defined? How is their implementation measured?

15. Objective 2: Activities • A 2-day meeting in Spring of 2008 with experts and stakeholders to understand promising performance improvement strategies. • Provide trainings for county administrators and treatment providers on how to use CalOMS data for performance and outcome improvement. • Develop CalOMS updates to facilitate communication between State and counties.

16. Summit: Using Performance and Outcomes Measures to Improve Treatment Los Angeles Sheraton March 20, 21, 2008

17. LACES Next Steps • Provide input to shape the way performance is measured in LA County • Refine data collection procedures • Advise on technical assistance support activities needed • Provide input into models for improving program performance with data.

18. The End