Ron Claus, PhD Missouri Institute of Mental Health

1. Moving Toward Integrated services Using the Dual Disorder Capability in Addiction Treatment (DDCAT) Index Ron Claus, PhDMissouri Institute of Mental Health Criminal Justice ACJT/ORP Grantee MeetingSeptember 29, 2010

2. Overview • Why measure program capacity for co-occurring disorder treatment? • Development of the Dual Diagnosis in Addiction Treatment Disorder (DDCAT) Index • Measure and method • Reliability and validity • Using the DDCAT with substance abuse programs • DDCAT research • What’s next?

3. Why focus on co-occurring disorders? • Substance use disorders are common in people with mental health disorders. • Mental health disorders are common in people with substance use disorders. • Co-occurring disorders lead to worse outcomes and higher costs than single disorders. • Evidence-based models exist and can be implemented. • Providers and consumers want a better system and services. • Few people (<10%) get the treatment they need.

5. Addiction treatment provider prevalence estimates by quadrant Substance Use Severity Psychiatric Problem Severity McGovern et al., 2006

6. Why do we need to measure co-occurring capability? • Generic terms such as “integrated” or “enhanced” are “feel good” rhetoric but lack specificity. • Systems and providers seek guidance, objective criteria, and benchmarks for providing the best possible services. • Consumers should be informed about the range of services, to express preferences, and make educated treatment decisions. • Change efforts can be focused and outcomes of these initiatives can be evaluated. (Mueser et al, 2003; SAMHSA, 2005; Watkins et al, 2005)

7. Development of the DDCAT

8. DDCAT Index Development • Practical program-level policy, practice, and workforce benchmarks: Based on scientific literature and expert consensus • Observational methodology: Interviews, document review, ethnographic observation (vs. self-report) • Iterative process of measure refinement through field testing and psychometric analyses • Materials: Index, Manual, Toolkit, and Excel workbook for scoring and graphic profiles

9. Is there a conceptual model that could guide research and practice for addiction treatment? • The American Society of Addiction Medicine (ASAM) Patient Placement Criteria Second Edition Revised (PPC-2R) outlined the framework for a model • The ASAM-PPC-2R is designed for addiction treatment services • The ASAM-PPC-2R patient placement criteria have been widely adopted in public and private community addiction treatment

10. ASAM Taxonomy of Dual Diagnosis Services • Addiction Only Services (AOS) • Dual Diagnosis Capable (DDC) • Dual Diagnosis Enhanced (DDE)

11. Addiction Only Services (AOS) Programs that either by choice or for lack of resources cannot accommodate clients who have psychiatric illnesses that require ongoing treatment, however stable the illness and however well-functioning the client

12. Dual Diagnosis Capable (DDC) Programs that have a primary focus on the treatment of substance-related disorders, but are also capable of treating clients who have relatively stable diagnostic or sub-diagnostic co-occurring mental health problems related to an emotional, behavioral, or cognitive disorder.

13. Dual Diagnosis Enhanced (DDE) Programs that are designed to treat clients who have more unstable or disabling co-occurring mental disorders in addition to substance-related disorders.

14. DDCAT Index Development Can a staff survey be used to determine dual diagnosis capability? (McGovern et al., 2007) • Directors, Clinical Supervisors & Clinicians (n=453) rated their own programs using brief definitions based on the ASAM taxonomy (AOS, DDC, DDE) • Only modest agreement among staff within programs (47.3%) • A relatively rapid and economical approach that allows initial data or program screening • Bias and error

15. The need for a more objective assessment of dual diagnosis capability • ASAM offers a road map, but no operational definitions for categories or services • Fidelity as adherence to an evidence-based practice or model, fidelity scales as objective ratings of adherence • Developing an evaluation measure using fidelity scale methods • Set of for quality benchmarks for COD treatment • Based on previous literature and expert consensus

16. DDCAT Index Development • Feedback on items from experts in dual diagnosis treatment and research, state agency administrators, addiction treatment and fidelity measurement experts • Initial field tests of a 60-item scale • Refined scoring and reduced to 35 items (Version 2.4) based on psychometric studies; McGovern, Matzkin, & Giard (2007) • DDCAT Collaborative (2007), further refined anchors and revised manuals (Version 3.2)

17. Dual Diagnosis Capability in Mental Health Treatment (DDCMHT) Index • Ability to compare capability across primary mental health and addiction treatment programs • DDCAT items revised into mental health terms, further refined to fit mental health settings (Gotham et al.) • Established reliability and validity • IDDT Fidelity Scale Total Score & DDCMHT Total Score, r = .70 (22 mental health providers in MO; Gotham et al., 2004) • Cronbach’s alpha for DDCMHT dimensions (.53-.85, 67 providers in 6 states, Gotham, Brown, Comaty, McGovern, Claus (2009))

18. Measure and Methods

19. DDCAT Dimensions

20. DDCAT Index and Item Ratings

21. The Dual Diagnosis Capability in Addiction Treatment Index • Half-day to one day site visit • Primary sources of information: • Staff interviews (director, supervisors, clinicians) • Client interviews (group of 2–4) • Chart review (current and discharged) • Ethnographic observation • Document review • Direct observation • Reviews by pairs of interviewers, with independent rating and consensus on final item score • Draft report and conference call

22. I - Program Structure • Primary treatment focus as stated in the mission statement. • Organizational certification and licensure. • Coordination and collaboration with mental health services. • Ability to merge funding streams to provide COD services.

23. II - Program Milieu • Expectation and welcome of clients with COD. • Display and distribution of substance abuse and mental health related literature and patient educational materials.

24. III - Clinical Process: Assessment • Routine screening methods for mental health symptoms. • Routine assessment methods for clients who screen positive for mental health symptoms. • Frequency and documentation of mental health and substance use diagnoses. • Documentation of mental health and substance use history in the medical record. • Capability to provide services to clients based on acuity of mental health symptoms. • Capability to provide services to clients based on the severity and persistence of disability. • Initial assessment of readiness for change with COD clients.

25. IV - Clinical Process: Treatment • Documentation of mental health and substance related disorders in treatment plans for clients with COD. • Ongoing capability to assess and monitor mental health and substance related disorders, separately and interactively. • Procedures for mental health related emergencies and crisis management. • Ongoing assessment of readiness for change with COD clients. • Policies and procedures for medication evaluation, management, monitoring, and compliance. • Specialized interventions with mental health content. • Client education about mental health disorder and treatment, and how it interacts with substance use disorder and treatment. • Education and support to family members of clients with COD. • Specialized interventions to facilitate use of COD self-help groups. • Peer recovery support for clients with COD.

26. V - Continuity of Care • COD addressed in discharge planning process. • Capacity to maintain treatment continuity for mental health and substance related disorders. • Focus on ongoing recovery issues for mental health and substance related disorders. • Documented facilitation of COD self-help groups. • Documentation of sufficient supply and compliance plan for medication.

27. VI - Staffing • Access to services from a psychiatrist or other prescribing professional. • On site staff with mental health licensure. • Access to supervision or consultation for clients with COD disorders. • Supervision, case management, or utilization review procedures that emphasize and support COD treatment. • Peer/Alumni supports available to clients with COD.

28. VII - Training • Basic training in prevalence, signs and symptoms, screening and assessment of mental health disorders. • Staff members are cross-trained in mental health and substance use disorders.

29. DDCAT Profile: Practical Guidance for Providers DDE DDC AOS

30. DDCAT Reliability & Validity

31. Reliable, Valid, Practical? • Single state studies show high DDCAT reliability and inter-rater reliability • Median alpha .81 (range .72-.93) • Kappa (14 programs in MO) = .76 • Sensitivity to change (MO, CT), p <.05 (9-12 MO) Gotham et al, 2004; Claus et al, 2006; McGovern et al, 2006, 2007)

32. Reliable, Valid, Practical? • Relationship with psychiatric severity levels • Increasing access for persons with COD from AOS to DDC to DDE level programs (p<.001) • DDCAT reliability and feasibility (Brown & Comaty, 2007) • 38 substance abuse and mental health agencies in LA rated by evaluation and external raters • High inter-rater reliability can be achieved, even with new raters Gotham et al, 2004; Claus et al, 2006; McGovern et al, 2006, 2007)

33. 163 programs in 9 states (2004-2008, DDCAT Collaborative) X = 2.72 SD = 0.64 Range = 1.43 to 4.67 Criterion : AOS 80.4% DDC 19.0% DDE 0.6%

34. Rasch Modeling Results Mean measure = -0.22 Item reliability = .96 Program reliability = .93 Separation, Gp = 3.66 -> 5 groups of programs can reliably be discriminated

35. Rasch Modeling Results • Principal components analysis supports a one-factor model • Observed pattern and expected item difficulty are fairly consistent, supporting validity: • “Easy” III-E, Program acceptance and MH acuity III-F, Program acceptance and MH severity • “Hard” IV-D, Stage-wise treatment IV-H, Family education and support • A developmental perspective

36. Using the DDCAT with Substance Abuse Programs

37. DDCAT PROFILE CASE STUDY: UNDERACHIEVING PROGRAM

38. DDCAT PROFILE CASE STUDY: OVERACHIEVING PROGRAM

39. DDCAT Profiles:3 Programs within a single agency DDE DDC AOS

40. DDCAT Change over time: A Women & Children’s Residential Program DDE DDC AOS

41. CLINICAL AND PROGRAMMATIC STAGES OF CHANGE: PARALLEL PROCESSES

42. Developing a program implementation or change plan • Identify the DDCAT dimension (Goal) • Identify the DDCAT items (Objectives) • Develop an Intervention • Identify the responsible parties • Decide upon a Target Date • Identify Measureable Outcomes

43. DDCAT Profile: An Outpatient Program in Baton Rouge DDE DDC AOS

44. Draft Implementation Plan for the Baton Rouge Program

45. DDCAT Research

46. 27 Substance Abuse & Mental Health grantees DDE DDC AOS/ MHOS Missouri Foundation for Health Co-Occurring Disorders initiative, 2007-10

47. Change in Dual Diagnosis Capability Missouri Foundation for Health Co-Occurring Disorders initiative, 2007-10

48. Change in Dual Diagnosis Capability Missouri Foundation for Health Co-Occurring Disorders initiative, 2007-10

49. DDCAT Collaborative: Current Activities • Learning collaborative among 13+ states: Data sharing, quality improvement exchange, and research studies • Combined DDCAT database (larger program n) • Baseline and follow-up DDCAT assessments: Qualitative & quantitative analyses of effective change strategies • Learning Community and Workgroups • Measurement, Outcomes, Products

50. DDCAT PROGRAM CATEGORIES:VARIABILITY ACROSS NINE STATES