Introduction

1. Effects of Integrated Treatment Fidelity on 1-Year Client Outcomes: A Comparison of Implementation Models Laurel Mangruma, Richard Spencea, & Molly Lopezb aUniversity of Texas at Austin, Addiction Research Institute, bTexas Department of State Health Services IDDT Implementation Models and Fidelity Scores Texas Integrated Treatment Integrity Scale Scores Integrated Treatment Adjunct Services Introduction Integrated Adjunct Treatment Services ( =3) ( =3) n n Staffing and Treatment Team Structure Caseload Size 4.5 4 The measurement of treatment fidelity is an important, yet often omitted component in examining the efficacy of various treatment models. Fidelity measures are useful in evaluation studies as a means of documenting appropriate implementation and adherence to the model under study, as well as confirming treatment differentiation in research designs that include control groups. Further, fidelity measures may also provide a method for determining critical ingredients in treatment models that are most strongly associated with client outcomes. Although research is limited, most studies examining the effects of treatment fidelity in the areas of supported employment and assertive community treatment have revealed a positive association between degree of fidelity and outcomes in clients with severe mental illness. Continued research is needed to determine the effects of departures from fidelity; as evidence-based practices are diffused in real-life clinical settings, treatment models are often adapted according to local resource constraints and other community and system characteristics. Measurement of the potential effects of these adaptations may inform technology transfer practices by identifying crucial aspects of treatment models associated with client outcomes that are less amenable to change. The current study examines the effect of integrated dual diagnosis treatment (IDDT) fidelity on 1-year outcomes of 256 clients with severe mental illness and substance use disorders. Treatment programs were participating in the Texas Dual Diagnosis Pilot Project, which was designed to develop integrated treatment programs through coalitions of state-funded substance abuse and mental health treatment providers. Each program was trained on the essential elements of the IDDT model and subsequently designed and implemented a treatment program unique to their local characteristics. DDX Provider Meeting 4 4.33 Team Approach 5 2.33 Attendance at Staffings 4.67 3.67 Continuity of Staffing 4.3 2 Staff Capacity 4.5 4 Intake Rate 5 5 Substance Abuse Specialist on Staff 5 3.67 Mental Health Specialist on Staff 5 5 Psychiatrist on Staff 2 3.33 Dual Diagnosis Training 3.33 2 Open Communication 4 3.33 Centralized Treatment 3.67 3 Total 4.21 3.51 * Treatment Program Structure and Services 2.33 4 Intensity of Service Frequency of Contact 1.33 1.5 Dual Disorders Groups 2.67 4.33 Adapted Substance Abuse Treatment 3 3.33 In-Vivo Services 2.33 1.67 Work with Support System 2.67 1.67 Assertive Outreach 1.33 3 Assertive Engagement Mechanisms 1 1.67 Housing Services 1 3 Employment Services 1.33 3 Tailored Intensity 2.67 2.33 Assertive Retention 2.67 1.67 Total 2.03 2.23 Integrated Treatment Documentation Admission Criteria 4 4 Integrated Treatment Plans 4.5 5 4 5 * Integrated Progress Notes Involvement in Discharge Planning 2.67 3.33 Stage of Treatment 3.33 4 Client Characteristics and 1-Year Outcomes Total 3.4 4.27 Overall Fidelity Score 3.21 3.34 * p < .05 Results and Conclusions Psychiatric Hospitalization Days During Past Year • Comparisons on the TITIS fidelity measure indicated a significant difference between the IT and AS models on the Staffing and Treatment Team Structure scale score but no differences were found on the Treatment Program Structure and Services, Integrated Treatment Documentation, or Overall Fidelity scores. Group contrasts on individual TITIS items revealed that the AS programs scored higher on one item measuring Integrated Progress Notes. Results of client outcome analyses revealed no differences between the two models in the number of psychiatric hospitalization days or arrests at 1-year post-baseline. • The two implementation models that emerged from the process evaluation reflected differences in service structure. One model displayed structural characteristics that had greater fidelity to the ideal implementation of IDDT, with the dual diagnosis team coordinating and delivering services. By contrast, the other model implemented a case manager structure with the dual diagnosis team as one of an array of adjunct service types. Few differences were found between these two models on TITIS fidelity scores other than treatment team structure, suggesting that although the method of service delivery may have differed, both model types implemented comparable levels of the essential elements of IDDT and achieved similar 1-year client outcomes. These findings indicate that, in this sample of IDDT programs, the departure from the ideal IDDT service delivery structure did not differentially affect client outcomes, suggesting that other critical elements of the treatment model may be a stronger factor in determining client outcomes than method of service delivery. Sample and Method Fidelity was assessed through two process evaluation site visits during the second year of the project, consisting of semi-structured interviews assessing the essential elements of IDDT. In addition, the Texas Integrated Treatment Integrity Scale (TITIS) was completed for each site using multiple data sources, including interview data, site self-report instruments, client chart review, and service records. The TITIS is based on the Dartmouth Assertive Community Treatment Scale (DACTS), adapted to capture service system features in Texas. Each TITIS item is scored on a 5-point scale, with higher scores indicating greater fidelity. The process evaluation site visit data identified two general modes of implementation across programs: the integrated treatment model (IT; n = 3) and the adjunct services model (AS; n = 3). Programs in the IT category were characterized by complete integration of substance abuse and mental health systems, with the dual diagnosis team coordinating and delivering services. In the AS model, a case manager coordinated a team of specialized service providers and the dual diagnosis team was one type of service in this array. TITIS ratings of the IT and AS models were compared using Wilcoxon rank sums tests to explore potential differences in fidelity scores. One-year outcome analyses were conducted contrasting clients who received IT (n = 120) versus AS (n = 136) on psychiatric hospitalization days and arrests. Baseline comparisons indicated that the IT group was older, had a higher proportion of schizophrenia-spectrum diagnoses, and had greater levels of psychiatric hospitalization in the year prior to admission; consequently, propensity scores were used to adjust for these initial group differences in outcome analyses. General linear modeling with repeated measures and Poisson distribution was conducted to compare psychiatric hospitalization and arrests during the 1-year periods pre- and post-baseline. Number of Arrests During Past Year Acknowledgements The authors acknowledge the Texas Department of State Health Services (TDSHS), Mental Health and Substance Abuse Services Division for their assistance in retrieving archival data for this study. The findings and conclusions of this presentation are the opinions of the authors and do not necessarily reflect the official position of TDSHS. Presented at the Addiction Health Services Research Conference, Little Rock, Arkansas, October 24, 2006