Michael L. Dennis, Ph.D., Melissa Ives, M.S.W. Chestnut Health Systems, Normal, IL

1. A Phase 4 replication of MET/CBT5 in 36 sites to examine how findings vary by site, client characteristics, and implementation fidelity Michael L. Dennis, Ph.D., Melissa Ives, M.S.W. Chestnut Health Systems, Normal, IL Richard D. Lennox, Ph.D. Psychometric Technologies, Hillsborough, NC Randy Muck, M.Ed. Substance Abuse and Mental Health Services Administration (SAMHSA) Center for Substance Abuse Treatment (CSAT), Rockville, MD College of Problems on Drug Dependence (CPDD) & Society of Adolescent Substance Abuse Treatment Effectiveness (SASATE), Reno, NV. June 23, 2009

2. Background • In 1997 the third wave of cannabis use was the largest and youngest cohort to date, double the number of adolescents presenting to publicly funded treatment • There were no publicly available manual guided evidenced based practices targeting this population • The Cannabis Youth Treatment (CYT) experiments (n=600) were designed to manualize and field test five promising intervention for short term outpatient treatment of adolescent with cannabis (and other) substance use disorders • Adapted from earlier studies with adult alcohol and cannabis users, Motivational Enhancement Therapy/ Cognitive Behavior Therapy for 5 sessions (MET/CBT5) was the briefest, one of the least expensive, similar in clinical outcomes, and hence one of the most cost-effective approaches evaluated (Dennis et al 2004; French et al 2003).

3. Effective Adolescent Treatment (EAT) • From 2003 to 2008 SAMHSA’s Center for Substance Abuse Treatment (CSAT) conducted a phase IV (i.e., post randomization) replication of MET/CBT5 in 36 sites. • All sites received standardized training, quality assurance and monitoring on their implementation of MET/CBT5, as well as the collection of data with the Global Appraisal of Individual Needs (GAIN) to facilitate comparison with the original CYT study in terms of implementation and outcome. • The objectives of this presentation are to : • Demonstrate that EAT used MET/CBT5 with a more diverse population • Replicate the implementation and outcomes of MET/CBT5 • Identify participant characteristics moderators and intervention mediators that are associated with outcomes

4. Sample Selection • The Target Population Inclusion Criteria for including cases from the EAT data set were adolescents who: • Were assigned to MET/CBT in Outpatient and • Reported lifetime abuse or dependence symptoms and • Reported substance use in the last 90 days they were in the community and • Who were due for 6 month follow-up • Of 36 sites, 12 were dropped because they did not collect treatment received data at 3 months or because they less than 50% with BOTH 3 and 6 month interview • Of the remaining 3556 clients from 24 sites, • 2540 (71%) have outcome data at 6 months • 2540 (86%) have outcome data at 3 or 6 months • This group was compared using GAIN data to a cohort of 199 (98% of 202) adolescents from CYT that met the same criteria

5. Included EAT: 24 Sites Excluded EAT: 12 Sites EAT More Geographically Diverse WA NH ME VT MT ND OR MN MA NY WI ID SD MI WY RI CT PA IA NE NJ IL OH NV IN UT DE WV CO VA CA KS MD MO KY DC NC TN OK AZ AR NM SC GA AL MS CYT: 4 Sites TX LA FL AK HI

6. Demographics Race groups* EAT Clients were more likely to be female, non-white, and have a wider age range Age groups* *p<.01

7. Clinical Characteristics Primary Substance EAT Clients less likely to have cannabis as primary substance, similar in their comorbidity, and to have more justice system involvement. Comobidity Delinquency Level* *p<.01

8. EAT did as well or Better as CYT on Service Engagement *p<.01

9. Days of Treatment in the First 3 Months 84% 94% *p<.01

10. Little related to family services Content of CYT’s 5 Treatments Varied Low end for external or wrap around services Similar on direct services *p<.01

11. MET/CBT5 in CYT and EAT had a Similar Range of Content EAT received less external services *p<.01

12. Comparison of Outcomes • Evaluate the Increase in the Days of Abstinence from Intake to Last Observation. • For all time periods days abstinent are adjusted by subtracting any days in a controlled environment during the period (average is less than 5 days). • Change scores are calculated as last observation minus intake. • The large sample sizes involved make even trivial differences statistically significant. Thus this comparison focuses as much on clinical significance by using effect sizes.

13. Within and Between Group Effect Size Calculations • Effect sizes for within group change reported as: Cohen’s d = (MLast -MIntake )/ Std Dev.Intake where small=.2, moderate=.4 and large=.8 • Effect sizes for group (G) differences in change scores (Last-Intake) are reported relative to the grand mean (GM) as: Cohen’s f = (ABS(G(change) –GM(change)))/ Std Dev.(GM Change) where small=.1, moderate=.2 and large=.4

14. Change in Days Abstinent by Study (f=.02) EAT more severe Slopes are NOT significantly different

15. Effect Size by Site

16. Intake Predictors of Change in Days Abstinent Between Intake and Last Follow-up \a Other than alcohol or marijuana

17. Change in Days Abstinent by Days of Abstinence at Intake (f=.48)* * P<.0001

18. Change in Days Abstinent by Days in a Controlled Environment at Intake (f=. 21)* * P<.0001

19. Change in Days Abstinent by Count of Major Problems at Intake (f=.32)* * P<.0001

20. Change in Days Abstinent by Cocaine/Crack Problem Severity at Intake (f=.27)* * P<.0001

21. Change in Days Abstinent by Any Opioid Use in community at Intake (f=.16)* * P<.0001

22. Change in Days Abstinent by Other Drug Problem Severity at Intake (excluding alcohol/marijuana) (f=.10)* * P<.0001

23. Change in Days Abstinent by Age group at intake (f=.05)

24. Change in Days Abstinent by Gender (f=.04)* * P<.05

25. Other Client Characteristics that did NOT Predict the Amount of Change • Race • Single Parent • Metropolitan size • Primary drug • Days of use or problem group for alcohol, cannabis, amphetamine • Victimization • Psychopathology • Delinquency levels

26. Baseline + Mediators of Change in Days Abstinent Between Intake and Last Follow-up \a Predicted from intake only

27. Change in Days Abstinent by Days in a Controlled Environment Mons 1-3 (f=.22)* * P<.0001

28. Change in Days Abstinent by Costs within SAMHSA cost bands (f=.04) * P<.001

29. Change in Days Abstinent by Any Incarceration in Mons 1-3 (f=.17)* * P<.001

30. Change in Days Abstinent by Any Other SA Treatment (f=.04)* * P<.05

31. Change in Days Abstinent by Sites (f=.14)* 2 sites had less change than average(all low severity) 6 sites had more change than average * P<.0001

32. Other Moderators that did NOT Predict the Amount of Change • Initiation & Engagement in but p>.05 • Length of stay and continuing care • Treatment Received Scales (direct, family, wrap around) • SA Days of residential, IOP, OP, Medication, ER or Urine test/breathalyzer or summary index • MH Days of Inpatient, OP, Medication, ER or summary index • PH Days of Inpatient, OP, Medication, ER or summary index • Study or other sites • Months from intake to last follow-up

33. Limitations • Primarily relied on adolescent self report (plus some records on implementation). It would have been useful to have collateral or urine test results. • First cut only examined days of abstinence, it is likely that different variables impact other outcomes • Could have used other ways of adjusting for time in a controlled environment • All variation by site not explained yet. • May need to look at environment and peer risk to explain differences

34. Conclusions • EAT grantees were more diverse geographically, demographically and clinically • EAT grantees implementation was better than CYT in terms of engagement, similar in dosage, and only slightly less in content • Baseline severity was the primary factor explaining differences in the amount of change observed • Engagement, dosage and content were not the major mediator of change – environmental variables were • Further investigation is needed to understand why some sites did better than average even after controlling for above factors

35. Acknowledgements This presentation was supported by the Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for Substance Abuse Treatment (CSAT) under contracts 207-98-7047, 277-00-6500, 270-2003-00006, and 270-07-0191 using data provided by the following CSAT grantees: (CYT: TI-11320, TI-11317, TI-11321, TI-11323, TI-11324, EAT: TI-15413, TI-15415, TI-15438, TI-15446, TI-15447, TI-15458, TI-15461, TI-15466, TI-15469, TI-15478, TI-15479, TI-15481, TI-15483, TI-15489, TI-15514, TI-15524, TI-15545, TI-15562, TI-15577, TI-15586, TI-15672, TI-15677, TI-15682, TI-15686). Any opinions about these data are those of the authors and do not reflect official positions of the government or individual grantees. Thanks to Rod Funk, Mark Lipsey, Barth Riley, Michelle White and Ken Winters for their suggestions. Suggestions, comments, and questions can be sent to Dr. Michael Dennis, Chestnut Health Systems, 448 Wylie Dr., Normal, IL 61761, mdennis@chestnut.org .