Choosing Effective Indicators for Stimulant Treatment Trials: A Comprehensive Guide

1. Kathleen Carroll & Brian Kiluk Division of Substance Abuse Yale University School of Medicine Supported by NIDA Supplement to R01 DA15969 and P50 DA09241, U10 DA015831, R01 DA019078, & R01 DA 10679 Conundrums in Selecting outcome indicators for stimulant treatment trialsOR50 WAYS TO CALCULATE URINES

2. Why do we need a sound and valid indicator? • Facilitation of comparisons across projects, meta-analyses • Set and monitor performance standards • Benchmarking • Clearly convey magnitude of treatment effects to stakeholders • Facilitate comparisons across common standard • Lack of incentive to improve performance and outcome (retention not appropriate standard)

3. Overview • Desirable characteristics of indicators • Strengths and weaknesses of common approaches • Overview of our project

4. “Traditional” indicators of clinical significance almost always translate to complete abstinence • Return to normative levels • Reliable change indices • Return to healthy functioning? (e.g.,equivalent of ‘no heavy drinking days’ for stimulant users)

5. What are we looking for in an indicator? • Easy to calculate, interpret • Psychometrically sound, reliable, replicable • Low susceptibility to missing data • Verifiable (biologic indicator, other) • Independence from baseline measures • Sensitive to treatment effects • Low(er) cost • Predicts long-term cocaine outcomes • Related to indicators of good long term functioning • Acceptable to field • Easily interpreted by clinicians, policy makers, payors

6. What is ‘success’ in treating stimulant users? • Durable periods of abstinence • Employment, productivity • Lack of criminal activity • Reduced use of expensive, avoidable health care resources 11% at end of treatment, 21% at end of 1 year follow up

7. Why not complete abstinence? • Insensitive to change • Difficult standard for most individuals (14% of our sample of 434) • Chronically relapsing disorder • Change is dynamic • Starting and remaining abstinent may imply questionable need for treatment • Our data: Weak relationship with cocaine use and functioning outcomes at one year

9. Retention Pros • Easy to calculate • Available for all participants • Indicator of treatment acceptability • Indicator of differential attrition/data availability across conditions Cons • May be more meaningful in some contexts than others • Participants leave treatment for different reasons • Is retention with continued use meaningful? • Is compliance with ineffective treatment meaningful? • Not related to long-term outcome in our sample

10. Percent negative urines Pros • Widely used and accepted • Less susceptible to demand characteristics, misrepresentation • Quantifiable, ability to detect new episodes • Very accurate, if appropriate schedule of collection and minimal missing data • Timing is critical (overlap, missing data) Cons • Recent use only (3-5) • High cost for frequent or quantitative • Sensitive to missing data, esp. with differential attrition • Depends on assumptions (missing, denominator) • Stimulants or all drugs? • Can’t back-fill • Problems with assuming missing=positive*

11. Calculating percent urine samples Example: 1 negative urine, 2 sessions, then dropout of 12 week trial. Based on submitted: 100% Based on possible: 50% Based on expected/ 1x 8% Based on expected/ 3x 3% Percent cocaine positive 0%

12. Longest consecutive x-free urine specimens Pros • Strong evidence of meaningful abstinence • Less susceptible to demand characteristics, misrepresentation • Quantifiable, ability to detect new episodes • Very accurate, if appropriate schedule of collection and minimal missing data • Timing is critical (overlap, missing data) Cons • High cost for frequent or quantitative • Very sensitive to missing data, esp. with differential attrition • Depends on assumptions (missing, denominator) • Stimulants or all drugs? • Can’t back-fill

13. Percent days abstinent, self report Pros • Widely used • Potentially available for all participants and all days if TLFB used with high data completion; highly flexible • True intention to treat possible • Can be reliable if methods to enhance reliability used (at a cost) • Our discrepancy rate=8-12% Cons • With high/differential dropout, what’s the denominator? Days in treatment versus days expected? • Not easy to correct with urine data if discrepancies high

14. Maximum days of abstinence,overall or in final x weeks Pros • Linked linked to longer-term cocaine use • Potentially verifiable if urines collected at appropriate intervals • Provides ‘grace period’ • Easily dichotomized (eg 3 plus weeks) Cons • High complexity with missing data, especially dropouts • High complexity if discrepant urine data • Participants last 2 weeks or last 2 weeks of trial? • End of treatment or sometime within treatment?

15. Reduction in use: Frequency and or quantity Pros • Alternative to abstinence; more achievable target? • Highly compatible with random regression models • Sensitive to treatments that may take time for effects to emerge • Provides ‘grace period’ • Easily dichotomized Cons • Complexity obtaining accurate estimates of frequency/quantity of use prior to baseline • When is reduction measured (last weeks? Entire course? • Costs of repeated quantitative urines, sensitivity to missing data

16. Issues in defining ‘reduction’ • Patterns vary widely (binge versus low use) • Reliable estimation of quantity complex (illicit, no standard units, adulterants common, potency varies, hard to standardize ‘hits’ ‘joints’ ‘dime bags’) • Difficulty of estimating dollar value (commerce, shared use, sex for drugs)

17. Which indicator of treatment response? Loss of power with dichotomous, but also easily interpretable, calculable for all, relevant to clinical significance • Candidates • *Complete abstinence • *3 or more weeks of abstinence • *End of treatment abstinence • *Reduction of x percent • “Good functioning”

20. Good empirical question!

21. So far… • Carroll, K.M., Kiluk, B.D., et al. (2014). Towards empirical identification of a reliable and clinically meaningful indicator of treatment outcome for illicit drug use. Drug and Alcohol Dependence, 137, 3-19. • Kiluk, B.D., et al. (2014). What happens in treatment doesn’t stay in treatment: Cocaine abstinence during treatment is associated with fewer problems at follow-up. J Consulting and Clinical Psychology, 82:619-27.  • DeVito, E.E., et al. (2014). Gender differences in clinical outcomes for cocaine dependence: Randomized clinical trials of behavioral therapy and disulfiram. Drug and Alcohol Dependence, 145: 156-167. • Decker, S.E., et al. (2014). Assessment concordance and predictive validity of self-report and biological assay of cocaine use in treatment trials. The American Journal on the Addictions, 23, 466-74.  • Kiluk, B.D., et al. (in press). Prompted to treatment by the criminal justice system: Relationships with treatment retention and outcome among cocaine users.