Bridging the Science / Practice Gap: The Promise and Perils of Evidence-Based Treatment

1. Bridging the Science / Practice Gap: The Promise and Perils of Evidence-Based Treatment WIPHL 3/08

2. Suppose you had a life-threatening illness and went to a physician who told you: “I really don’t pay much attention to medical research. I’ve been treating people like you for 30 years, and I know what works. Medical research isn’t all that relevant to my practice, and besides I’m too busy to be reading journals.”

3. In behavioral health care: • Practice has been guided by whatever approach the provider prefers • There has been no requirement to use science-based methods • Reimbursement has been linked to generic contexts of care like “group therapy,” “evaluation,” and “inpatient treatment” • What goes on behind closed doors has been left to professional judgment • However . . . .

4. Standards of care are changing • It is abundantly clear that not all “treatment works” • > 1000 clinical trials published in addiction • Cities, states, and other funding sources are increasingly demanding the use of EBTs • Closer integration of behavior health with healthcare will apply same standards

5. The writing is on the wall • Those who are not providing empirically supported treatment are going to have a harder time getting paid for their services • “Anything goes” is gone.

6. Six questions • Why use EBTs? • Which are EBTs and who decides? • Where can EBTs be used in services? • When should EBTs be used? • How do clinicians learn EBTs? • What are the potential pitfalls?

7. 1. Why should we use EBTs? • All treatments are not created equal • Treatment methods and therapists differ widely in efficacy • It’s going to be required; be an early adopter rather than playing catch-up • Closer integration with healthcare • We owe it to our clients

8. The Change Point mission is to provide the most effective substance abuse, mental health, and domestic violence intervention treatment utilizing culturally competent, evidence-based approaches designed to give the best possible outcomes to our clients.Change Point, Portland Oregon, 1998

9. CASAA shall operate only those treatments, services and programs for which there exists evidence of efficacy in the current scientific literature. Services with unproven efficacy will be designated as experimental procedures and offered only within the context of appropriately designed research to determine their efficacy.University of New Mexico Center on Alcoholism, Substance Abuse and Addictions Adopted 1994

10. 2. Which are EBTs and how is that decided? • Few clinicians have the time or expertise to analyze hundreds of clinical trials • Relies on reviews of the literature • Two refinements to reduce bias in reviews • Systematic reviews • Meta-analysis

11. What is admissible evidence? • 1. Strongest evidence: Randomized clinical trials • Well-designed randomized trials provide a persuasive, though imperfect, correction for human self-deception.

12. A hierarchy of evidence • Randomized clinical trials • Quasi-experimental designs that control for some sources of bias • Correlational studies with systematic observation • Case reports, professional opinion, and “best practice” consensus guidelines How much evidence is enough for an EBT? Consistency of evidence Cross-site replication

13. Agreement across ten reviews of substance abuse outcome studies Documentation: Miller, W. R., Zweben, J. & Johnson, W. R. (2005). Evidence-based treatment: Why, what, where, when and how? Journal of Substance Abuse Treatment, 29, 267-276.

14. 9 out of 10 reviews agree . . . • Cognitive-behavioral treatment • Community reinforcement approach • Motivational interviewing • Relapse prevention (cognitive-behavioral) • Social skill training

15. Less consensus on . . .

16. Methods shown in multiple trials to be ineffective • Educational lectures and films • Exploratory psychotherapies • Undifferentiated counseling • Confrontation • Mandated AA • Time in milieu (inpatient/residential)

17. Some Treatment Methods with No Controlled Studies • CENAPS Relapse Prevention (Gorski) • Rational Recovery • Reality Therapy (Glasser) • Solution-Focused Therapy • Spiritual Counseling • Transactional Analysis • Women for Sobriety

18. Commonly Practiced? Minnesota Model Confrontation Education Films General Counseling Group Therapy Mandated AA Milieu Therapy

19. The gap “could hardly be larger if one intentionally constructed treatment programs from those approaches with the least evidence of efficacy” Miller, Wilbourne & Hettema (2003) Handbook of Alcoholism Treatment Approaches: Effective Alternatives

20. Why the wide gap? • Providers are rarely trained in EBTs • “School” allegiance or eclecticism • Disease model linked to less use of EBT • Low accountability – anything goes • Reimbursement is not linked to EBT • Licensure is not linked to EBT • Training programs rarely teach EBT • No QA for delivering EBT

21. Why Isn’t Practice Self-Correcting? • Surgeons vs. Psychotherapists • Lack of reliable feedback • CPE has little effect on practice • Very low expectations in behavioral health • Staff turnover; devaluing of addiction professionals

22. When Should EBTs Be Used? • Whenever possible • With more than 1000 controlled clinical trials in the literature for alcohol, tobacco and illicit drug use, it is no longer defensible to say that there is too little research from which to draw conclusions.

23. When would one not use an EBT? • When there is no EBT available • Solvent abuse • Specific comorbidity • Draw on EBTs for similar problems • For example, EBTs for the separate concomitant problems

24. “Not my population” ? • Insufficient research available for efficacy of treatment methods with specific populations • What then? • Draw on EBTs from other populations • Remove specific barriers • Evaluate in your own population

25. Where EBT can be provided? • Individual provider level • Program level • Program policy to provide EBT • Necessary system changes to support the use of EBT • Treatment system level • Beware unfunded mandates • System support for retraining • Complexity of quality assurance

26. How do clinicians learn EBTs? • In order to provide EBTs, clinicians must develop proficiency in them • Often EBTs were not learned during initial training

27. How Substance Abuse Clinicians Learn New Treatment Methods Heidi Erickson, UNM Master’s Thesis, 1999 Informal Methods (Colleagues+Experience+Reading) = 62%

28. What does it take to learn EBTs? • Limitations of CPE workshops (let alone treatment manuals) • Feedback and coaching • Need for taping and supervision • Hire staff who are competent in EBTs • Start NOW teaching EBTs to the next generation of providers!

29. Pitfalls with EBTs • Efficacy versus effectiveness • Efficacy varies across sites and providers • Without QA monitoring, EBT policy simply requires saying that you deliver EBTs • Clinician self-reported proficiency can be unrelated to actual proficiency • Program directors may be clueless about what actually happens behind closed doors

30. Problems with lists of EBTs • Arbitrary criteria (e.g. APA Division 12) • Need for continual updating • Limitations of available research • Ossification • Inhibition of innovation • What about unevaluated methods? • Effective until proven otherwise?

31. Evidence-Based Relationships • Consistent evidence that substance abuse treatment providers differ significantly in effectiveness • Often the largest predictor of clients’ outcome is the counselor to whom they were assigned • Accurate empathy, as defined by Carl Rogers, is a particularly strong predictor

32. Take-Home Messages

33. It makes a difference what we do • It makes a difference how we do it (and who does it) • We already know how to do better than we do • Changing to EBTs is difficult; requiring it even moreso • EBTs are learnable • The real beneficiaries are our clients