Separating Snake Oil from Penicillin: How to Determine Which Treatments Really Work

1. Separating Snake Oil from Penicillin: How to Determine Which Treatments Really Work Douglas W. Woods, Ph.D. University of Wisconsin-Milwaukee May 1, 2009

2. Snake Oil • a derogatory term for compounds offered as medicines which imply they are fake, fraudulent, or ineffective. • also applied to any product with exaggerated marketing but questionable or unverifiable quality.

3. Examples of Snake Oil

4. How is it Sold?

5. Why Did People Buy Snake Oil? • Very few physicians available to treat illnesses • Very few effective treatments were available for many illnesses • Claims and claimers were convincing • People were desperate for relief

6. Penicillin • Metaphorical term to describe treatments that are effective in treating a particular disorder/disease • Only applies to those treatments who have been subjected to the rigors of scientific inquiry

7. Examples of Penicillin

8. Why Do People Buy Penicillin? • It works • We know what we’re paying for • Because it’s researched, we can predict how likely it is to work for us • We can understand how safe it is

9. If Penicillin Exists, Why Is There Still a Market for Snake Oil? • For certain illnesses, there are very few knowledgeable treatment providers • For certain illnesses, there are very few effective treatments • Claims and claimers are convincing • People are desperate for relief • Sound familiar?

10. How Can We Tell the Difference? • Understand the Levels of Evidence • Snake Oils and Penicillins have various levels of evidence to support their claims • The higher the level of evidence, the more confident you can be in the claims made • Understand how penicillins are tested and why. • Read the literature • Scientific Journal articles are subject to review by independent scientists • Stuff on the web is not subject to independent review

11. Levels of Evidence • “Worked for me!” • Claims are based on theory, and theory alone • Claims are based on data, but data are of poor quality and/or from a few isolated cases • Claims are based on good data with a lot of cases, but are uncontrolled • Claims are based on results of a randomized controlled trial (RCT). • Claims are based on RCT, and treatment works for reasons developers think it works

12. “Works for Me” • Friends, family members, chatroom members, or providers tell you they have a treatment that works….but they don’t have data • Treatment may be effective, but we don’t really know • A number of possible explanations exist for why the effect happened • Claims about effectiveness of treatment are premature

13. Based on Theory…and Theory Alone • Claims are only as good as the theory they’re based on • In the absence of controlled studies, bad theory = bad claims • Even with bad theory, a treatment may still work, but then becomes ripe for abuse • Facilitated Communication

14. Based on Bad Data or a Few Cases • Claims are only as good as the data you collect • Types of Data • Subjective questions (How are you doing?) • Objective questions (MGH-HS) • Direct Observation (measuring bald spots) • Blinded evaluation • Problem with a Few Cases • These may be the only 2 cases in the world for whom the treatment had an effect

15. What is “controlled” When scientists try to rule out other explanations for an effect, they “control for” these explanations What else could explain why someone got better? Time Resolution of other issues Change in the environment/world events Having attention paid to you Placebo effect Etc. Open label trials These will often show up in medical journals Again, doesn’t mean the treatment is ineffective Good data, lots of cases, but uncontrolled

16. Open trial Measure Subjects Symptoms After Treatment Subjects Receive Treatment Measure Subjects Symptoms Before Treatment

17. Randomized Controlled Trial • RCTs attempt to control other explanations for effects • Used to compare 2 or more conditions • Randomized • Control groups • Wait-list • Placebo • Other established treatment • Double-blind • RCTs tell you if a treatment works, and if it is that treatment causing the change

18. RCT Person who did the assessment of progress does not know which treatment the person received Random Assignment ACT/HRT N=12Completed 10 sessions in 12 wks All do 2nd Assess N=25 1st Assess Wait-List N=13No Contact for 12 wks MGH-HSPulling CountsNIMH-CIRAAQPAI-AnxietyPAI-DepressionTEI-SF*Tx Compliance*Tx Utilization* *ACT/HRT only MGH-HSPulling CountsNIMH-CIRAAQPAI-AnxietyPAI-DepressionSCIDWASI

19. Treatment works, and it works for why we think it works • Penicillin works, and it works because it breaks down bacterial cell walls • CBT works for depression. • We used to think it worked because it changed your negative thoughts • Not sure that’s true anymore • If it works, and is proven by RCT, who cares? • Proprietary issue (think generic drug vs. brand name drug) • EMDR example…

20. What Do Snake Oil Salesmen Emphasize? • “Worked for me!” • Advertisers or promoters with testimonials and flashy websites in the absence of data • Claims based on theory and theory alone • Scientific or logical sounding • Resistance to challenging or testing the theory • Claims are based on data, but data are of poor quality and/or from a few isolated cases • Claims go beyond data • Resistance to complete testing of the treatment, or claims that testing the treatment isn’t possible

21. How to Find Penicillin • Claims are based on good data with a lot of cases, but are uncontrolled • Usually the starting point to figure out if something is really going to be penicillin • Not an end point • RCT results

22. Are Snake Oil Salesmen Charlatans? • People promote treatments for a variety of reasons • Profit • They really want to help people with a disorder • They were trained a certain way, and don’t know of other treatment options • The treatment may actually work but hasn’t been studied adequately enough to be considered penicillin

23. What about TTM Treatments • “Works for Me” & Theory Alone • Diet • “No Trichs Trigger” • Energy therapies • Acupuncture • Etc. • Poor data or few cases • Hypnosis • Awareness enhancement devices • Bupropion • Risperidone • Quentiapine • Inositol • Haloperidol • Paroxetine • Aversion (shock) therapy • Child-centered play therapy

24. What About TTM-Part 2 • Uncontrolled larger studies • Escitalopram • Topiramate • Olanzapine • Hydroxyzine • Venlaxafine • Citalopram • RCTs • Clomipramine* • Sertraline* • Habit reversal • Acceptance-enhanced Behavior Therapy • RCTs with support for mechanisms of change • Preliminary for Acceptance-enhanced Behavior Therapy

25. Questions?