Evolution of the Revolution: How Can Evidence-Based Practice Work in the Real World?

1. Evolution of the Revolution: How Can Evidence-Based Practice Work in the Real World? Bruce F. Chorpita Wing Institute Annual Summit on Evidence Based Special Education April 24, 2008

2. Acknowledgments

3. Acknowledgments

4. Acknowledgments

5. Acknowledgments

6. A Tale of Two Cultures The Story of John

7. This is John.

11. Evidence Based Practice:Not As Easy As It Looks

12. Diffusion of Innovation • Attributes of Innovations • Relative Advantage • Compatibility • Complexity • Observability Rate of Adoption • Communication Channels • Mass Media • Interpersonal • Change Agent Properties • Effort • Homophily Adapted from Rogers (1995)

13. Diffusion Curve Time 

14. Attributes of the Innovation

15. Fixed content Fixed intensity Fixed length Single target approach Replacement Empty cell problem Crowded cell problem Expiration problem Practitioner Concerns Aarons (2004); Addis & Krasnow (2000); Addis, Wade, & Hatgis (2004); Chorpita, Daleiden, & Weisz (2005); Kimhan & Chorpita (2006); Persons (1995)

16. Some EBS Effect Sizes(Chorpita et al. 2002) • CBT for anxiety: ES =1.05 • Exposure for anxiety: ES=2.02 • Modeling for anxiety: ES=0.55 • CBT with Parent/Child for anxiety: ES=1.68 • Behavior Therapy for ADHD: ES=1.24 • CBT for depression: ES=1.74 • IPT for depression: ES=1.51 • Relaxation for depression: ES=1.48 • Parent training for ODD: ES=0.89 • MST for delinquency: ES=0.50 Diffusion strategies:Relative Advantage

17. Comparison with Usual Care(Weisz et al. 1995) Diffusion strategies:Relative Advantage

18. Throw out the Bath Water:Keep the Baby • Move away from the idea that “Problem A gets Treatment B” • We decided to map the decisions made at the system level – not just the IF-THEN of treatment selection • Goal: To build an Evidence Based System

19. Where should we treat the client? Service Setting

20. How should we treat the client? Service Setting Therapeutic Practices

21. Are we providing quality service to the client? Service Setting Therapeutic Practices Treatment Integrity

22. Treatment Integrity Client Progress Is the client getting better? Service Setting Therapeutic Practices

23. Treatment Team Treatment Integrity Client Progress Who should treat the client? Service Setting Therapeutic Practices

24. Treatment Team Treatment Integrity Client Progress Supervision How should we manage the treatment? Service Setting Therapeutic Practices

25. Treatment Team Treatment Integrity Client Progress Supervision Treatment programs formalize these elements Service Setting Therapeutic Practices

26. Treatment Team Treatment Integrity Client Progress Supervision Selecting a program structures other decisions Service Setting Tx Program Selection Therapeutic Practices

27. Treatment Team Treatment Integrity Client Progress Supervision Selecting a program structures other decisions Multisystemic Therapy (MST) Example Service Setting Home & Community Based Tx Program Selection Therapeutic Practices MST Family Therapy, etc. 4 – 5 Members Team Supervisor Cross-Team Supervisor TAMS & SAMS Instrumental & Ultimate Outcomes

28. Treatment Team Treatment Integrity Client Progress Supervision Valid alternatives to deciding by program? Service Setting Tx Program Selection Therapeutic Practices

29. Treatment Team Treatment Integrity Client Progress Supervision How should we make decisions? Service Setting Tx Program Selection Therapeutic Practices

30. Treatment Team Treatment Integrity Client Progress Supervision How should we make decisions? General Services Research Service Setting Tx Program Selection Therapeutic Practices Evidence-Based Services Model

31. Treatment Team Treatment Integrity Client Progress Supervision How should we make decisions? Causal Mechanism Research Service Setting Tx Program Selection Therapeutic Practices Case-Specific Historical Information Individualized Case Conceptualization Model

32. Treatment Team Treatment Integrity Client Progress Supervision How should we make decisions? Service Setting Local Aggregate Evidence Tx Program Selection Therapeutic Practices Case-Specific Historical Information Practice-Based Evidence Model

33. Treatment Team Treatment Integrity Client Progress Supervision The Full System Model General Services Research Causal Mechanism Research Service Setting Local Aggregate Evidence Tx Program Selection Therapeutic Practices Case-Specific Historical Information

34. The Phases of Evidence • Data: Discretely identifiable units • Information: Data in a context that provides it meaning • Knowledge: Information helpful to decision-making • Wisdom: Knowing when to apply our knowledge c.f., Speigler, I. (2000). Knowledge management: A new idea or a recycled concept? Communications of the Association for Information Systems, 3, 1 – 23.

35. The Phases of Evidence: Example • Data: 70 • Information: 70º F • Knowledge: It is warm enough to wear shorts. • Wisdom: I am giving a professional talk today, so my knowledge that it is warm enough to wear shorts is irrelevant to my attire.

36. General Services Research:Turning Data into Knowledge I • Meta Analysis of Children’s Treatment Research • Over 350 studies, 40 years • Largest meta-analysis to date • Made dynamic for providers Diffusion strategies:Compatibility, Observability, Complexity

37. Treatment Team Treatment Integrity Client Progress Supervision How do we get this knowledge to the treatment team? General Services Research Causal Mechanism Research Service Setting Local Aggregate Evidence Tx Program Selection Therapeutic Practices Case-Specific Historical Information

41. Other Options • SAMHSA NREP • Published meta-analyses • Evidence based practice guidelines • Paper/web reports • Hawaii CAMHD Biennial Report • Hawaii’s “Blue Menu”

42. “Blue Menu” - Evidence-Based Child and Adolescent Psychosocial Interventions This tool has been developed to guide teams (inclusive of youth, family, educators and mental health practitioners) in developing appropriate plans using psychosocial interventions. Teams should use this information to prioritize promising options. For specific details about these interventions and their applications (e.g., age setting, gender) see the most recent Evidence Based Services Committee Biennial Report (http://www.hawaii.gov/health/mental-health/camhd/library/webs/ebs/ebs-index.html). <<NOTICE>> Note: Level 5 refers to treatments that were tested and found ineffective. Risk of harm is noted by the symbol (!!), which indicates that at least one study found negative effects on the main outcome measure. The risk of using such treatments should be weighed against potential benefits.

43. General Services Research:Turning Data into Knowledge II • Common Elements approach • Identified components of evidence based practices • Complements integrated program approach Diffusion strategies:Compatibility, Complexity

44. Is there a different level of analysis? Family Protocol Protocol Protocol

45. Is there a different level of analysis? Parent Training Incredible Years PCIT Defiant Children

46. Is there a different level of analysis? Family Protocol Protocol Protocol Practice Element Practice Element Practice Element Practice Element Practice Element Practice Element

47. Is there a different level of analysis? Parent Training Incredible Years PCIT Defiant Children Commands Attending Commands Time Out Time Out Rewards These are “practice elements.”

48. Example Attending

49. Anxiety

50. ADHD