Training School Psychologists in Evidence-Based Practices for Tertiary Students with Emotional Disturbance | University

1. Evidence Based Practice University of Utah Training School Psychologists to be Experts in Evidence Based Practices for Tertiary Students with Serious Emotional Disturbance/Behavior Disorders Emily Haygeman 11.15.2009 US Office of Education 84.325K H325K080308

2. Evidence Based Practice in Psychology • The Second Special Issue on Evidence-Based Psychosocial Treatments for Children and Adolescents: A 10-Year Update • Wendy K. Silverman and Stephen P. Hinshaw

3. Background • 1998 First special issue focused on “empirically supported treatments.” • Primary task: to examine the evidence for the efficacy of psychosocial interventions to be used with children and adolescents • 5 articles reviewed the evidence for interventions for : Autism, Depression, Phobic and Anxiety Disorders, Disruptive Behavior Disorders (ODD and CD), and ADHD • The issue became widely used and cited, suggesting a growing interest in interventions that are supported by research

4. Current Considerations • Urgent issues remain in the field • Especially relevant to this article is the gap between research and practice • This means the gap between what we know works and what clinicians actually do in practice • One goal of the current article is to narrow this gap

5. The Second Issue: Functions • This issue has the following functions: • To contribute to scholarship in the area of clinical child and adolescent treatment • To educate • It should be “descriptive and evaluative rather than prescriptive.” • In short, to “present a scholarly treatise of the current status of the psychosocial treatment research literature with respect to empirical evidence for efficacy.”

6. Functions cont’d • Should inform the audience (clinicians, parents, teachers) of • 1. The treatments that have been reviewed and scrutinized, and • 2. Which treatments have been found to possess a “greater level of evidence relative to other treatments.” • All articles were peer reviewed, and additionally reviewed by an anonymous evaluator, changes were made when necessary and then submitted for reevaluation

7. Limits • The treatment studies reviewed varied greatly by domain of coverage (for example, a SSD would be appropriate for an autism or ADHD case but would not make sense to use with anxiety or disruptive disorders which have a larger base rate) • Contributors were given room to decide individually which interventions to evaluate in their respective areas (this can be both positive and negative) • Evidence was contextualized in one area • Can contributors be “unbiased” in the evidence they present? • Conclusions confined to what is currently being most used and researched in the field (ie cognitive-behavioral therapy)

8. Reader Considerations • Always critically evaluate, even when something is labeled “evidence-based” • This Issue can serve as a starting point, but always include other criteria (such as APA Presidential Task Force on EBP) when making a final decision re: treatment methods

9. “Evidence based” vs “Empirically supported” or “Empirically validated” • “Evidence based” is now preferred over “empirically supported” (1998) and “empirically validated” (1995) • “Evidence” is a word that is easily understood by nonpsychologists (health care administrators, policymakers, teachers, parents), and therefore can reach and benefit a wider range of people involved in the treatment of children and adolescents

10. Specific Disorders Reviewed in the Second Issue • Autism • Depression • Phobic and Anxiety Disorders • Disruptive Disorders (ODD and CD) • ADHD • Also includes: OCD (now recognized as distinct from other anxiety disorders), eating disorders and substance abuse disorders, and trauma and related reactions and symptoms • Also an article on the efficacy of psychosocial treatments specifically with minority youth

11. Contributors Included: • Narrative evaluative summaries (sample size, age ethnicity, sex, recruitment procedures, inclusion/exclusion criteria, outcome measures, therapist characteristics and specific treatment procedures) • Efficacy vs effectiveness • Generalizability • Mediators and Moderators

12. Efficacy vs Effectiveness • Efficacy: • Whether a treatment has been found to work by means of experimental procedures such as random assignment, control groups, and manualized protocols • Effectiveness: • Whether a treatment has been found to provide benefit in settings/conditions in which treatment will actually be administered (mental health, private practice, etc.)

13. Mediation and Moderation • Mediation: How therapeutic change is produced • Moderation: For whom or under what conditions this change occurs • An understanding of what needs to be changed and for whom allows the clinician to be able to adapt to variations • By studying the mechanisms of treatment we can improve clinical practice and patient care

14. Criteria for Classifying Evidence-Based Psychosocial Treatments • Criteria 1: Well-Established Treatments • 1.1 Must be at least 2 good group-design experiments, conducted in at least 2 independent research settings and by independent investigatory teams, demonstrating efficacy by showing the treatment to be: • Statistically significantly superior to pill or psychological placebo or to another treatment OR • Equivalent (or not significantly different) to an already established treatment in experiments with statistical power being sufficient to detect moderate differences

15. Criteria cont’d • AND • 1.2 Treatment manuals or logical equivalent were used for treatment • 1.3 conducted with a population, treated for specific problems, for which inclusion criteria have been delineated in a reliable, valid manner • 1.4 reliable and valid outcome assessment measures, at minimum tapping the problems targeted for change were used, and • 1.5 appropriate data analyses

16. Criteria cont’d: Criteria 2 • Criteria 2: Probably Efficacious Treatments • 2.1 There must be at least two good experiments showing the treatment is superior (statistically significantly so) to a wait-list control group OR • 2.2 One or more good experiments meeting the Well-Established Treatment Criteria with the one exception of having been conducted in at least two independent research settings and by independent teams

17. Criteria cont’d: Criteria 3 and 4 • Criteria 3: Possibly Efficacious Treatments • At least one “good” study showing the treatment to be efficacious in the absence of conflicting evidence • Criteria 4: Experimental Treatments • Treatment not yet tested in trials meeting task force criteria for methodology

18. Strength of Treatment Effects • Authors could combine the effect sizes across studies using meta-analyses or could calculate the effect size for each study individually • Effect size is a uniform metric • However, such metrics have a limited qualitative value

19. Six Types of Treatment Studies • Type 1: most rigorous, involve randomized, prospective clinical trial methodology; have comparison groups with random assignment, blind assessments, clear presentation of inclusion and exclusion criteria, state-of-the-art diagnostic methods, adequate sample size and clearly described statistical methods. • Type 2: Clinical trials testing an intervention, but one aspect of Type 1 is missing, or there is a clear but “not necessarily fatal” flaw (such as no follow-up) • Type 3: Methodologically limited. Open trials aimed at obtaining pilot data. Subject to bias but do indicate whether the study would be worth pursuing more in depth • Type 4: Reviews with secondary data analyses (such as meta-analyses) • Type 5: Review without secondary data analyses • Type 6: Case studies, essays, and opinion papers

20. These Treatment Studies… • The majority were Type 1, 2, and 3 • Type 6 lacks rigor and Type 4 and 5 were excluded because the contributors themselves were writing those types of reports

21. Concluding Comments • Much progress has been made in this area in the past decade • Considerable work remains • Hopefully this issue will begin to bridge the gap between research and practice • These articles call for more research that is more sophisticated, rigorous and statistically powerful