Program will start 3:00PM Eastern 2:00PM Central 1:00PM Mountain Noon Pacific 11:00AM Alaska

1. Program will start 3:00PM Eastern 2:00PM Central 1:00PM Mountain Noon Pacific 11:00AM Alaska

2. Conference Planning Committee Jack Cummings Peg Dawson Susan Gorin Patti Harrison Ron Palomares Rick Short

3. Participant Evaluation www.indiana.edu/~futures/eval.html

4. RESEARCH TO PRACTICE: The Role of Evidence-Based Interventions in Practice Thomas Kratochwill University of Wisconsin-Madison

5. Purpose of Presentation • Provide an Overview of the Evidence-Based Intervention (EBI) Movement and Implications for School Psychology • Review Implications of the EBI Movement for Graduate Education Programs

6. Purpose (Cont.) • Review Implications of the EBI Movement for Practice • Implications for the Futures Conference

7. Resources • Chambless & Ollendick (2001). Empirically Supported Psychological interventions: Controversies and evidence. Annual Review of Psychology, 52, 685-716

8. Resources (Cont.) • Kratochwill & Callan-Stoiber (in press). Evidence-based interventions in school psychology: Conceptual foundations of the Procedural and Coding Manual… School Psychology Quarterly, 17 (4),

9. The Evidence-Based Intervention Movement in Perspective • Evidence-Based Medicine • The Scientist-Practitioner Model • The Managed Care Movement • Reform in Special Education • Professional Organizations

10. Professional Organizations • Committee on Science and Practice, Society for Clinical Psychology, Division 12 American Psychological Association (APA) • Committee for Empirically –Supported Practice, Division of Clinical Child Psychology Division 53, APA • Interdisciplinary Committee on Evidence-Based Youth Mental Health Care

11. Professional Organizations • Task Force on Evidence-Based Interventions in School Psychology (Sponsored by APA Division 16 and the Society of the Study of School Psychology and Endorsed by the National Association of School Psychologists)

12. Purpose of the Task Force • To identify prevention and intervention outcome studies relevant to the practice of school psychology • To code those studies according to Task Force criteria • To determine to what degree the interventions are evidence-based

13. Purpose (Cont.) • To offer the field of school psychology and related fields some guidelines for using and adopting effective programs • To provide a template for improving research in the field of school psychology and education

14. Task Force Domains of Interest • School-and community-based intervention programs for social and behavioral problems • Academic intervention programs • Family and parent intervention programs

15. Domains (Cont.) • School-wide and classroom-based programs • Comprehensive and coordinated school health care programs

16. Methodological Focus • Group Research • Single-Participant Research • Qualitative Research Procedures • Confirmatory Program Evaluation

17. The Review Process • Six components considered • Each component involves validity issues that need to be addressed during the review process

18. EXISTING RESEARCH REPORTS ARE CONSIDERED ALONG VALIDITY LINES • Statistical Conclusion Validity • Internal Validity • Construct Validity • External Validity

19. ORGANIZATION OF RESEARCH DOMAIN • The Problem Formulation Challenge • Validity Issues in Problem Formulation (e.g., operational specification and operational detail)

20. IDENTIFICATION OF RESEARCH • Threats of Validity (e.g., fugitive literature, file drawer problem, publication bias) • Resources needed to search

21. REVIEW OF STUDIES • Unrepresentative Studies • Methods of Gathering Studies • Retrieval Bias • Sample Representiveness

22. EVALUATION AND ANALYSIS OF RESEARCH SYNTHESIS • Narrative Reviews of Research (experimental and nonexperimental) • Quantitative Reviews of of Research (meta-analysis and generalized causal inference)

23. SUMMING UP: INTERPRETATION, PRESENTATION, AND DISSEMINATION • Omission of Details • Omission of Evidence on Moderators • Telling the Story to the Profession

24. RESEARCH TO PRACTICE: The Role of Practitioners in Evidence Based Interventions (EBIs) What are the Big Issues?

25. Research to Practice (cont.) • Issues surrounding the role of practitioners in the development and use of evidence-based interventions (EBIs). • Issues related to the research-practice gap.

26. Next Step Agendas • Review five assumptions that guide the integration of EBIs into practice. • Suggest potential solutions to problems surrounding the adoption of EBIs in practice.

27. Five Assumptions Guiding the Integration of EBIs into Practice Settings • The development and use of EBIs in practice needs to be a shared responsibility (agenda) among researchers and practitioners. • Implementation of EBIs in practice may require development and use of both practice guidelines and expert consensus guidelines.

28. (Assumptions cont.) • Offering professionals a menu of EBIs is insufficient to promote application in practice. Manuals and guidelines must be accompanied by additional strategies for effective integration of EBIs into practice.

29. Assumptions (Cont.) • To close the research-practice gap we must facilitate professional development for faculty and practitioners.

30. (Assumptions cont.) • To strengthen the connection between research and practice we must promote and use a scientist-practitioner model in graduate training and professional work, especially when evaluating intervention outcomes.

31. Shared Responsibility for EBI Development • Involvement of Practitioners on the Task Force • Participation in Practice Research Networks • Evaluation of EBIs in Practice With Case Studies and Ethnographic/Qualitative Methods

32. Development of Guidelines • Practice Guidelines • Expert Consensus Guidelines

33. AACAP Practice Parameters for the Treatment of Children and Adolescents with Conduct Disorders • Treat comorbid disorders (e.g., ADHD, specific developmental disabilities, intermittent explosive disorder, affective or bipolar disorder, anxiety disorder, and substance use disorder).

34. (cont.) • Family interventions included parent guidance, training, and family therapy. • Identify and work with parental strengths. • Train parents to establish consistent positive and negative consequences and well-defined expectations and rules. Work to eliminate harsh, excessively permissive, and inconsistent behavior management practices. • Arrange for treatment of parental psychopathology (i.e., substance abuse).

35. (cont.) • Individual and group psychotherapy with adolescent or child. Technique of intervention (supportive versus explorative; cognitive versus behavioral) depends on patient’s age, processing style, and ability to engage in treatment. Usually a combination of behavioral and explorative approaches is indicated, especially when there are internalizing and externalizing comorbidities.

36. (cont.) • Psychosocial skill-building training should supplement therapy. • Other psychosocial interventions should be considered as indicated. • Peer intervention to discourage deviant peer association and promote a socially appropriate peer network. • School intervention for appropriate placement, to promote an alliance between parents and school, and to promote prosocial peer group contact. Vocational training may be useful. • Juvenile justice system intervention, including court supervision and limit-setting, and special programs when available.

37. (cont.) • Social services referral, to help the family access benefits and service providers, (e.g., case managers). • Other community resources, such as Big Brothers and Big Sisters programs, Friends Outside, and Planned Parenthood, as indicated.

38. (cont.) • Out-of-home placement (crisis shelters, group homes, residential treatment) • Job and independent-living skills training (pp. 1335-1345). Source: From Steiner, H. (1997). Practice parameters for the assessment and treatment of children and adolescents with conduct disorder. American Academy of child and Adolescent Psychiatry, 36(10 Suppl), 1225-1395.

39. Guidelines Must Be Enhanced • Use Enhanced Guidelines that focus on context • Use Enhanced Guidelines that focus on theoretical mechanism for intervention • Use Enhanced Guidelines that focus on theoretical mechanisms for change (at the individual, classroom, and system levels)

40. Facilitate Professional Development • Focus on Graduate Programs and Practitioners in Schools • Adopt Competency-Based Models of Education • Increases Acceptability • Promotes Integrity of Interventions

41. Promote Evaluation of Intervention Outcomes • Consistent with the Scientist-Practitioner Model • Facilitate the Intervention Utility of Assessment

42. Specific Task Force Strategies and Tactics to Guide The Use of EBIs in Practice • Develop a Practice Research Network in School Psychology • Promote Effectiveness Studies • Develop and use Consumer Guidelines for practitioners when implementing interventions

43. (Strategies and Tactics cont.) • Use Enhanced Guidelines for Interventions • Dissemination and Education Efforts

44. Practice Research Network • Practitioners engaged in the evaluation of EBIs • Example: Hawaii Empirical Basis to Services Task Force [Clinical Psychology: Science and Practice, 9 (2) 2002]

45. Promoting Effectiveness Studies • Efficacy vs. Effectiveness • Effectiveness Studies Focus on Generalization of Interventions • Moving Beyond Effectiveness to “Real Life Study Design” (Hohmann & Shear, 2002)

46. Practitioner Partnerships • Practice Evaluation of EBIs • Establishing the Data Base for Practice

47. Example Dimensions to Consider in Use of Evidence-Based Intervention in School Psychology Practice Dimension Considered in Practice • Does your client appear similar to those described in the EBI? • Are you able to replicate the intervention based on the description provided in the manual and/or procedures? • Are the conditions of implementation of the EBI similar to those of your setting?

48. (cont.) • Are the measures you used to assess outcome identical to those used to establish the EBI? • Were all the measures recommended in the EBI used to evaluate the intervention? • Was ongoing evaluation (repeated assessment) of student progress conducted?

49. (cont.) • Have new outcome measures been added to the intervention evaluation? • Did you use an intervention manual or protocol that specified the intervention? • Can individual characteristics of students be identified that are related to intervention outcomes?

50. (cont.) • When group intervention data are reported, is the percentage of individuals showing the effect reported? • When individual data are reported, have the data been presented in graphic form? • Have the EBI positive effects reported in research been replicated with your student(s)?