Research-Practice Gap in Era of Evidence-Based Mental Health

1. An International Project on the Effectiveness of Psychotherapy and Psychotherapy Training (IPEPPT): Research Framework and Protocols Robert ElliottUniversity of Toledo

2. Research-Practice Gap in Era of Evidence-Based Mental Health • Numerous contemporary attempts to link research & practice in psychotherapy • Top-down solutions: • Empirically-supported treatments • Evidence-Based Practice • Based on: • Randomized Clinical Trials research model • Therapist-as-research-consumer model • Results have been mixed

3. Research-Practice Integration as a Two-way, Dialectic Process • Success is more likely if we add a more integrative, bottom-up strategy • Building on Mental Health Services/ Therapy Effectiveness paradigm • Existing RCT research makes space for grass-roots-based research in real world practice and training settings • =Practice-based Evidence

4. Example: Practitioner Research Networks (PRNs) • USA: Pennsylvania (Ragusea, Borkovec, Castonguay) • UK: National Health Service CORE research team (Barkham, Evans et al.) • Latest trend: Practice-based research in training clinics and centers (e.g., Castonguay et al.)

5. Practice-Based Therapy Research in Training Sites • Training site research movement: USA, Europe • Research on psychotherapy process/outcome is essential for understanding and improving psychotherapy practice in all orientations • Being able to use and carry out research is an important aspect of therapist competence • Best way to learn therapy research methods: • Do research during basic therapy training • Primary professional socialization process • Create habits that carry over into later practice

6. Principles for Practice-based Research • Make research relevant to actual practice of therapy • Use methods that support therapy rather than interfere with it • Actively and continuously involve therapists in selection of research questions and methods • Include inexpensive and easy-to-use instruments of key elements (therapeutic alliance, client problem severity) • Encourage variety of research methods (qualitative & quantitative; group & single-case) • Create research networks of training sites using similar, pan-theoretical instruments

7. International Project on the Effectiveness of Psychotherapy and Psychotherapy Training (IPEPPT) • Formally initiated, June 2004, by: • Italian Coordinamento Nazionale Scuole di Psicoterapia (CNSP; >5,000 therapists) • Italian Federation of Psychotherapy Associations (FIAP; 21 psychotherapy associations: >10,000 therapists) • General Goal: To improve psychotherapy and psychotherapy training in a broad range of theoretical approaches by encouraging systematic research in therapy training institutes and university-based training clinics.

8. IPEPPT General Scientific Steering Committee • Robert Elliott, Scientific Director (University of Toledo-USA) • Alberto Zucconi, Coordinator (University of Siena-Italy) • David Orlinsky (University of Chicago-USA) • Franz Caspar (University of Freiburg) • Louis Castonguay (Pennsylvania State University-USA) • Glenys Parry (University of Sheffield-UK) • Bernhard Strauss (Friedrich Schiller University Jena-Germany)

9. IPEPPT: Current Status • Still in formation stage • Not a single study • The “Project” = Promoting practice-based research in Europe, North American and elsewhere • Finding partners • Creating/finding tools • E.g., conceptual/organizing concepts

10. IPEPPT Specific Objectives • 1. To construct a list of agreed-upon general pantheoretical recommendations for evaluating: • Key aspects of therapy, especially in training centers • Key aspects of therapy training outcome • Not a “Core Battery” • 2. To facilitate the development of specific treatment and training outcome protocols for particular: • Therapy approaches (e.g., Systemic therapy) • Client populations (e.g., people living with schizophrenia) • Linguistic/national groups (e.g., Italy) • 3. To facilitate national/international collaborations

11. IPEPPT Draft Research Framework • Such a project requires a guiding conceptual framework for determining what to measure and how to measure it • Work-in-progress • 8 measurement domains: • 4 Research themes • 2 Levels (Star design)

12. Framework: Eight Therapy Measurement Domains, with examples of key concepts

13. Structure: (1) “Star” Design • Main body of the star = General outcome/ process protocol • Shared by all orientations (General/ Pantheoretical) • Provides common metric • Star rays = Specialized protocols for different therapy approaches and different countries (Treatment/Population/Language Specific)

14. Dysfunctional Attitudes Self-Ideal Discrepancy CBT Experi- ential Target Problems Experiential Access Implicit Cognitive Biases Self-Esteem General problem severity Interpersonal/ relational issues Qualitative perceptions of change CCRT Change Relational Satisfaction Maturity of Defenses Family Environment Level of Object Relations Interpersonal Empathy Psycho- dynamic Family/ Couples “Star” Design for Sample Concepts within Therapy Outcome Domain for Studies of Four Different Therapies

15. Structure: (2) Nested Priority Lists • Not a single “core battery” • Allow flexibility while encouraging consistency within & across approaches • Three Levels of Priorities: • Measurement domains are prioritized • Within each measurement domain, key concepts are ranked by approximate importance • For each concept, available instruments are also described (researchers prioritize)

16. Framework: Eight Therapy Measurement Domains, with examples of key concepts

17. Example: General Therapy Outcome Domain • Key concepts in a possible recommended priority order: (“Star”) • (1) General problem severity (quantitative) • Give every 2 sessions to reduce data loss from drop-out • (2) Interpersonal/relational functioning • (3) Qualitative perceptions of change • (4) Individualized problems/goals • (5) Health care utilization/costs • (6) Quality of life/life satisfaction/well-being

18. Common General Symptom Severity Instruments

19. Framework: Eight Therapy Measurement Domains, with examples of key concepts

20. Example: General Therapy Process Domain • Key concepts in possible recommended priority order: • (1) Therapeutic alliance • (2) Therapist and client response modes • (3) Perceived helpful aspects of therapy • (4) Perceived session effectiveness

22. Different Levels of Research Protocol are Possible • I. Minimum Protocol • II. Systematic Case Study Protocol • III. Maximum Protocol Other Protocols: • IV. General Training Protocols • V. Specific Research Protocols

23. I. A Recommended Minimum Protocol: Applications • Easy to use: Limited to one measure from each of the first three research domains • Can use with own clients • Provides basic treatment monitoring for individuals & agencies • Other versions are possible (e.g., different outcome or process measures)

24. I. A Recommended Minimum Protocol: Elements • (1) General therapy outcome instrument • Client problem severity • Give at odd-numbered sessions (short form) • (2) General therapy process • Therapeutic alliance (use short from) • (3) Client/therapist background measure • Standard practice: • Client/ therapist demographics • Client diagnosis, presenting problems • Type of therapy

25. II. Systematic Case Study Protocol: Applications • Use for student case study requirements • Meets emerging standards for systematic single case research • New online journal: Pragmatic Case Studies in Psychotherapy (Rutgers University, Editor: Fishman)

26. II. Systematic Case Study Protocol: Elements • A. Therapy Outcome: • (1) Weekly/biweekly outcome measure • (2) At least one other quantitative outcome measure • (3) Qualitative outcome assessment (e.g., post-therapy interview) • B. Therapy Process • (1) Therapeutic alliance • (2) Detailed record of therapy (process notes and/or recordings) • (3) Qualitative perception of helpful aspects (post-session and/or post-therapy) • C. Client/therapist background • Client/ therapist demographics; client diagnosis, presenting problem; type of therapy

27. II. Systematic Case Study Protocol: Research Questions • (1) Did the client change substantially over the course of therapy? • (2) If the client changed, did therapy make a substantial contribution? • (3) What brought about the client’s changes?

28. II. Systematic Case Study Protocol: Emerging Evidence Standards • (1) Rich case record, including both quantitative & qualitative data • (2) Replication/convergence across methods • (3) Critical examination of alternative views (e.g., Hermeneutic Single Case Efficacy Design, Elliott, 2002): • Non-change explanations (e.g., measurement error) • Non-therapy explanations (e.g., extra-therapy events) • (4) Narrative coherence • Narrative model of predisposing and process factors • Use for generalizing to other cases

29. III. Maximum Protocol • Include measures of at least one concept in each of the eight domains • Appropriate for research centers (e.g., Center for the Study of Experiential Psychotherapy) • Also consortia of cooperating centers: • Each center measure some variables

30. IV. General Training Research Protocols: Issues • Outcomes of therapy training not well understood • Difficulties: • Lack of agreed-upon measures of therapist functioning and skill • Must measure therapist change longitudinally over several years of training • Possible applications: • Use research to improve training • Meet requirements of accrediting and funding agencies

31. IV. General Training Research Protocols: Promising Concepts • General therapist facilitative interpersonal skills (e.g., coping with common difficulties) • Quality of therapist professional involvement and growth (e.g., Orlinsky; Collaborative Research Network [CRN]) • Qualitative perceptions of effects and important aspects of training (e.g., qualitative interviews) • Change in therapist self concept (e.g., Scilligo, SASB Introject scales)

32. V. Specific Protocols • = Star rays • Applications: For specific theoretical approaches, client populations, or language groups • Requires working committee for each group • Identify relevant therapy outcomes, processes, background variables (or training outcomes) • Do protocol and measure development research • Establish virtual communities for exchanging ideas

33. V. Specific Protocol Example: • Person-Centered and Experiential Psychotherapy International Research Group (PCEP-IRG) • Current core members: • University of Toledo (Elliott & team) • Ohio University (Anderson & team) • Katholieke Universiteit Leuven (Leijssen & team) • Universities of Strathclyde & Abertay, Scotland (McLeod, Cooper)

34. V. PCEP-IRG Outcome Protocol: Promising Developments • Center for the Study of Experiential Therapy Research Protocol (CSEP- 2): • Self--determined problems/goals: • Personal Questionnaire (PQ-10) • Self-concept (content & coherence) • Qualitative Self-Description interview • Tennessee Self-Concept Scale 2 (long, short forms) • Experiential processing: • Toronto Alexithymia Scale (TAS-20) • Need positive mental health measures, self-coherence, etc.

35. V. Specific Training Research Protocols • Some Possible Types of Specific Training Outcomes: • Treatment-specific intervention skill • Case formulation skill • Therapist personal development (e.g., maturity, identification with orientation, values)

36. Promising New Therapy Research Methods Make this Work Possible • Systematic qualitative research methods • Interpretive single case designs (Fishman, Elliott) • Using early outcome to identify & repair problems (Lambert: Signal alarm methods) • New, powerful psychometric methods (Rasch analysis/Item Response Theory) • Virtual communities (Community Zero)

37. Invitation to Dialogue - 1 1) Provide comments and suggestions on the framework & concepts presented here: Robert.Elliott@utoledo.edu 2) Form or join online discussion groups or virtual communities • Closed sites; must apply for membership • General info: www.communityzero.com/ipeppt • Example: www.communityzero.com/pcepirp 3) Begin implementing the minimum protocol design with your own clients and in your own training setting.

38. Invitation to Dialogue - 2 4) Convert traditional case presentation training requirements into systematic case study exercises 5) Help with translations of key research instruments 6) Contribute to psychometric research: • Improve existing instruments • Equate different instruments for same concepts 7) Collaborate with groups with similar interests to generate data for pooling.