Initiating International Collaborative Research Projects in Mental Health:

1. Initiating International Collaborative Research Projects in Mental Health: A Progress Report on the International Project on the Effectiveness of Psychotherapy and Psychotherapy Training (IPEPPT)

2. International Project on the Effectiveness of Psychotherapy and Psychotherapy Training (IPEPPT) • Formally initiated, June 2004, by: • Italian Coordinamento Nazionale Scuole di Psicoterapia (CNSP; >5,000 therapists) • Italian Federation of Psychotherapy Associations (FIAP; 21 psychotherapy associations: >10,000 therapists) • General Goal: To improve psychotherapy and psychotherapy training in a broad range of theoretical approaches by encouraging systematic research in therapy training institutes and university-based training clinics. • Political purpose: Increase status of involved organizations

3. IPEPPT General Scientific Steering Committee • Robert Elliott, Scientific Director (University of Toledo-USA) • Alberto Zucconi, Coordinator (University of Siena-Italy) • David Orlinsky (University of Chicago-USA) • Franz Caspar (University of Freiburg) • Louis Castonguay (Pennsylvania State University-USA) • Glenys Parry (University of Sheffield-UK) • Bernhard Strauss (Friedrich Schiller University Jena-Germany)

4. IPEPPT: Current Status • Still in formation stage • Not a single study • The “Project” = Promoting practice-based research in Europe, North American and elsewhere • Finding partners/applying for EU funding • Creating/finding tools • E.g., conceptual/organizing concepts • Finding/translating instruments • KU Leuven piloting Dutch-language systematic case study protocol

5. IPEPPT Immediate Objectives • 1. To construct a list of agreed-upon general pantheoretical recommendations for evaluating: • Key aspects of therapy, especially in training centers • Key aspects of therapy training outcome • Not a “Core Battery” • 2. To facilitate the development of specific treatment and training outcome protocols for particular: • Therapy approaches (e.g., Systemic therapy) • Client populations (e.g., people living with schizophrenia) • Linguistic/national groups (e.g., Italy) • 3. To facilitate national/international collaborations

6. IPEPPT Draft Research Framework • Such a project requires a guiding conceptual framework for determining what to measure and how to measure it • Work-in-progress • 8 measurement domains: • 4 Research themes • 2 Levels (Star design)

7. Framework: Eight Therapy Measurement Domains, with examples of key concepts

8. Structure: (1) “Star” Design • Main body of the star = General outcome/ process protocol • Shared by all orientations (General/ Pantheoretical) • Provides common metric • Star rays = Specialized protocols for different therapy approaches and different countries (Treatment/Population/Language Specific)

9. Dysfunctional Attitudes Self-Ideal Discrepancy CBT Experi- ential Target Problems Experiential Access Implicit Cognitive Biases Self-Esteem General problem severity Interpersonal/ relational issues Qualitative perceptions of change CCRT Change Relational Satisfaction Maturity of Defenses Family Environment Level of Object Relations Interpersonal Empathy Psycho- dynamic Family/ Couples “Star” Design for Sample Concepts within Therapy Outcome Domain for Studies of Four Different Therapies

10. Structure: (2) Nested Priority Lists • Not a single “core battery” • Allow flexibility while encouraging consistency within & across approaches • Three Levels of Priorities: • Measurement domains are prioritized • Within each measurement domain, key concepts are ranked by approximate importance • For each concept, available instruments are also described (researchers prioritize)

11. Framework: Eight Therapy Measurement Domains, with examples of key concepts

12. Example: General Therapy Outcome Domain • Key concepts in a possible recommended priority order: (“Star”) • (1) General problem severity (quantitative) • Give every 2 sessions to reduce data loss from drop-out • (2) Interpersonal/relational functioning • (3) Qualitative perceptions of change • (4) Individualized problems/goals • (5) Health care utilization/costs • (6) Quality of life/life satisfaction/well-being

13. Common Client Problem Severity Instruments

14. Framework: Eight Therapy Measurement Domains, with examples of key concepts

15. Example: General Therapy Process Domain • Key concepts in possible recommended priority order: • (1) Therapeutic alliance • (2) Perceived helpful aspects of therapy • (3) Therapist and client response modes • (4) Perceived session effectiveness

17. Different Levels of Research Protocol are Possible • I. Minimum Protocol • II. Systematic Case Study Protocol • III. Maximum Protocol Other Protocols: • IV. General Training Protocols • V. Specific Treatment Protocols

18. I. A Recommended Minimum Protocol: Applications • Easy to use: Limited to one measure from each of the first three research domains • Can use with own clients • Provides basic treatment monitoring for individuals & agencies • Other versions are possible (e.g., different outcome or process measures)

19. I. A Recommended Minimum Protocol: Elements • (1) General therapy outcome instrument • Client problem severity • Give at odd-numbered sessions (short form) • (2) General therapy process • Therapeutic alliance (use short from) • (3) Client/therapist background measure • Standard practice: • Client/ therapist demographics • Client diagnosis, presenting problems • Type of therapy

20. II. Systematic Case Study Protocol: Applications • Use for student case study requirements • Meets emerging standards for systematic single case research • New online journal: Pragmatic Case Studies in Psychotherapy (Rutgers University, Editor: Fishman)

21. II. Systematic Case Study Protocol: Elements • A. Therapy Outcome: • (1) Weekly/biweekly outcome measure • (2) At least one other quantitative outcome measure • (3) Qualitative outcome assessment (e.g., post-therapy interview) • B. Therapy Process • (1) Therapeutic alliance • (2) Detailed record of therapy (process notes and/or recordings) • (3) Qualitative perception of helpful aspects (post-session and/or post-therapy) • C. Client/therapist background • Client/ therapist demographics; client diagnosis, presenting problem; type of therapy

22. II. Systematic Case Study Protocol: Research Questions • (1) Did the client change substantially over the course of therapy? • (2) If the client changed, did therapy make a substantial contribution? • (3) What brought about the client’s changes?

23. II. Systematic Case Study Protocol: Emerging Evidence Standards • (1) Rich case record, including both quantitative & qualitative data • (2) Replication/convergence across methods • (3) Critical examination of alternative views (e.g., Hermeneutic Single Case Efficacy Design, Elliott, 2002): • Non-change explanations (e.g., measurement error) • Non-therapy explanations (e.g., extra-therapy events) • (4) Narrative coherence • Narrative model of predisposing and process factors • Use for generalizing to other cases

24. III. Maximum Protocol • Include measures of at least one concept in each of the eight domains • Appropriate for research centers (e.g., Center for the Study of Experiential Psychotherapy) • Also consortia of cooperating centers: • Each center measure some variables

25. IV. General Training Research Protocols: Issues • Outcomes of therapy training not well understood • Difficulties: • Lack of agreed-upon measures of therapist functioning and skill • Must measure therapist change longitudinally over several years of training • Possible applications: • Use research to improve training • Meet requirements of accrediting and funding agencies

26. IV. General Training Research Protocols: Promising Concepts • Quality of therapist professional involvement and growth (e.g., Orlinsky &Rønnestad; Collaborative Research Network [CRN]) • Qualitative perceptions of effects and important aspects of training (e.g., qualitative interviews) • General therapist facilitative interpersonal skills (e.g., coping with common difficulties) • Change in therapist self concept (e.g., Scilligo, SASB Introject scales)

27. V. Specific Treatment Protocols • = Star rays • Applications: For specific theoretical approaches, client populations, or language groups • Requires working committee for each group • Identify relevant therapy outcomes, processes, background variables (or training outcomes) • Do protocol and measure development research • Establish virtual communities for exchanging ideas

28. V. Specific Protocol Example: • Person-Centered and Experiential Psychotherapy International Research Group (PCEP-IRG) • Current core members: • University of Toledo (Elliott & team) • Ohio University (Anderson & team) • Katholieke Universiteit Leuven (Leijssen & team) • Universities of Strathclyde & Abertay, Scotland (McLeod, Cooper)

29. V. PCEP-IRG Outcome Protocol: Promising Developments • Center for the Study of Experiential Therapy Research Protocol (CSEP- 2): • Self-determined problems/goals: • Personal Questionnaire (PQ-10) • Self-concept (content & coherence) • Qualitative Self-Description interview • Tennessee Self-Concept Scale 2 (long, short forms) • Experiential processing: • Toronto Alexithymia Scale (TAS-20) • Need positive mental health measures, self-coherence, etc.

30. V. Specific Training Research Protocols • Some Possible Types of Specific Training Outcomes: • Mastery of therapy theory/knowledge • Case formulation skill (e.g., use clinical vignettes) • Treatment-specific intervention skill • Therapist personal development (e.g., maturity, identification with orientation, values)

31. Promising New Therapy Research Methods Make this Work Possible • Systematic qualitative research methods • Interpretive single case designs (Fishman, Elliott) • Using early outcome to identify & repair problems (Lambert: Signal alarm methods) • New, powerful psychometric methods (Rasch analysis/Item Response Theory) • Virtual communities (Community Zero)

32. Lessons Learned So Far • Importance of vision and leadership • Leading by example is important • Both immediate and longterm payoffs offs are important • These things eat up a lot of time • Grant funding is not necessary and can complicate broad collaborations • Clinically interesting examples are useful to persuading skeptical possible participants • Progress is possible in creating a research climate in training institutes and schools

33. Invitation to Dialogue - 1 1) Provide comments and suggestions on the framework & concepts presented here: Robert.Elliott@utoledo.edu 2) Form or join online discussion groups or virtual communities • Closed sites; must apply for membership • General info: www.communityzero.com/ipepp • Example: www.communityzero.com/pcepirp • Open site under development: www.ipeppt.net 3) Begin implementing the minimum protocol design with your own clients and in your own training setting.

34. Invitation to Dialogue - 2 4) Convert traditional case presentation training requirements into systematic case study exercises 5) Help with translations of key research instruments 6) Contribute to psychometric research: • Improve existing instruments • Equate different instruments for same concepts 7) Collaborate with groups with similar interests to generate data for pooling.