Treatment Adherence and Outcome in Psychotherapy - First Results From the PAP-S-Study

1. Treatment Adherence andOutcome in Psychotherapy -First Results From the PAP-S-Study Volker Tschuschke, Aureliano Crameri, Margit Koemeda, Peter Schulthess, Agnes von Wyl, Rainer Weber, Miriam Köhler, Katharina Muth, Jessica Mallmann, Pia Pulte, Antonia Roth-Ehrang

2. Treatment integrity in psychotherapy • ... is essential for psychotherapy research • ... it is obvious that no control of treatment delivery by therapists in research studies leads to questionnable results • ... therefore process research is indispensable • ... there is a need to examine and determine that treatments said to have been provided were indeed the treatments provided (Kendall et al., 2004) • ... and its implications for treatment outcomes is still unknown. A review of relevant psychological journals revealed that systematically addressed treatment integrity procedures are virtually absent in the literature (Perepletchikova et al., 2007), only 3.5% of the studies or less than 6% (Waltz et al., 1993) FMPP-IFP Congress Luzern 2010

3. Process Study of Outpatient Psychotherapy Treatment Switzerland (PAP-S) • process-outcome study (start in 2006, end in 2012) • large outcome battery (pre, post, follow-up) • diagnostic assessments by trained raters (ICD, OPD, SKID-II) in major Swiss cities (Basel, Berne, Lausanne, Luzern, St. Gallen, Zurich) • several process measures administered each 5th session • 11 different therapy schools • objective ratings of treatment adherence • each session being audiotaped • three sessions (beginning, middle, and end of treatment) will be depicted for integrity ratings • raters are blind toward school affiliation of therapists and patient data • 4 out of 5 raters are trained psychotherapists • so far the average interrater reliability (Kappa) lies between .50 and .60 for three raters (codings of event-by-event, each therapist intervention, approx. 30 - 50 ratings per session) • a detailed rating manual including 101 intervention categories was being developed • rating manual‘s categories comprise 7 - 13 intervention categories from each school including 22 nonspecific intervention categories • each therapist intervention is being rated taking into account context (contents of patient‘s utterances) • therapists code their school specific behavior after each session using a check-list of their used particular school categories including nonspecific interventions (subjective ratings) FMPP-IFP Congress Luzern 2010

4. Ratings • concept true intervention ratings in relationship to nonspecific intervention ratings • example Gestalt Therapy session: • raw score interventions true to Gestalt therapy = 35 • 13 intervention categories • raw score nonspecific interventions = 28 • 22 intervention categories • scores > 1 mean more specific codings compared to nonspecific, and scores < 1 mean less specific codings compared to nonspecific ones 35 / 13 = 2.12 28 / 22 FMPP-IFP Congress Luzern 2010

5. Score FMPP-IFP Congress Luzern 2010

6. Score Sessions FMPP-IFP Congress Luzern 2010

7. Score FMPP-IFP Congress Luzern 2010

8. Score Sessions FMPP-IFP Congress Luzern 2010

9. Conclusions • subjective ratings by therapists tend to estimate treatment integrity significantly higher compared to observer ratings as is found in other studies (Martino et al., 2009) • even therapists consider a fairly high amount of nonspecific ingedients in their intervention style • observers‘ ratings suggest that most of the intervention activity of psychotherapists is nonspecific, thus leaving therapists‘ ratings as probably not valid (Waltz et al., 1993) • although both therapists seem to have little treatment integrity, their treatments seam to be efficient (at least the given two cases) • both therapists used mainly unique and essential, essential but not unique and acceptable but not necessary interventions and very little proscribed interventions • so far the question is unanswered whether higher treatment integrities are correlated with better outcomes (the PAP-S Study will address this question) • it may also be the case that more severely disturbed patients - complex interventions, demanding, tedious, extended in time etc - generate serious problems in maintaining treatment integrity (Yeaton and Sechrest, 1992) • the PAP-S-study will be able to look at different disturbance levels and therapists‘ techniques in these treatments FMPP-IFP Congress Luzern 2010

10. References • Kendall et al. (2004): Methodology, design, and evaluation in psychotherapy research. In Lambert (ed.): Handbook of Psychotherapy and Behavior Change. 5th ed., John Wiley & Sons, New York, pp 16-43 • Martino et al. (2009): Correspondence of motivational enhancement treatment integrity ratings among therapists, supervisors, and observers. • Psychotherapy Research 19: 181-193 • Perepletchikova et al. (2009): Barriers to implementing treatment integrity procedures: survey of treatment outcome researchers. • Journal of Consulting and Clinical Psychology 77: 212-218 • Waltz et al. (1993): Testing the integrity of a psychotherapy protocol: assessment of adherence abd competence. Journal of Consulting and Clinical Psychology 61: 620-630 • Yeaton / Sechrest (1992): Critical dimensions in the choice and maintainance of successful treatments: strength, integrity, and effectiveness. In Kazdin (ed.): Methodolgical Issues & Strategies in Clinical Research. American Psychological Association, Washington D.C., pp. 137-156 FMPP-IFP Congress Luzern 2010