Tracking Client Outcomes as Therapists Mature from Trainee to Professional

1. Tracking Client Outcomes as Therapists Mature from Trainee to Professional Tyler R. Pedersen, Ph.D. Kara Cattani, Ph.D. Zach Ellison Brigham Young University Counseling & Career Center

2. What does the therapist contribute to outcome? • The therapist contribution • Techniques • Skillfulness • Personal qualities • As practitioners, we would like to believe that we not only influence outcomes, but that the outcomes of our clients improve with experience or training • Evidence has been hard to come by • Mixed results in the literature with a few studies showing a positive relationship between experience and outcome but most meta-analyses suggesting little support for the belief that experienced clinicians have better outcomes • Smith & Glass, 1977; Shapiro & Shapiro, 1982; Stein & Lambert, 1995

3. Difficulties inherent in studying the therapist contribution • Methodological issues: • Few studies have been designed to explicitly investigate therapist experience. This is looked at post-hoc in studies designed to identify if therapy works. • Designs are cross-sectional, comparing different groups of therapists. In such instances, it is hard to disentangle therapist experience from individual differences between therapists on a variety of attributes (e.g. personality, theoretical orientation) • Experience level is typically classified by degree, which may ignore variance attributed to years in practice. • Masters level vs PhD level therapists

4. How this study improves on past studies • Experience was classified by years trained rather than comparing different degrees • All therapists trained in Ph.D. programs in psychology • Experience levels could be compared more easily • Similar training time line/pattern of skill acquisition over time • Training was broken down into four different levels or status • The same therapists were tracked over time, providing a longitudinal/within-subjects design. • Decrease in error variance • Therapists had many years experience compared to most studies (minimum of 8, maximum of 11) • Therapists in the study had seen many clients (minimum of 72)

5. Method • Participants • 14 Therapists • All therapists provided services at the CCC at some point during their graduate training as well as post licensure. • 5 females, 9 males • All Ph.D.s: 5 Clinical Psych, 9 Counseling Psych • Range of years of total experience = 8-11 years • Minimum number of clients tracked was 72, maximum was 1015, mean was 383.29, (SD=269.29); median=348.5 • Client participants included all students/patients who agreed to participate in archival outcome research and were seen by one of the participating therapists during the years 1996-2011 • Database • Since 1996, 304 different professionals and trainees have provided treatment. The CCC's database includes approximately 273,000 appointments, with progress notes; over 181,000 individual therapy appointments; over 184,000 OQ-45s have been gathered since 1996.

6. Measure of Outcome: Outcome Questionnaire-45 45 items across 4 domains Yields a total score reflecting amount of distress a person has been experiencing during the last 7 days Scores range from 0 to 180 Higher scores signify more distress 64 represents the cutoff between dysfunctional & functional populations

7. Outcome Questionnaire-45

8. Results

9. Results There is a significant effect of training status on OQ change at the p<.05 level. F(3,5362)=3.73, p=0.011 Post-hoc Tukey HSD tests showed that licensed professionals OQ change scores are 2.18 points smaller than practicum students OQ change scores at the p<.05 level. No other comparisons were significant. To investigate this data more thoroughly, we need to account for clustering of clients within therapists. To account for the clustering, doing a repeated measures ANOVA or HLM would be preferable. There was no significant difference in pretest OQ scores between training status.

10. Mean Change, Across Experience Levels, For Clients Seen Only in Individual Therapy

11. Individual Therapist Patterns

12. Individual Therapist Patterns

13. Individual Therapist Patterns

14. Individual Therapist Patterns

15. Individual Therapist Patterns

16. Interesting Findings • Therapist Experience was NOT associated with better client outcomes. • As years of therapist experience increased, mean change in client outcome scores trended downward. • Licensed professionals change scores were significantly lower than practicum students. • There was ambiguous support for small differences in OQ scores at intake across training status, with licensed professional averaging higher intake scores (seeing more distressed clients). • What’s going on??? • Patterns in treatment in our center: • Most experienced therapists have increased client load, fewer opportunities for consultation, and they see clients less frequently • The mean level of client distress was higher for the Professional therapists

17. Limitations • Difficulty measuring/operationalizing experience • Training status may not effectively measure ‘experience’ • Experience may be better captured by looking at total number of sessions provided, familiarity or specialty with a specific type of intervention or patient problem • Difficult to eliminate factors, other than experience, that influence client treatment (e.g. referrals to group, medication, biofeedback, frequency of visits, therapist consultation) • Problems inherent in ‘real world’ research • This study investigated patterns of outcome in a counseling center. • In contrast to controlled studies, the therapy followed many varied courses. • For instance, in this study, different therapists provided different numbers of sessions at each level of training. • Length of therapy and gaps in sessions varied • Patients changed providers from time to time • There were only 14 therapists • Need to account for clustering of the clients within therapists, perhaps by using hierarchical linear modeling.