Improving Psychotherapy Outcome for Poorly Responding Clients via Measuring, Monitoring & Feedback

1. Improving Psychotherapy Outcome for Poorly Responding Clients via Measuring, Monitoring & Feedback

2. Overview Define Components of OUTCOME Develop Measure Define Treatment Success Define Treatment Failure Develop Failure-Alert Recognition System Apply Procedures to Routine Practice in Experimental Studies

3. Components of Outcome Symptomatic Complaints Interpersonal Functioning Social Role Functioning Quality of Life

4. The OUTCOME QUESTIONNAIRE-45 • Reliability- .93/.84 • Validity- .55-.88 • Sensitive to patient change

5. 75 70 65 60 Total OQ Score 55 50 45 40 1 2 3 4 5 6 7 8 9 OQ Administration High Student Control EAP Low Student Control Patient and Nonpatient Total Scores for the OQ

6. 2 1.8 1.6 1.4 OQ Item 20 Score 1.2 1 0.8 0.6 1 2 3 4 5 6 7 8 9 OQ Administration High Student Control EAP Low Student Control Scores for Item 20 “I feel loved and wanted” (Poor discrimination)

7. 2 1.8 1.6 1.4 OQ Item 23 Score 1.2 1 0.8 0.6 1 2 3 4 5 6 7 8 9 OQ Administration High Student Control EAP Low Student Control Scores for Item 23“I feel hopeless about the future” (Good discrimination)

8. Defining Treatment Success & Failure • Clinically significant change = Recovery • Based on Jacobson & Truax Two step definition • Reliable change = 14 points • Functional Cut-off = 63 • Failure(Deterioration)= 14 points worse

9. Movement into the Functional Distribution

10. Level of Psychopathology: 95% Confidence Intervals 100 Inpatient (M=93.80, S=23.72, N=110) 90 MH Center (M=86.07, S=19.33, N=100) 80 MBH (M=76.89, S=22.46, N=1395) 70 EAP (M=69.74, S=21.59, N=3871) Clinically Significant Cut-off Score 60 50 Community Sample (M=42.50, S=17.30, N=210) 40 0

11. Identifying Cases for Review High Little or No Need (75%) Actual Change Moderate Need (18%) Great Need (7%) Low Low High Predicted Change

12. THE RATIONAL METHOD • EXPERTS IMAGINE PATIENTS BEGINNING TREATMENT AT DIFFERENT LEVELS OF DISTURBANCE • USE INFORMATION ABOUT EXPECTED TREATMENT RESPONSE TO EVALUATE CHANGE AFTER VARIOUS DOSES OF TREATMENT • EXPERTS REACH CONSENSUS ABOUT PATIENT TREATMENT RESPONSES THAT ARE ALARMING • CREATION OF DECISION MATRIX

14. Empirical method using expected recovery curves • Data on session by session change and termination outcome are subjected to HLM statistical analysis • Tolerance bands are created based on 10% and 15% slowest responding patients at each session for 50 score groupings • Patients who cross boundary at any session are identified as alarm-signal cases and targeted for special treatment

16. Subject 38 0 -15 -18

17. Subject 29 +2 -6 -17

18. Subject 19

19. Decision Rules/Warning Report • Yellow Rule: Rate of change less than expected. Consider altering treatment plan, e.g. intensifying treatment shift strategies, etc. • Red Rule: The patient is not making the expected level of progress. Chances are they may drop out or have a negative outcome. Steps should be taken to carefully review the care and decide on a new course of action.

20. Decision Rules/Warning Report • Green Rule:The rate of change the patient is making is in the adequate range. No change in treatment plan is recommended. • White Rule: The patient is functioning in the normal range. Consider termination.

21. Identification of Deterioration • Of 456 clients that were studied, 36 deteriorated. • The empirical method identified all 36(100%). • The rational method identified 29 ( 82 %), but • The empirical method “misidentified” 83 & the • Rational “misidentified” 95 as treatment failures who did not deteriorate. • The overall hit rate across all clients was 390/492 (79%) for rational and 409/492(83%) for the empirical method

22. OUTCOME OF MISCLASSIFIED SIGNAL-ALARM CASES

23. OUTCOME IN SIGNAL-ALARM CASES WHO RECEIVED A RED VS YELLOW WARNING

24. Can Clinical Outcomes be Enhanced by Providing Feedback on Patient Progress to Therapists?

25. Participants • 609 clients randomly assigned to experimental or control conditions • Of 307 experimentals 35 were expected to have a poor outcome (11.4 %) • Of 302 controls 31 were expected to have a poor outcome (10.3 %)

27. 64 Cut-off

28. 64 Cut-off

29. Conclusions • Feedback to therapists increases the number of sessions given to clients who are predicted to prematurely dropout of psychotherapy • Feedback significantly improved outcomes in these same clients with a moderate effect size • Future research should test the effects of a stronger experimental manipulation

30. Clinical Support Tools Decision TreeRed or Yellow Feedback Cases Assess therapeutic alliance with HAq-II. Does the client report concerns with the therapeutic alliance? See Therapeutic Alliance Interventions handout. YES NO Assess readiness to change with the SCS. Does the client report being in a precontemplation or contemplation stage of readiness to change See Readiness to Change Interventions handout. YES NO Assess social support resources with the MSPSS. Does the client report low social support? See Social Support Interventions handout. YES NO Reassess the diagnostic formulation. Is there an effective treatment option that has not been attempted? Consult relevant resources and alter the treatment plan. YES NO Is medication an effective treatment option? Refer for psychiatric consultation. YES

31. Testing Feedback • On the Revised Helping Alliance Questionnaire (HAq-II), client SSN: rated the therapeutic alliance as average with a score of . The average HAq-II score is 5.15 (SD = .58). • On the Revised Helping Alliance Questionnaire (HAq-II), client SSN: rated the therapeutic alliance as below average with a score of . The average HAq-II score is 5.15 (SD = .58).

32. Therapeutic Alliance Interventions • Discuss therapeutic alliance with client • Give and ask for feedback on relationship • Spend more time exploring client’s experience • Discuss shared experiences • Reassess/Agree on therapeutic tasks and goals • Clarify possible misunderstandings • Give more positive feedback • Use more empathic engagements • Discuss therapist and therapeutic style match • Process transference

33. Testing Feedback • On the Stages of Change Scale (SCS), client SSN: is currently reporting a precontemplation readiness to change. Individuals in the precontemplation stage are not considering or choosing to change themselves. • On the Stages of Change Scale (SCS), client SSN: is currently reporting a contemplation readiness to change. Individuals in the contemplation stage are aware of problems and may seek help by participating in counseling. • On the Stages of Change Scale (SCS), client SSN: is currently reporting an action readiness to change. Individuals in the action stage are actively working on their concerns. • On the Stages of Change Scale (SCS), client SSN: is currently reporting a maintenance readiness to change. Individuals in the maintenance stage are consolidating changes made in counseling and preparing for termination.

34. Readiness to Change Interventions • Discuss readiness to change with client • Give and ask for feedback about readiness for change • Adjust goals and tasks to be challenging, but not too difficult • Discuss consequences of changing or not changing • Discuss the processes involved with change and specific skills that help

35. Testing Feedback • On the Multidimensional Scale of Perceived Social Support (MSPSS), client SSN: rated social support from family, friends, and significant others as average with a total score of and scores of , , and , respectively, on the subscales. The average MSPSS scores are 5.58 (SD = 1.07) total, 5.31 (SD = 1.46) family, 5.5 (SD = 1.25) friends, and 5.94 (SD = 1.34) for significant others.  • On the Multidimensional Scale of Perceived Social Support (MSPSS), client SSN: rated social support from at least one of the family, friends, significant others subscales as below average with a total score of and scores of , , and , respectively, on the subscales. The average MSPSS scores are 5.58 (SD = 1.07) total, 5.31 (SD = 1.46) family, 5.5 (SD = 1.25) friends, and 5.94 (SD = 1.34) for significant others.

36. Social Support Interventions • Refer to group therapy • Refer to biofeedback lab • Refer to assertiveness workshops • Role play social situations • Assign related homework • Assess client’s self beliefs • Bring others to sessions • Encourage activities with family, friends, roommates, and faculty • Encourage involvement in clubs, service projects, and hobbies • Work on concerns related to trusting others

37. Clinical Support Tools • To improve the outcome findings for NOT patients, therapists were provided with Clinical Support Tools (CST) in addition to feedback on the patient’s progress • The CST were proposed as an empirically based problem solving strategy to assist therapists, regardless of their orientation, with NOT patients

39. Results (Outcome)

41. Patient Feedback Message Please note that the following information is based on your responses to the questionnaire that you have completed prior to each therapy session. It appears that you have not experienced a reduced level of distress. Because you may not be experiencing the expected rate of progress, it is possible that you have even considered terminating treatment, believing that therapy may not be helpful for you. Although you have yet to experience much relief from therapy, it is still early in treatment and there is the potential for future improvement. However, we urge you to openly discuss any concerns that you may be having about therapy with your therapist because there are strategies that can be used to help you receive the most out of your therapy. It may also require your willingness to complete additional questionnaires that may shed light about why you are not experiencing the expected rate of progress.