Gary M. Burlingame, Kris Kristensen & Matt Bishop Brigham Young University

1. Using deviations from expected change trajectories adjusted for initial severity to predict non-response in children & adolescents GaryM. Burlingame, Kris Kristensen & Matt Bishop Brigham Young University

2. Overview • Outcome instrument—Development, subscales & norms • Patient Change—Categorization & example of typical change trajectories • Algorithms for predicting change • Preliminary test of algorithms

3. Instrument—Youth Outcome Questionnaire • Brief—64-item parent completed (7 minutes) • Psychometrically Sound—Alpha .97, stability =.84 • Ecologically Valid—Focus groups produced items • Grounded in Literature—Subscale from meta-analyses • Breadth of use—Language, coverage in U.S., settings • No age or gender difference X total score • Broad-band of Symptoms • Total score varies by normative sample

4. Instrument—Subscales • YOQ Subscale Description • Intrapersonal Distress (ID): anxiety, depression, fearfulness, etc. • Somatic (S): headache, stomachache, bowel, dizziness, etc. • Interpersonal Relations (IR): attitude, communication and interaction with parents, adults, and peers • Social Problems (SP): delinquent or aggressive behaviors, breaking social mores • Behavioral Dysfunction (BD): organize and completes tasks, handle frustration, impulsivity, inattention, etc. • Critical Items (CI): paranoid ideations, suicide, hallucinatory, delusions, etc.

5. Instrument—Norms using 95% Confidence Intervals Residential Range 110.7 to 118.7 m=114.7, SE=1.96, N=287 Inpatient Range 96 to 108 m=102, SE=3.05, N=131 Outpatient Range 74.8 to 78.8 m=76.8, SE= .92, N=342 Juvenile Justice Range 47.1 to 53.1 m=50.12, SE= 1.44, N=719 Cut off Score = 46 Community Sample Range 20.8 to 23 m=21.43, SE= .80, N=1091

6. Improvement Symptom reduction of 13 or more points Recovery Improvement plus final Y-OQ score in normal range No Change Change of less than 13 points Deterioration Increase in symptoms of 13 or more points at treatment termination Change—Classification of Patient Change

7. Change—Avg. days between adm. & change trajectory X category

8. Algorithm--Rational • Experts imagine patients beginning treatment at different levels of disturbance • Use information about expected treatment response to evaluate change after various doses of treatment • Experts reach consensus about patient treatment responses that are alarming • Creation of decision matrix

9. Algorithm--Decision Rules • Green Rule:The rate of change the patient is making is in the adequate range. No change in treatment plan is recommended. • White Rule: The patient is functioning in the normal range. Consider termination. • Yellow Rule: Rate of change less than expected. Consider altering treatment plan, e.g. intensifying treatment shift strategies, etc. • Red Rule: The patient is not making the expected level of progress. Chances are they may drop out or have a negative outcome. Steps should be taken to carefully review the care and decide on a new course of action.

10. 160 155 150 145 140 135 130 125 120 115 110 105 100 95 90 85 80 75 70 65 60 55 50 45 40 35 30 25 20 15 10 30 25 20 15 10 5 0 -5 -10 -15 -20 -25 -30 -35 -40 Example of Rationale Method Y-OQ Early Warning Sessions 2-4 Initial Score Red Green Yellow White Change Score

11. 160 155 150 145 140 135 130 125 120 115 110 105 100 95 90 85 80 75 70 65 60 55 50 45 40 35 30 25 20 15 10 30 25 20 15 10 5 0 -5 -10 -15 -20 -25 -30 -35 -40 Example of Rationale Method Y-OQ Early Warning Sessions 9+ Red Initial Score Green Yellow White Change Score

12. Preliminary Test--Sample • Sample: • No more than 14-days elapsed between administrations • Out of 4560, 301 met criteria representing residential (n=149) and outpatients (n=152) • Status at end of treatment • 167 (55.5%) ended treatment with a positive outcome defined as • Recovered—n = 52 or 17.3% • Improved—n = 115 or 38.2% • 108 (35.9%) showed no change • 26 (8.6%) ended with a negative outcome defined as deterioration

13. Accuracy of Prediction PREDICTED POSITIVE OUTCOME N % Non-res. pts PREDICTED NEGATIVE OUTCOME N % Res. Pts. TOTAL N % Totals HIT 111 (85%) 129 (75%) 240 (80%) Actual Positive Outcome 223a 81.1 HIT 52b 18.9 False_Neg 275 91.4 MISS 38 (15%) 23 (25%) 61 (20%) Actual Negative Outcome 9 34.6 False_Pos 17 65.4 HIT 26 8.6 Marginal Totals 149 152 301 Total Number Classified 232 77.1 69 22.9 301 100 Comparison of Predicted vs. Actual Outcome—All cases Overall hit rate = 80%; Overall miss rate = 20% Notes. a. Includes 76 patients who ended treatment with no reliable change b. Includes 32 patients who ended treatment with no reliable change * Predicted Positive Outcome includes both white and green signals given after 2nd YOQ score * Predicted Negative Outcome includes both yellow and red signals given after 2nd YOQ score * Actual Positive Outcome includes Reliably Recovered, Reliably Better, and No-Reliable-Change * Actual Negative Outcome includes both Reliably Worse and Reliably Deteriorated

14. Predicted Positive Outcome N % Predicted Negative Outcome N % Total without reliable change (108) N % Actual Positive Outcome 147 88.0 HIT 20 12.0 False_Neg 167 86.5 Actual Negative Outcome 9 34.6 False_Pos 17 65.4 HIT 26 13.5 Total Number Classified 156 80.0 37 19.2 193 100 Comparison of Predicted vs. Actual Outcome—Exc. No change cases Hit rate = 85%; Miss rate = 15% Notes. * Predicted Positive Outcome includes both white and green signals given after 2nd YOQ score * Predicted Negative Outcome includes both yellow and red signals given after 2nd YOQ score * Actual Positive Outcome includes both Recovered and Reliably Improved * Actual Negative Outcome includes Reliably Deteriorated

15. Better/Recovered No reliable change Worse/Deteriorated Total Signal type N % N % N % Yellow 10 33.3 16 53.3 4 13.3 30 Red 10 25.6 16 41.0 13 33.3 39 Total 20 29.0 32 46.4 17 25.0 69 Final Outcome for Signal Cases Outcome in signal-alarm cases who received a red or yellow signal

16. Conclusions • Dose response curves can be created to plot change trajectory for C/A • Change varies by age & setting • Tentative support for rationale algorithms for C/A • Similar pattern of accuracy for adults & replication of setting finding