Longitudinal Outcomes in Private, Regulated Residential Programs:

1. How are youth one year later? Longitudinal Outcomes in Private, Regulated Residential Programs: IECA Convention Boston, MA April 27, 2007 Ellen Behrens, Ph.D.

2. Goals • 1. Review the published research • 2. Describe the study • 3. Present highlights of the study

3. 1. Review the published research

4. Residential Outcome Research • Most outcome research based on public programs (Child Protective Services & Judicial System referrals). • Unlike private programs, these samples were predominantly male, lower SES, and disproportionately from ethic minority backgrounds. (Curtis, et. al., 2001; Epstein, 2004; Hair, 2005)

5. Findings in Public Residential Programs • Dozens of studies, published over 40 years. • Reviewers criticize the research for poor samples, retrospective designs, un-standardized measures, and unsophisticated statistical analyses (Curry, 1991; Curtis et al., 2001; Epstein, 2004, Hair, 2005).

6. Findings in Public Residential Programs • Though study outcomes vary widely, literature reviews conclude that 60%-80% of adolescents improve during residential treatment. (Curry, 1991; Curtis et al., 2001; Epstein, 2004; Hair, 2005; Wells, 1991). • However others conclude“little evidence of behavioral change during treatment”Hussey, 2002 andresidential treatment is best for higher functioning, less vulnerable youth Connor et al., 2002.

7. Findings in Public Residential Research • National Adolescent and Child Treatment Study- 3 years of treatment for teens with serious emotional disturbance to move from clinical to normal range Greenbaum et al. 1996 • Multiple high profile literature reviewsconclude “no evidence” of lasting benefits Curry, 1991; 2004; Epstein 2004; Little, Kohn, & Thompson 2005

8. Implications of the Public Residential Research “Given the limitations of current research, it is premature to endorse the effectiveness of residential treatment for adolescents.” Report of Surgeon General, 1999 Based on the research in public residential programs, policy authorities and advocacy groups conclude residential treatment (public and private) is not effective. Conclusion:refer youth to other services.

9. Findings in Private Residential Research • only 1 outcome research study (Menninger Clinic, Leichtman et al., 2001) • and 1 multi-center study presented at American Psychological Association Convention (9 Aspen Education Group Programs, Behrens & Satterfield, 2006)

10. Findings in Private Residential Research • Contrary to studies in public programs, studies in private residential programs reported favorable outcomes, with maintenance of gains post-discharge. • But, these 2 studies are recent and have not received attention from policy or advocacy groups.

11. Conclusion More research is needed in private residential treatment

12. 2. Describe the Study

13. First large-scale, systematic study of treatment outcomes in private residential programs. 3 years & Two phases • 1) Treatment Outcomes Released at APA, August 2006 • 2) Add Maintenance Outcomes • Released today

14. What companies were involved? • Aspen Education Group:Nine residential programs participated. Aspen funded the study. www.aspeneducation.com • Aspen Ranch, Academy at Swift River, Copper Canyon Academy, Mount Bachelor Academy, Stone Mountain School, Pine Ridge Academy, Sun Hawk, Turn About Ranch, Youth Care • Canyon Research & Consulting:Independent research company that conducted the study. www.canyonrc.com • Western Institutional Review Board:Independent board that approved research and audited the study. www.wirb.org • National Association of Therapeutic Schools and Programs:All Aspen participating programs are members of NATSAP. www.natsap.org

15. What were the questions? • 1. Do adolescents improve during the program? • 2. Are gains maintained in the year after discharge? Secondary Questions: • Do some problemsimprove more than others? • Do outcomes depend on baseline differences (e.g., age, gender)? • Do outcomes depend on treatment factors (e.g. number or types of problems, discharge status)?

16. What were the primary measures? • 1. Child Behavior CheckList (CBCL) • Completed by parents • 2. Youth Self-Report (YSR) • Completed by youth > 6,000 publications & in 72 languages. Psychological & social functioning

17. Demographic:age, gender, ethnicity, parental income, *Prior & Aftercare Treatment:hospitalization, outpatient, wilderness, residential, medication, *Academic Functioning:grade point average, matriculation, *Psychosocial Functioning (Achenbach Variables): Internalizing, Eternalizing, Total Problems, Anxious/Depressed Withdrawn/Depressed, Somatic Complaints, Thought Problems, Attention Problems, Rule-Breaking Behavior, Aggressive Behavior, What was measured? * Measured 10 times More…

18. Presenting Problems: number of problems, mood disorder, substance abuse, learning disorder, disruptive behavior, anxiety, legal problems, *Functioning in Family: - Communication, - Compliance with Rules, - Relationships, Treatment Factors: reaction to placement satisfaction, length of stay, report of overall improvement during treatment, effort in treatment, discharge status. What was measured? * Measured 10 times

19. 1027 Youth and 1027 Parents, both reporting on youth functioning With comparable participation rates from the 9 Aspen Education Group Residential Programs Who was in the study?

20. Who was in the study?Treatment History 94% had previous treatment.

21. Who was in the study?Number of Problems N = 1027 82% had more than one problem

22. Who was in the study?Problem Types Percent of Students N = 1027

23. Who was in the study?Academic & Legal Problems • 69% were > 1 semester behind in school • 60% had a G.P.A. <3.0 • 22% had a legal record

24. Conclusions The multi-center study: - used “gold standard” measures, - studied dozens of outcomes and factors, - and asked parents and youth, “Do youth improve and does it last?” The sample: • “failed” at prior levels of care • have multiple problems / co-morbidity • experience impairments in academic, social, behavior, and familial contexts.

25. 3. Present study highlights

26. Primary Questions: Psychosocial Problems • 1. Do adolescents improve during the program? • 2. Are gains maintained a year post-discharge? Psychological and Social Functioning:

27. Extreme Range Borderline Range Mean Raw Score Normal Range 51-52 T-Score post dc 55-56 T Score post dc Yes, youth and parents report problems improve during treatment and gains are maintained. CBCL/YSR Scores, Statistic: Hierarchical Linear Modeling, Parent N = 250-650, Youth N =139 -773

28. CBCL Total Score Problem Ranges N = 250-650

29. More Primary Questions • Youth Functioning Within the Family • 1. Do adolescents improve during the program? • 2. Are gains maintained a year post-discharge?

30. Excellent Good Satisfactory Excellent Good Poor Very Poor Every factor improved significantly and remained in the “satisfactory/good” range. N = 230-730

31. Conclusions • Youth psychological, social, and family problems improve during treatment, according to parents and youth. • One year after discharge youth functioning: • remains well within the normal range, • is better than functioning at admission, • is relatively stable.

32. Secondary Questions Do some problemschange more than others, according to Youth and Parents? Internalizing, Eternalizing, Total Problems, Anxious/Depressed, Withdrawn/Depressed, Somatic Complaints, Thought Problems, Attention Problems, Rule-Breaking Behavior, Aggressive Behavior,

33. YSR/CBCL Scales Eternalizing = Problems mainly with others Aggregates the aggression & rule-breaking scales Internalizing = Problems mainly within the self Aggregates depression, anxiety, somatic complaints scales

34. Externalizing Problems Extreme Borderline Mean Raw Score Normal 55-58 T Score Post DC 55-58 T Score Post DC Problems decrease significantly during treatment, then relatively stable. Test Statistic: Hierarchical Linear Modeling, Parent N = 250-650, Youth N =139 -773

35. Internalizing Problems Extreme Borderline Mean Raw Score Normal 56 T Score Post DC 50-52 T Score Post DC Problems decrease significantly during treatment, then relatively stable. Test Statistic: Hierarchical Linear Modeling, Parent N = 250-650, Youth N =139 -773

36. Anxious/Depressed, e.g.,fears, perfectionist, worthless, self-conscious Withdrawn/Depressed, e.g.,sad, prefers to be alone Somatic Complaints, e.g.,overtired, aches, nausea Thought Problems, e.g.,obsesses, strange ideas Attention Problems, Rule-Breaking Behavior, e.g.,steals, runs away, truant Aggressive Behavior, e.g.,stubborn, temper, mean YSR/CBCL Scales

37. Similar trajectory for problems Mean Raw Score All effect sizes moderate to large, with largest effect sizes for aggression, rule-breaking, & attention problems Test Statistic: Repeated Measures ANOVA, Parent Report, N = 106

38. Conclusions • Youth improve during treatment and maintained gains, on all outcome variables. • Although all outcomes improved, aggressive behavior, rule-breaking, and attention problems improved most.

39. More Secondary Questions • Does change depend on • treatment factors (discharge status, number of problems, type of problems)or • background factors (gender, age)? Numerous Test Statistics: Repeated Measures Analysis of Covariance, HLM, Regression, & Chi-Square

40. Conclusions • Change is the same, whether youth: • Are male or female, • 13, 14,...17 years old, • Have 1, 2… 6 problems, • Are treated for mood, substance, learning, anxiety, or behavior problems, • Exception: Discharge Status. Those discharged with maximum benefit had significantly better outcomes/maintenance of outcomes.

41. Mean Raw Score 55th percentile 83.5rd percentile (borderline?)

42. Summary Conclusions FIRST MULTI-CENTER STUDY IN PRIVATE RESIDENTIAL TREATMENT: 1027 Parents and Youth from 9 Aspen Education Group programs reported significant improvements with all problems during treatment. Improvement was maintained in the year post discharge.

43. Study Acknowledgments Aspen Education Group program staff For 3 years of recruiting participants and assisting with data collection. Aspen Education Group Leadership Teams For their support of the study and respect for the research process.

44. Influence policy and compete for funding (e.g., SAMHSA, NIMH, Capitol Hill) aggregate/archive data industry-wide outcome research Next Steps

45. What can residential & outdoor programs do? Conduct quality outcome research, then publish. Consider joining NATSAP Research Network. Research Coordinator: Michael C Young michael.young@unh.edu

46. What can consultants do?

47. Thank you! Ellen Behrens, Ph.D. Ellen@cayonrc.com Please email to request a copy of the research paper summarized in the presentation.

48. Selected References • Connor, D.F., et al. (2002). What does getting better mean? Child improvement and measure of outcome in residential treatment. American Journal of orthopsychiatry, 72, 110-117. • Curry, J.F., (2004). Future directions in residential treatment outcome research. Child Adol Psychiatric Clinics of North America, 13, 429-440. • Epstein.A. (2004). Inpatient and Residential Treatment Effects for Children and Adolescents. A Review and Critique. Child Adol Psychiatric Clinics of North America, 13, 412-428. • Greenbaum, P., Dedrick, R., Friedman, R., Kutash, K., et al., (1996). National adolescent and child treatment study (NACTS): Outcomes for children with serious emotional and behavior disturbance. Journal of Emotional and Behavioral Disorders, 4, 130-146. • Hair, H. J. (2005). Outcomes for children and adolescents after residential treatment: A review of research from 1993 to 2003. Journal of Child and Family Studies, 14, 551-575.

49. Selected References • Leichtman, M. Leichtman, M., Barber, C., & Neese, D. (2001). Effectiveness of intensive short-term residential treatment with severely disturbed adolescents. American Journal of Orthopsychiatry, 71, 227-235. • Little, M., Kohm, A., Thompson, R. (2005). The impact of residential placement on child development: research and policy implications. International Journal of Social Welfare, 14, 200-209. • Weis, R., Wilson, N., & Whitemarsh, S. (2005). Evaluation of a a voluntary, military-style residential treatment program for adolescents with academic and conduct problems. Journal of Clinical Child and Adolescent Psychology, 34, 692-705. • Whittaker, J.K. (2004). The re-invention of residential treatment: an agenda for research and practice. Child Adol Psychiatric Clinics of North America, 13, 267-278.