Day Treatment Program Evaluation 2010-11 Academic Year Project Summary

1. Day Treatment Program Evaluation2010-11 Academic YearProject Summary

2. Day Program Overview • The program provides youth (14 -19) an alternative learning opportunity in a smaller and more focused academic environment, with therapeutic and life skills support. • The DTP has a capacity for serving 24 students. • 3 classrooms, each staffed with a TDSB teacher and a Child and Youth Worker (CYW) • Offers high school credit courses in a range of subjects • Provides support and treatment to youth enrolled in the program

3. Evaluation Overview • Centre of Excellence grant to carry out evaluation plan (plan developed with funding from MCYS) • Purpose: • To assess the degree to which the program is producing positive outcomes for students enrolled in the program • To determine what program areas/activities/processes, if any, might be improved • To utilize the findings of the evaluation to implement changes that will improve program effectiveness and inform decisions about future program planning • Mixed methods evaluation: quantitative and qualitative • Program outcome data (e.g. attendance, CAFAS scores, credits etc.) • Stakeholders interviews (teachers, staff, youth, families) • Steering committee oversaw the progress of evaluation

4. Program Evaluation Questions - Outcomes • Is the DYS day treatment program admitting the right type of student? • Are DYS day treatment program students committed to the program? • Are DYS day treatment program students doing better in school? • Are DYS day treatment program students earning educational credits? • Are DYS day treatment program students' behavioural/emotional challenges improving? • Have DYS day treatment program students found the program helpful?

5. Program Evaluation Questions (continued) Program Processes/Components • In what ways do the key components of the DTP admission and intake process help to facilitate (or hinder) the enrolment of youth most likely to benefit from the program? • In what ways do the key program components help to facilitate (or hinder) the success of youth enrolled in the program? • In what ways do the key elements of the DTP discharge planning process help to facilitate (or hinder) a youth's preparedness for their post-DTP experience?  

6. Are Students meeting program admission criteria? Target: 100% of DTP students will meet admission criteria

7. Are Students meeting program admission criteria? Insights from Stakeholder interviews: • the program’s admission and intake process might be improved by having teachers more directly involved in this area of the program • Concern expressed for the program policy of allowing both predominately internalizing youth to coexist in the same program with predominately behavioral youth “we have to decide if we are going to be a program that’s going to do one thing well or we are going to try to be all things to all people…there is a distinction between students who are physically and verbally aggressive and those that have severe anxiety or depression.”

8. Are students committed to and engaged in the program? Target: 75% of DTP students attend at least 80% of scheduled school days • This suggests that the program was not successful in ensuring participant commitment in at least 20% of the youth during the 2010-11 school year.

9. Are students committed to and engaged in the program? Insights from Stakeholder Interviews: • Highlighted deficiencies with our measurement approach to participant commitment and engagement (e.g. multidimensional nature of student engagement---cognitive, emotional & behavioral types of engagement) • Highlighted other potential indicators of participants’ commitment to the program: • completion of school related work • attainment of high school credits • participation in extracurricular activities • establishment of positive relationships with program peers and staff

10. Are Students functioning better in a school based environment? Insights from Stakeholders • Program activities that may contribute to improved client outcomes: • Emphasis on goal setting and working towards goals • Small class sizes • Availability of a child and youth workers in the classroom setting • Individualized programming and flexibility • Strength based approach to both the academic and treatment component of the program Target: 80% of DTP students will show at least a 25% improvement in the CAFAS school subscale

11. Are students earning educational credits? Target: 60% of DTP students will achieve at least 50% of attempted credits. • Overall, the day treatment program shows a 58% percent credit achievement rate for the entire 2010-11 academic year.

12. Have Students improved their behavioral/emotional functioning? Target: 80% of DTP students will show at least a 25% improvement in their CAFAS ‘moods and emotions’ subscale or their ‘behavior towards others’ subscale • A third of the group showing improved emotional or behavioral functioning was rated as having no significant dysfunction at program exit.

13. Have Students improved their behavioral/emotional functioning? • Insights from Stakeholders • the most important goal for program participants is for them to work towards treatment goals and learn strategies that will help them cope with their issues and challenges • DTP teachers have limited involvement in creating or reviewing treatment goals. They would like greater input into the process.

14. Client Satisfaction: Do students have a positive view of their program experience? • Overall Response rate = 72% of all program participants • ~78% of respondents report overall program satisfaction levels that are either good or excellent. • ~77% of respondents would recommend DYS’programs to others. • Over 85% of respondents report that their primary program workers are a good match for them and that they’ve had a positive relationship with their primary workers. • Over 75% of respondents indicate that they felt empowered and respected in decision making related to the service that they received. • ~48% of respondents agreed or strongly agreed that their personal goals were successfully achieved during their stay in the day treatment program (not inconsistent with our outcome data…moderate levels of success)

15. Further suggestions for improving the DTP: Stakeholder Insights • Life skills worker to provide skill-building activities for program youth. • Youth psychiatrist to help youth with treatment issues. • Access to a guidance counselor with more specialized knowledge of the available school programs in the GTA. • Nutrition program or incentives (e.g. breakfast program). • Code of conduct with consistent rules, expectations and consequences for program participants. • More opportunities for youth to participate in physical activities or gym class, clubs, sports teams and extracurricular activities. • Greater access to George Harvey and its resources. • Follow up with youth who leave the program.