Evaluation Design

1. Evaluation of the New Century High Schools InitiativeElizabeth ReisnerAmerican Youth Policy ForumOctober 27, 2006

2. Evaluation Design Central question: Did NCHS schools promote students’ academic success? Longitudinal collection and analysis of data spanning the initiative’s first 4 years Data sources: surveys, observations, interviews, and agency data on students Student growth assessed against a matched comparison group of NYC students

3. Basis of Findings to Date June 2006 report and this presentation based on data from first 3 years of the initiative, spanning school years 2002-03 through 2004-05 Evaluation findings available on NCHS 9th, 10th, and 11th graders Timing driven by availability of data from New York City Department of Education, which is just now able to share data files on 2005-06 student characteristics and performance

4. Implementation of Quality Features School-level implementation assessed on 10 NCHS quality elements, which include: ▪ rigorous instruction ▪ personalized relationships ▪ focus on learning ▪ instructional leadership ▪ community partnerships Teachers in 2nd and 3rd groups of schools provided higher initial quality ratings than did teachers in 1st group of schools

5. Implementation (continued) Student ratings of initial quality were high and consistent over time across successive groups of new schools For individual schools, high and low quality ratings became less extreme over time Quality features that relied on small school size leveled off as schools grew in enrollment

6. Conditions Observed by Evaluators NCHS schools were small, safe, and focused on instruction and youth development Use of themes and quality of partnerships varied across schools but did not determine overall school effectiveness Small size and availability of academic supports contributed consistently to student learning

7. Measurement of Student Outcomes Relative to NYC, NCHS students more likely to be female, African American or Hispanic, and poor At entry to high school, NCHS students less likely to be proficient in reading and math However, NCHS students more likely than matched comparison students and citywide students to: ▪ attend school ▪ earn credits for graduation ▪ be promoted to next grade

8. Student Outcomes (continued) NCHS students appeared more likely to persist in school but difference not significant Results mixed in analysis of Regents exams passed Suspension rates of NCHS students higher than comparison group Overall, NCHS students more likely than comparison group to be on track for graduation

9. Challenges Evident from Evaluation Suspension rates indicated growing pains as school enrollments grew to 400-450 students each NCHS staff saw their schools as crowded and reported resource competition with other schools NCHS staff saw NYC student admissions system as assigning students who didn’t want to be in NCHS schools

10. Central Findings NCHS students outperformed carefully matched comparison students, especially in terms of likelihood of graduation Positive learning outcomes tracked school implementation of proven instructional practices Convergence of implementation findings and student outcomes suggests that NCHS schools contributed importantly to achievement

11. Reflections on Findings The success of these new schools is fragile and requires continued work on: ▪ building their instructional core ▪ promoting youth development ▪ providing learning supports Going forward, schools need ▪ effective principal leadership ▪ time to build instructional core and extend student learning opportunities

12. Next Steps in Evaluation Currently preparing final report on first 4 years of the initiative, which will take initial class of 9th graders through 12th grade Final report will revisit preceding topics to assess the contribution of NCHS schools to student achievement over time Final report expected late spring 2007