Evaluating Harlem Children’s Zone

1. Evaluating Harlem Children’s Zone

2. Evaluating Harlem Children’s Zone • Harlem Children’s Zone • Started by Geoffrey Canada in a 97 block area of Harlem in NYC. • Combines high energy “no excuses” type charter schools (Promise Academy) with a web of other community services. • 1,300 students attend “Promise Academy” schools, community programs served 8,058 kids and 5,291 adults. • Meant to support children from “birth to college.”

3. Evaluating Harlem Children’s Zone • Schools • Extended school day and school year. • Saturday school for students needing extra help. • Overall, roughly twice as many hours as other comparable students around city. • High quality teachers and use “value” added pay incentives. • High turnover, but lots of support and feedback. • Schools provide medical, dental, mental health services, meals, incentives for achievement. • Notably, they aim to provide a “culture of achievement” with no excuses. • Widely stated goal (outside and inside of schools) is to send every student to college.

4. Evaluating Harlem Children’s Zone • Community Programs • Baby college early childhood programs, • After-school programs (e.g. karate, dance, etc) • Family health programs • Over 20 programs available to members of the community.

5. Evaluating Harlem Children’s Zone • Questions of interest: • How much do kids benefit from this program relative to other alternatives in the area? • Is it the schooling or community services or both that provide benefit? • What strategies do Dobbie and Fryer use to try to “identify” the impact of HCZ?

6. Evaluating Harlem Children’s Zone • Lottery • Too many applicants for spaces in schools, so there is a lottery to determine who gets offered a slot. • Empirical design is essentially to compare lottery winners to lottery losers in terms of standardized test scores.

7. Evaluating Harlem Children’s Zone

8. Evaluating Harlem Children’s Zone

9. Evaluating Harlem Children’s Zone

10. Evaluating Harlem Children’s Zone • Cohort and Zone Boundaries (Instrumental Variable or IV) • Due to timing of when the programs were started, only some aged kids had access (i.e., had to be entering sixth grade or kindergarten the year the school was opened, those entering those grades the year before weren’t eligible) • While all kids in eligible cohorts can apply for lottery to attend schools, only those within the HCZ were actively recruited. Argument is that this made those kids in HCZ more likely to enroll relative to otherwise similar kids outside of HCZ.

11. Evaluating Harlem Children’s Zone • So basic design is a diff-in-diff. • Compare test score outcomes of eligible cohorts to non-eligible cohorts within boundary, difference out how these two groups compare just outside of boundary. • Estimated impact = (TSE,In - TSNE,In) - (TSE,Out - TSNE,Out) • Via regression? TS = α + β1EligibleCohort + β2InHCZ + β3EligibleCohort*InHCZ • Key Assumptions?

12. Evaluating Harlem Children’s Zone

13. Evaluating Harlem Children’s Zone

14. Evaluating Harlem Children’s Zone

15. Evaluating Harlem Children’s Zone • Is it the school or the “other” services? • How can they try to identify one from the other?

16. Evaluating Harlem Children’s Zone • IV results • If it was the “other” community services, then cohort/zone IV shouldn’t have shown anything. • That method only produces an “effect” if treated cohort outperforms untreated cohort within HCZ by more than treated cohort outperforms untreated cohort outside HCZ. • Since both treated and untreated cohorts in HCZ are eligible for other services, the estimated effects must be coming primarily from schools.

17. Evaluating Harlem Children’s Zone • Estimate the effect of having a “sibling” in one of the academies for those who aren’t enrolled in one of the academies. • Idea: Siblings of those enrolled in one of the academies should have similar access to “other” community programs, so if it is those programs that primarily impact kids’ outcomes, siblings of those enrolled should do better than siblings of those not enrolled.

18. Evaluating Harlem Children’s Zone • Why is Promise Academy so effective at educating the poorest minority students? • Teacher quality? • Extremely high turnover (48% left after first year) • Curriculum • Seem to focus on what has shown to be effective (210 instruction days per year, 8 hour instructional days, Saturday school, after-school program, avg class size of 20, no broken windows discipline policy, administers internal evaluations of teachers, requires school uniforms and a dress code, requires a parental contract, etc.) • Money • NYC Dept of Ed provides $12,443 per student, HCZ spends and estimated $19,272 per pupil.

19. Evaluating Harlem Children’s Zone • Summary • Growing body of evidence seems to suggest that early, often relatively large scale interventions can lead to both short-term and long-term impacts. • Questions still remain how to effectively do this at a large level. • Things like Perry Preschool and HCZ are very specialized, can they be scaled up? • While some evidence that large scale programs like Head Start have impacts, impacts are much more modest. • KIPP schools (Knowledge is Power Program) have tried to develop a replicable but flexible platform for effectively educating at-risk kids. • A relatively recent paper suggests that at least one of the KIPP schools in the Boston area is having substantial success.