Findings from the Head Start Impact Study Third Grade Follow-Up

1. Findings from the Head Start Impact Study Third Grade Follow-Up Jennifer Brooks Office of Planning, Research, and Evaluation

2. Study Team • Implemented by: • Westat • Chesapeake Research Associates • Ronna Cook Associates • Abt Associates • American Institutes for Research • Urban Institute • Decision Information Resources • Thanks to the Head Start community for its engagement and participation!

3. Background of Study • Mandated by Congress in the 1998 reauthorization of Head Start. • Congress asked that the study be • nationally representative, and • a comparison of HS children with a group of comparable non-participants.

4. Background of Study • 1999 Report from the Advisory Committee on Head Start Research and Evaluation : • Providing recommendations for the design of the study; • Noting the need to understand not only average impacts but “what works for whom”; • Highlighting the importance of understanding any one study in the context of broader literature

5. Research Questions What difference does Head Start make to key outcomes of development and learning (and in particular, the multiple domains of school readiness) for low-income children? What difference does Head Start make to parental practices that contribute to children’s school readiness?

6. Research Questions (cont.) Under what circumstances does Head Start achieve the greatest impact? What works for which children?

7. Study Sample • Nationally representative sample of a large number of grantees, but does not include (per Congressional mandate) AI/AN, migrant and “special population” programs • 84 randomly selected grantees across 23 states • 383 randomly selected centers • Total of 4,667 randomly assigned children in two cohorts: 2,559 3-year-olds and 2,108 4-year-olds

8. Study Sample (cont.) Approximately 15% of children served by Head Start eligible programs were not represented in the study. This was due to the decision to only include children from Head Start programs where there were fewer slots available than children applying.

9. Study Timeframe Baseline data collected in fall 2002 Annual spring follow-up data collection through 2006, the end of 1st grade for youngest children, and then again in spring of 2007-2008, the end of 3rd grade

10. Randomization • Newly entering 3-and 4-year-old Head Start applicants in 2002 were randomly assigned to either: • Treatment group to enroll in Head Start, or • Control group that did not; parents found other available services for their child or the child was cared for at home. • Control group had access to Head Start in second year. • For both age cohorts, this study assesses the impact of one program year of Head Start. • 3-year-old cohort is not a test of a 2-year Head Start program; rather, the added benefit of an early year.

11. Outcome Domains Cognitive/Academic Social and Emotional Health Parenting Practices

12. Data Sources Children Parents/primary caregivers Teachers Care providers Center directors Care setting observations—pre-school years Secondary information on schools Census data

13. This Study is Unique • One of very few studies to: • Examine an ongoing established program. • In a nationally representative sample. • Using a Randomized Control Trial (RCT) design.

14. This Study is Unique (cont.) • Not directly comparable to other studies of early childhood care and education • Randomized control trial • Representative sample of children and programs • Examination of a comprehensive set of outcomes over time • Control group children did not all stay at home • Impacts represent the effects of one program year

15. Key Impact Findings

16. SAMPLE EXERCISE – WHAT WE MEAN BY “IMPACT”

17. Experiences of the Program and Control Groups • Randomization affected what types of early childhood settings children entered. • For both 3- and 4-year-olds: • Control Group (Non-Head Start) children were about five times more likely to be exclusively in parent care • Head Start children were about twice as likely to use a center-based program (including HS) • Still, about 60% of children in the Control group were in some form of non-parental care at least five hours a week

18. Early Care Experiences (cont.) • Average length of participation in Head Start in that first year was about 8 months • For those in care, average number of hours per week: • For 3-year-olds: 28 hours for Head Start group, 33 hours for control group • 4-year-olds: 25 hours for Head Start group, 29 hours for control group

19. Early Care Experiences (cont.) • Providing access to Head Start had a positive impact on the quality of children’s early care in that first year. • Differences in: • teacher qualifications, • classroom literacy and math activities, • teacher-child ratios • teacher-child interactions (Arnett) and global quality (ECERS-R and FDCRS) • Some of this difference is driven by the fact that 40% of the control group was in parental care.

20. Early Care Experiences (cont.) After that first year: • By the second year, the care experiences did not vary much for the 3-year old cohort: • 50% of the control group and 63% of program group in Head Start, Quality of care was similar • Minimal differences in K, 1st, or 3rd grade experiences for program and control group • Both groups were in schools with large populations of children receiving free- and reduced-price lunch and minority children. Statewide test scores were about average for the state in which program was located.

21. Cognitive Impacts • Clear evidence that Head Start has an impact on language and literacy development while in Head Start • Impacts found for both age cohorts (vocabulary, letters and letter sounds.) • Impact were modest in size. • Also impacts on early writing for 4s and math skills for 3s.

22. Cognitive Impacts (cont.) • By the end of 1st and again at 3rd grade, the Head Start children and the control group children were at the same level on many of the cognitive measures. • Few statistically significant impacts in 1st or 3rd grade.

23. Sample Longitudinal Impacts: PPVT Scores for the 3-year-old Cohort

24. Social-emotional Domain • No clear benefits for 4-year-old cohort in earlier reports • In 3rd grade, few impacts, and those differed in direction depending upon who was describing the child: • Favorable impacts on parents’ reports of child’s behavior • Unfavorable impacts on teachers’ reports of teacher-child relationship and mental health • One unfavorable impact on children’s reports of their relationships with peers

25. Social-emotional Domain (cont.) • 3-year-old cohort: favorable impacts on parent reports in all prior years, only a single impact at 3rd grade:

26. Health Domain • In Head Start years, strong evidence of increases in dental care. • Also some impacts in age 4 year (for 3 year old group) and K on other measures (health insurance, overall health. ) • In 1st grade, one impact on 4-year old cohort (increased health insurance) • By 3rd grade, no impacts remained.

27. Parenting Domain • Minimal impacts on parenting for 4-year-old cohort • 3-year-old cohort: favorable impacts in all prior years and one impact in 3rd grade:

28. Exploratory Analysis of Child and Family Subgroups In earlier years found: • Favorable patterns for several groups in each age cohort • A few subgroups showed patterns of mixed or unfavorable impacts, most notably children of parents with moderate depressive symptoms in the 3-year-old cohort

29. Exploratory Subgroup Findings in 3rd grade Fewer subgroup patterns overall A few subgroups with remaining favorable impacts. A couple of subgroups showed remaining unfavorable impacts.

30. What to make of the findings? The Advisory Committee on Head Start Research and Evaluation (2012) • Committee’s 2010-2012 charter: • Interpret the findings of the Head Start and Early Head Start evaluations in the context of broader literature. • Help identify improvements, based on research evidence, for the early childhood field more broadly.

31. Statements of the Advisory Committee on Head Start Research and Evaluation (cont.) • Statements about the Head Start Impact Study findings: • Clear impacts during time in Head Start of comparable size to impacts for other large-scale programs • Few impacts remaining at 1st grade • Possibly some sustained impacts for some subgroups.

32. Statements of the Advisory Committee on Head Start Research and Evaluation (cont.) • Could there be long-term effects? • Some studies have shown impacts in adulthood after initial effects dissipated. • Some non-experimental studies have shown long-term benefits of Head Start. • Not clear whether this study might show the same over time.

33. Statements of the Advisory Committee on Head Start Research and Evaluation (cont.) • “Appropriate and Inappropriate Interpretations” • Cautions about comparing HSIS findings to earlier studies (e.g., Perry Preschool, Abecedarian), state pre-Kindergarten studies, or EHS evaluation, given: • Differences in programs • Differences in populations served • Differences in methodology • Differences in comparison conditions • The study does highlight the need to both improve intensity of Head Start impacts while children attend and to improve schools abilities to maintain and build on these impacts.

34. Next Steps • Westat contract and a grant with NYU will address quality of Head Start programs and schools as they relate to impacts • Data archived and being used by other researchers • Held a meeting of these teams in December to discuss challenges and approaches to analyzing quality questions.

35. Questions? Jennifer.Brooks@acf.hhs.gov The Head Start Impact Study reports are available at: http://www.acf.hhs.gov/programs/opre/research/project/head-start-impact-study-and-follow-up-2000-2012