Deborah Lowe Vandell SRHD Biennial Conference San Antonio TX March 2010

1. OUT-OF-SCHOOL TIMEAS A DEVELOPMENTAL CONTEXTLessons from Successful (and Unsuccessful Afterschool Programs Deborah Lowe Vandell SRHD Biennial Conference San Antonio TX March 2010

2. The Policy Context • Disconnect between the length of the school day and children’s academic, social, behavioral, and physical needs • High rates of maternal employment • 74% of the mothers of children 6-17 yrs • Concerns about • Low test scores • Unmet needs of English Language Learners • Negative effects of low supervision • Youth as victims & perpetrators of crime • Childhood obesity • Some evidence of beneficial effects of afterschool programs but not always • An element in Race to the Top and other school reform efforts

3. What is an after-school program? • Narrow definition – programs that are offered by schools or other organizations on a daily basis throughout the school year • Broad definition – includes extracurricular activities, sports, clubs, and sports offered by schools, libraries, and youth organizations

4. After-School Programs Sometimes Offer Academic enrichment & remediation • Homework help • Science, computer & math clubs • Book clubs Non-academic enrichment • Organized sports & recreational games • Music, drama • Arts and crafts • Scouts, 4-H, YMCA

5. Public and Private Investments in Programs • After-school programs • Serve 7 million+ children • 21% of 6 to 9 yr olds & 14% of 10-12 yr olds • CA After-School Education and Safety Program • 21st Century Community Learning Centers • National Network of State Afterschool Networks • Participation in at least 1 organized activity in a yr • 81% of 6- to 11-yr-olds • 83% of 12- to 17-yr-olds • 90% of non-poor vs 60% of poor children

6. Do afterschool programs have positive effects on child developmental outcomes?

7. Effect Size • a statistic that measures the magnitude of a program’s impact on a particular outcome • One common metric – Cohen (1988) • “small” - d = .2 • "medium” - d = .5 • “large” - d = .8 • Effect sizes also can be benchmarked against those reported in other studies. • Aspirin on heart disease d = .03 • School-based substance abuse prevention programs on drug & alcohol use d = .09 • Class size reductions on math achievement d = .23

8. A RECENT META-ANALYSIS Durlak, J. A., & Weissberg, R. P. (2007) 49 reports of 73 programs Meta-analyses of 1. All programs 2. Programs with Sequential & Active activities and Focused & Explicit content (SAFE) • .

9. Durlak & Weissberg Meta-Analysis

10. ADDITIONAL COMPONENTS OF PROGRAM QUALITY (NRC, Eccles & Gootman, 2002) • Positive relationships between students and staff • Positive relationships between students • Mix of academic and non-academic skill-building activities • High levels of student engagement • Mastery orientation • Appropriate levels of structure • Opportunities for autonomy and choice

11. Study of Promising Afterschool Programs(Vandell, Reisner, Pierce et al. 2007) • A longitudinal study of almost 3,000 low-income, ethnically diverse elementary and middle school student • Eight states: CA, CO, CT, MI, MT, NY, OR, RI • 14 communities: rural, small towns, mid-size cities, large cities • All located in high-poverty communities • First year devoted to identifying & describing promising programs • Studied student academic & social outcomes over two years

12. Programs received consistently high ratings on all of the program quality dimensions • Positive staff-student relationships • Positive relationships between students • Mix of skill-building activities • High levels of student engagement • Mastery orientation • Appropriate levels of structure • Opportunities for autonomy and choice

13. Study Participants Elementary School Sample 1,796 students in Grades 3 & 4 from 19 schools 89% free- or reduced-price school lunch 88% students of color Middle School Sample 1,118 students in Grades 6 & 7 from 16 schools 63% free- or reduced-price lunch 69% students of color

14. Other Afterschool Experiences • Organized activities such as team sports, school-based activities and lessons • Home alone and home with a sibling • “Hanging out” with peers without adult supervision • Each reported on 4-point scales • 1 = not at all/ once or twice • 2 = about once a week • 3 = 2 – 3 times a week • 4 = 4 or more days a week

15. FOUR CLUSTERS WERE IDENTIFIED • Programs only (PO) • Programs plus other activities (PP) • Low supervision with activities • Other

16. Participation in Promising Programs and Other Activities Over 2 Years Elementary Sample • 54% regular program participants (about 90 day/yr) 2/3 Program Only; 1/3 Program Plus • 15% low supervision Middle School Sample • 49% regular program participants (about 55 days/yr) 2/3 Program Only; 1/3 Program Plus • 16% low supervision

17. Child OutcomesMeasured at Baseline, at the end of Year 1, and at the end of Year 2

18. Analytic Plan • HLM analyses • Level 1 - child • Level 2 – school/program • Difference scores • 10 multiple imputations

19. Covariates • Gender • Race-ethnicity • Grade level • Household income (per $1000) • Maternal (or guardian) education (in yrs) • Mother (or guardian) works full time • Household structure

20. Elementary Sample: Child Report Program Only vs. Low SupervisionEffect Sizes

21. Elementary Sample: Child Report Program Plus vs. Low SupervisionEffect Sizes

22. Elementary Sample: Teacher Report Program Only vs. Low SupervisionEffect Sizes

23. Elementary Sample: Teacher Report Program Plus vs. Low SupervisionEffect Sizes

24. Elementary Sample Math Achievement Test Scores Effect Sizes

25. Middle School Sample: Youth ReportProgram Only vs. Low SupervisionSignificant Effect Sizes

26. Middle School Sample: Youth Report Program Plus vs. Low Supervision Effect Sizes

27. Middle School Sample Math Achievement Test ScoresEffect Sizes

28. Conclusions: Elementary School Sample Standardized Tests Gains in math for PO after one year; gains for both PO and PP groups after 2 years Child Reports Gains in work habits & reductions in misconduct for both PO & PP groups; larger effects after 2 years Teacher Reports Gains in work habits, task persistence, social skills, prosocial behaviors & reductions in aggression for PO after first year; improvements for both PO and PP after 2 years

29. Conclusions: Middle School Sample Standardized Tests Gains in math achievement for PO & PP groups after 2 years (not 1 year) Youth Self-Reports Both PO and PP youth reports reported gains in work habits after 2 years (not 1 year) and larger reductions in misconduct and substance use after two years Teacher Reports Little evidence of differences between Program and low supervised youth

30. Why are after-school programs beneficial?

31. Study 2: Experience Sampling - Out-of-School TimeVandell, Shernoff, et al. (2007, 2009) • 191 8th grade students, primarily low-income students of color • 8 middle schools in 3 communities • Students wore watches that beeped them at random times during non-school hours – afterschool, evenings, weekends • 35 signals during a one week period in the fall and 35 signals during a one week period in the spring

32. Students Filled Out Logbooks • After each signal, students recorded • Who they were with • Where they were • What they were doing • How they were feeling • And their levels of effort, concentration, motivation

33. Very Little Missing Data!! • On average, students responded to 33 of the 35 signals during the week. • 12,143 after-school, evening, and weekend experiences were reported. • 5, 136 of the experiences occurred after school.

34. Students’ Activities at Programs and Elsewhere

35. Differences in Supervision (% of time)

36. Differences in Motivation, Effort, and Feelings (4-pt ratings)

37. Choice, Concentration, and Effort

38. Study 3: Are Particular Program Features Associated with Child Developmental Outcomes? (Pierce, Bolt, Vandell, 2010) 3-year longitudinal study of children who attended 30 programs of varying quality n = 150 in 1st grade n = 120 in 2nd grade n = 91 in 3rd grade 49% male 89% white 60% parents have college degrees

39. Observations Of Program Quality 4 program observations in 2nd grade and 3 observations in 3rd grade staff-child relationships availability of age-appropriate activities programming flexibility Analyses control for family background and child prior functioning

40. Child Developmental Outcomes • Teacher ratings • Mock Report Card • Reading • Math • Work habits (6 items, 5 pt ratings) • Social skills with peers (7 items, 5 pt ratings • Collected from classroom teachers in G1, G2, and G3.

41. Analytic Strategy HLM analyses in Grade 2 & Grade 3 Level 1 – child Level 2 – program Covariates: gender, minority (yes/no), maternal education, family income, 2-parent household (yes/no), firm/responsive parenting, prior child functioning ,

42. Effects Associated With Program Quality Features in G2 and G3

43. Five Take-Home Messages • Regular participation in high quality afterschool programs is linked to positive social and academic outcomes • Gains are more evident after two years than after one year, suggesting that duration is important • Youth reports of experiences reveal differences in activities, social partners, motivation, effort, and affect while attending afterschool programs • There is evidence that particular aspects of children’s experiences are related to child developmental outcomes • Out of school time IS an important developmental context

44. http://childcare.gse.uci.edu/ for more information