Research Brief: Bridging Research and Practice in After-School Programs

1. Research Brief:Bridging Research and Practice in After-School Programs Theresa Ferrari & Graham Cochran State 4-H In-Service February 7, 2005

2. Evidence Suggests . . . • Youth benefit from after-school programs, provided they involve: • Consistent participation • Quality, well-run programs

3. Evidence-Based Practices • “Gold standard” (U.S. Dept. of Ed, 2003) • Most evaluations assess overall program impact rather than linking specific program activities with outcomes. (Little & Harris, 2003) • Programs offer “intangibles” that are hard to quantify . (Miller, 2003)

4. Be There or Be Square: Participation Matters • If you’re not there, you can’t benefit! • Significant positive relationship between time spent in program and desired outcomes. • Limited or sporadic attendance not likely to produce desired effects.

5. Measuring Attendance = “Dosage” • Frequency/Intensity – how often • Duration – how long • Breadth – variety of activities (Harvard Family Research Project, 2004)

6. Program Activities: What You Do Matters • Variety of activities • Flexibility of programming • Positive emotional climate (Rosenthal & Vandell, 1996) • Collecting activity implementation data is a critical first step in evaluation.

7. Program Activities: What You Do Matters • Organized activities: • Involve voluntary participation • Contain structure • When engaging in these activities youth experience: • High intrinsic motivation • Positive mood • Cognitive engagement • When these conditions exist: • Develop initiative Dworkin, Hansen, & Larson, 2003 Hansen, Larson, & Dworkin, 2003 Larson, 2000 Larson, Hansen, & Walker, 2005

8. Staff: Who You’re With Matters • Caring youth-staff relationships may be the most critical element to program success. (Rhodes, 2004; Shortt, 2002) • Ability of staff member leading the activity may be more important than the activity itself. (Grossman et al., 2002)

9. Staff: Who You’re With Matters • Balancing act: • Following youths’ lead • Asking questions • Providing intermediate structures • Monitoring to keep youth on track (Larson, Hansen, & Walker, 2005)

10. Staff: Who You’re With Matters • Evidence from mentoring studies: • Engaging in social activities • Engaging in academic activities • Meeting regularly • Using youths’ interest to guide interaction • Seeking input and making decisions jointly • Taking a less judgmental approach Jekielek, Moore, Hair, & Scarupa, 2002 Rhodes, 2004

11. Staff: Who You’re With Matters • Processes by which staff mediate effects of after-school programs: • By enhancing youths’ social skills and emotional well-being • By improving youths’ cognitive skills through instruction and conversation • By serving as role models and advocates (Rhodes, 2004)

12. Academic Achievement • Indirect contribution • By increasing student engagement in learning • Greater school attendance • Improved work habits and behavior • Positive attitudes toward school Miller, 2003

13. If you expect certain content to be delivered . . . • It must be reflected in the program design in intentional ways. • Success ultimately rests with the staff’s ability to deliver the content effectively.

14. Vandell’s (2003) Theory of Change - Study of Promising After-School Programs Family Background and Child Prior Functioning Improved Work Habits, School Attendance; Social Skills; Reduced Misconduct Improved Grades, Achievement, Future Orientation; Reduced Risky Behaviors Program Structural & Institutional Features Program Processes and Content Program Dosage

15. Research Snapshot Adventure Central

16. 4-H Youth Development in the 21st Century Key Elements of Youth Development Programming • Positive relationships with caring adults • Physically & emotionally safe environment • Welcoming environment • Opportunity for mastery • Engagement in learning • Opportunity to value & practice service • Opportunity for self-determination • Active participant in the future (from National 4-H Impact Study, 2001)

17. Experienced positive adult relationships. Attendance contributed to relationships. Support provided is both emotional and instrumental. Relationships contributed to continued participation. Relationships were more positive than with teachers and neighborhood adults. Adult Relationships

18. Adult Relationships • Observation using checklist found several behaviors to occur frequently: • Talking in a positive tone • Giving directions • Listening • Using a child’s name

19. It keeps us away from the streets. So you won’t get caught up in anything. Safe Environment • Participants felt safe at Adventure Central. • 2/3 felt safer than in their neighborhood. • If not at Adventure Central, might be watching TV, hanging out, or getting into trouble.

20. Engagement in Learning • Youth experienced opportunities to try new things. • Particularly evident in activities in which they were treated with importance (“mattering”): • Teen Assistant • Youth Board • These activities contributed to continued participation.

21. Making decisions on the Youth Board . . . makes me feel like an important businesswoman

22. Working as a teen assistant at Adventure Central . . . is the beginning of having my first job in my life. Adventure Central

23. Mastery and Competence • Activities helped with learning. • Received homework assistance from caring adults.

24. Self-Determination • Are capable of taking a leadership role. • Can deal with problems that might come up in the future. • Have set goals to achieve.

25. Sense of Future • See connection with academic support and success later in life.

26. Since I have been coming here my grades have been coming up . . . I should . . . keep coming so I can succeed and try to get a scholarship to college.

27. Adventure Central is. . . . . . a nice environment to be in. I feel comfortable here. I like the people.

28. References • Chaput, S. S., Little, P. M. D., & Weiss, H. (2004). Understanding and measuring attendance in out-of-school time programs. Retrieved from the Harvard Family Research Project website, http://www.gse.harvard.edu/hfrp/content/projects/afterschool/resources/issuebrief7.pdf • Dworkin, J. B., Larson, R., & Hansen, D. (2003). Adolescents’ accounts of growth experiences in youth activities. Journal of Youth and Adolescence, 32, 17-26. • Grossman, J. B., Price, M. L., Fellerath, V., Jucovy, L. Z., Kotloff, L. J., Raley, R., & Walker, K. E. (2002). Multiple choices after school: Findings from the extended-service schools initiative. Philadelphia: Public/Private Ventures. Retrieved from • Jekielek, S. M., Moore, K. A., Hair, E. C., & Scarupa, H. J. (2002). Mentoring: A promising strategy for youth development. Washington, DC: Child Trends. • Larson, R., Hansen, D., & Walker, K. (2005). Everybody’s gotta give: Development of initiative and teamwork within a youth program. In J. Mahoney, R. Larson, & J. Eccles (Eds.), Organized activities as contexts of development: Extracurricular activities, after-school and community programs (pp. 159-183). Hillsdale, NJ: Erlbaum. • Miller, B. M. (2003). Critical hours: Afterschool programs and educational success. Quincy, MA: Nellie Mae Educational Foundation. Retrieved from http://www.nmefdn.org/uimages/documents/Critical_Hours(4).pdf • Rhodes, J. (2004). The critical ingredient: Caring youth-staff relationships in after-school settings. In G. G. Noam (Ed.), After-school worlds: Creating a new social space for development and learning (New Directions for Youth Development, no. 101, pp. 145-161). Hoboken, NJ: Wiley. • Rosenthal, R., & Vandell, D. L. (1996). Quality of school-age child care programs: Regulatable features, observed experiences, child perspectives, and parent perspectives. Child Development, 67, 2434-2445. • Shortt, J. (2002). Out-of-school time programs: At a critical junction. In G.G. Noam & B.M. Miller (Eds.), Youth development and after-school time: A tale of many cities. New Directions for Youth Development, No. 94. • U.S. Department of Education. (2003). Identifying and implementing educational practices supported by rigorous evidence: A user friendly guide. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from: http://www.ed.gov/rschstat/research/pubs/rigorousevid/rigorousevid.pdf • Vandell, D. L. (2003). Promising after-school programs. University of Wisconsin. Retrieved from www.wcer.wisc.edu/childcare/statement3.html