Marcia Hughes May 31, 2006

1. Creating “Turning Points” in the Lives of Youth Residing in High Risk Communities:Participation and Response to School-Based Mentoring and Impact on Academic Outcomes Marcia Hughes May 31, 2006

2. The Intervention Project: GEAR UP • (Gaining Early Awareness and Readiness for Undergraduate Programs) • Federally-funded: US Department of Education • Collaborative effort: University of Connecticut & Hartford Public Schools Purpose of the Study Examine a youth program that was designed to help participants from high risk communities do well in school, and plan and prepare for post secondary education. The study asks for whom was the program effective, in what ways, and under what circumstances.

3. Acknowledgements Principal Investigators:John C. Bennett, Sara Harkness, Kay A. Norlander-Case, Charles W. Case Advising Committee:Sara Harkness, Preston Britner, Orv Karan, and Charlie Super Undergraduate, Graduate Students, Participating Youth: Many thanks to all the undergraduate and graduate students who have served on the project, & to all the participating youth.

4. Background: Evolution of Youth development field (Small & Memmo, 2004) • - Prevention: target a particular youth population deemed at risk of problem • - Fix-it approach: Runs counter to what is known about • human motivation • Resilience: Risk and protective mechanisms exist together • basis of mentoring programs • assist with coping, problem solving • Youth Development Field: • Positive, asset building orientation • Builds on strengths rather than categorizing according to deficits • Get involved, develop competencies, experience success as intervention

5. Background:“Process is Product” (McLaughlin, 2000) • Developmental outcomes of interest: • Learning to be productive; learning to connect; learning to navigate (Gambone, Klem, Connell, 2002) • Constructive use of time, commitment to learn, positive values, & social competence (Small & Memmo, 2004) • Confidence, character, connection, competence, contribution (Pittman, Irby, Tolman, Yohalem, & Ferber, 2003) Getting “Buy-in” and Sense of Belonging: - “Just opening the doors and getting youth involved marks a major accomplishment.” (McLaughlin, 2000) - Belonging is of equal significance and utility in youth programs as social, behavioral, and academic outcomes (Barkdull, 2004)

6. Background: Broad Based Comprehensive Strategy • Contemporary models of youth development and problem prevention: • Incorporates all 3 paradigms, framed in a developmental-ecological model: • Prevention • Resilience • Positive youth development • (Catalano, Boglund, Ryan, Lonczak, & Hawkins, 1998; Gambone et al., 2002; Kerpelman, 2004; Small & Memmo,2004)

7. “The State of the Field” (Catalano et al, 1998) Evaluation of 25 National programs: • Promote bonding • Foster resilience • Promote social competence • Promote emotional competence • Promote cognitive competence • Promote behavioral competence • Promote moral competence • Foster self-determination • Foster spirituality • Foster self-efficacy • Foster clear and positive identity • Foster belief in the future • Provide recognition for positive behavior • Provide opportunities for prosocial involvement • Foster prosocial norms

8. Background:Research and Program Evaluation • In order to advance the Youth Development Field, calling for: • - More meaningful measures (Gambone et al, 2002; McLaughlin, 2000) • - Assessing the “whole” child (Catalano et al, 1998; Riggs & Greenber, 2004) • Linking the chain of effects (Catalano et al, 1998; Gambone et al, 2002) • Getting inside the “Black Box”: Document/describe/understand day-to-day realities and challenges of youth programs (Kalafat & Illback, 1998; Koss-Chioino & Vargas, 1999; Small, 2005)

9. PROGRAM PARTICIPATION & RESPONSIVENESS Post-Secondary Outcomes Individual level • University or • Community College • - Technical School • Working • Retained • - Drop out • Social Support - Emotional bond/support • - Academic Support • - Cognitive Skills • - Self-Awareness • - Belonging • - Sense of Competence • Sense of Control/ • Coping Strategies • - Improved relationships Research Model PROBLEM BEHAVIORS RISK FACTORS & PROMOTING PROCESSES Family level School & Community level Last 2 years of project participation

10. Research Questions 1) By what means, if at all, have the components of the project fostered processes that facilitated positive change for the youth? • 2) How are youths’ level of participation and response to the intervention modified by individual adolescent needs and individual situations at home and at school? Specifically: • What are the overlapping problem behaviors and support? • What are the overlapping risk factors and promoting processes? • In what ways did the project mitigate risk exposure? • In what ways did the project promote positive processes? 3) How are differences in youth’s participation and responsiveness related to their post-secondary status (i.e., long-term goal of the project)?

11. GEAR UP: Current Status • Longitudinal project, began in 1999: • - Two cohorts of students in the 6th & 7th grades (N=130) • - Have now completed the 11th and 12th grades (N=110) • Indicators of Success: • - Continued high rate of participation (Approx. 82%) • - Low school dropout rate (Approx. 10%) • - High rate of students accepted for post secondary education (Approx. 50%)

12. Intervention Model:Program Description(Harkness, Hughes, Muller, & Super, 2004) • Site-based program: Ongoing presence in the schools • Continuity across contexts of home and school • Transitional support: early adolescence - early adulthood • One-to-one mentoring assignments • Strong infrastructure: close monitoring and accountability

13. Intervention Model: Practicing Principles(Harkness, Hughes, Muller, & Super, 2004) • Intervene with youth in the context of their environments • Not one-size-fits-all intervention/Environmental Fit • Ethnographic process of inquiry, collaborator, not “expert” • Focus: strengths, resources, & broad indices of development • Process first, outcomes second

14. Methodology:Study Sample Wide range of participants: From the student who is excelling academically to the student who appears to be near drop-out. • Total # of participants/subjects in this study: • 45 from older cohort: 23 Male/20 Female • 34 from younger cohort: 22 Male/12 Female • Total: 79 Subjects: 45 Male/32 Female • Ethnic Distribution: • 65 are Hispanic (all but 5 are Puerto Rican) • 12 are Black (all but 3 are African-American) • 2 are White

15. Methodology: Data Collection • Data Collection (secondary data) • Documentation of process data for each participant • Collected by mentors/practitioners • In the form of a “service note” and “summary reports” • Instruction on Data Collection • Service notes: Every 2 weeks with feedback • Summary reports: End of each semester • Analyzed data for the last 2 years of the project • Older Cohort: Fall semester 2003 - Summer 2005 • Younger Cohort: Fall semester 2004 - Present (2006)

17. “Checklist” for organizing observations and impressions

18. Summary Reports: Different Children, Different Needs

19. Methods of Analysis I Analysis of Qualitative Data: Coding system; analyzing service notes 1) Responsiveness 2) Problem behavior/risk factors 3) Promoting processes II Factor Analysis: Reduced the large number of variables (i.e., codes) down to a few factors for each of the 3 subsets of variables -responsiveness, problem/risk factors, and promoting processes III Cluster analysis: Students were “clustered” into groups based on similar profiles across the factor scales (i.e., on responsiveness, problem/risk factors, and promoting processes) IV Chi-Square to test the effect of cluster group (each group with its own “profile” across responsiveness, problem/risk factors, and promoting processes ) on post secondary outcomes

20. Coding System for Analysis of Qualitative Data

21. Improved sense of competence/ability; positive school experience; looks for attention and approval from mentor

22. Goal oriented and goal setting; self-awareness including self evaluation and discussion of future self

23. Improved sense of control & improved sense of competence: Hopeless and giving up but with sustained effort (versus avoidance), reduction of negative chain reactionsand “corrective” experience

24. Results of Qualitative Analysis: Fine tuning and reliability of coding system • Coding system: fine tuning • For each service note, code was noted if present (0 or1) • Progress/report cards to confirm related mentor reports • Double-coded 18 cases • Reliability: • Aggregated data by obtaining % across observations • Calculated alphas: two sets of coded data for same youth • Mean reliability = .92

25. Factor Analyses: Description • Data Screening • Aggregated data by obtaining % across observations • One outlier that was removed • Transformed variables where necessary (skewness or kurtosis) • Separate Analysis(responsiveness, problem behaviors/risk factors, promoting processes) • Principal Components Analysis: Decide on number of factors • Factor Analysis: Varimax rotation to maximize the spread in loadings • Factors • Responsiveness to Project: 4 Factors/16 “observed” variables • Problem Behaviors/Risk Factors: 4 Factors/17 observed variables • Promoting Processes: 3 Factors/10 observed variables

29. Cluster Analyses: Responsiveness, Problem/Risk factors, Promotive Processes

30. Results of Cluster Analysis

31. Chi-Square: Count/Percentage within Groups (χ2=.35, p<.00)

32. Conclusions • Using process data obtained from a comprehensive program that had noted success, was able to meet the challenges that are current in the youth development field: • Used meaningful measures • Assessed the whole child • Demonstrated that developmental outcomes can be linked to long term academic outcomes • Illuminated the day-to-day processes of the program

33. Conclusions • For whom, in what ways, and under what circumstances? • Was able to identify patterns of change among subgroups • Meaningful overlap on student responsiveness, problem behaviors and risk factors, and promoting processes. • Results support a developmental-ecological framework: • In some cases, the project was able to mitigate exposure to risk. • In other cases, the project was able to promote positive development.