1 / 18

National Mentoring Summit January 24, 2013

theresia
Download Presentation

National Mentoring Summit January 24, 2013

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Toward Better Quality Control and Enhanced Outcomes: The Value of Nuanced and Comprehensive Assessment of Match CharacteristicsJohn HarrisApplied Research Consultingwww.MentoringEvaluation.com Michael NakkulaUniversity of Pennsylvania Graduate School of Educationwww.gse.upenn.edu/faculty/nakkula National Mentoring Summit January 24, 2013 NOTE: an updated and annotated version of this presentation will be available at www.mentoringevaluation.com prior to 1/24/13.

  2. John Harris • Principal, Applied Research Consulting • Match Quality Surveys • Youth Mentoring Survey (YMS) • Match Characteristics Questionnaire (MCQ) • www.mentoringevaluation.com • Connecting research and practice • Promoting high-quality mentoring • Fostering efficiency and accountability • Advocating for practitioners to be able to focus on what matters most

  3. Mike Nakkula • Chairs the Division of Applied Psychology and Human Development at Penn GSE • As a practice professor, works to link theory and research with applied challenges in child and youth development • Primary mentoring interest: how match characteristics interact to promote better mentoring outcomes within specific contexts and for different purposes

  4. Agenda • Problem: modest outcomes • Part of the solution: improved understanding of how outcomes are achieved • Evidence in support of that solution: new findings

  5. The Problem as We See It • Modest findings • Small average effect sizes belie what effective mentoring is capable of accomplishing • Reasons: • Mentoring is an individualized service • Outcomes are specific to individual need • Pathways to achieving outcomes vary • Broad guidelines but little specific, evidence-based insight • Reduces efficiency and effectiveness • Increases variation in match quality within and between programs

  6. Understanding the Engine: Quality vs Characteristics • Mentoring epitomizes growth through relationship • What should the relationship look like to promote broad gains? Particular outcomes? • What is “match quality?” • General usage: how participants feel about their match • Our definition: the ability of a match to facilitate outcomes for served youth • What is known about “quality?” • Why we prefer “characteristics”

  7. Framework for Organizing Match Characteristics* *(Reprinted with permission) Nakkula, M., & Harris, J. (in press) Assessing mentoring relationships. In Dubois, D. & Karcher, M. (Eds.), Handbook of youth mentoring (2nd Ed.). Thousand Oaks, CA: Sage Publications.

  8. Early Evidence • Qualitative findings • Surveys’ usefulness in match supervision • Coherence/interrelatedness • Intuitive relationships between different match characteristics • Expected relationships found consistently

  9. Current Study: Program Characteristics • School-based • Elementary students • Moderately structured • Multiple sites • Key innovation: mentor partners

  10. Current Study:Design • Pre/Post design • Self-selected sites • Practitioners collected youth data • Adults responded via Survey Monkey • Sample • Over 200 matches with youth data • Multiple sites of varying size • Kindergarten through 5th grade • Predominantly white mentors and mentees • What was measured • Program characteristics • Personal characteristics • Match characteristics • Outcome change over one year • Mentor, mentee, teacher, parent perspectives

  11. Current Study:Instrumentation • Tools Used (partial or in entirety) • Match Characteristics Questionnaire (MCQ) • Youth Mentoring Survey (YMS) • Hemingway Measure of Pre-Adolescent Connectedness (Karcher) • Self-Perception Profile for Children (Harter) • Strengths & Difficulties Questionnaire (Goodman)

  12. Modeling Core Academic Functioning Most economical model explaining variance in Core Academic Functioning Youth-reported Growth Focus5% Unexplained Variance 49% Interaction: Mentor’s Partner x Mentee’s Age33% 38% Pretest Score12% Note: Engagement of Mentor’s Partner explained 15% of the variance when not included in the interaction. Mentee’s Age accounted for 2% of the variance when not included in the interaction (not significant).

  13. Modeling Academic Self-Esteem Most economical model explaining variation in Academic Self-Esteem Sharing Purpose x Instrumental Purpose 10% MCQ Handle Issues 6% YMS Internal Relational Quality 10% Unexplained Variance 56% 26% Pretest Score18%

  14. Modeling Peer-Related Self-Esteem Most economical model explaining variation in Peer-Related Self-Esteem MCQ Academic Purpose 8% YMS Prescription 2% YMS Growth Focus 11% Unexplained Variance 47% Pretest Score32% 21%

  15. Key Points, All Three Models • The best predictors vary according to what’s being predicted. • Match characteristics were stronger predictors than program or individual characteristics. • Modeling core academic functioning, external MQ was the strongest predictor (as part of an interaction). Modeling academic self-esteem, internal MQ was the best. Modeling peer-related self-esteem, structure was the best. • Structure is the only match characteristic reflected in each model. • Each model included both youth- and mentor-reported data. • Specific subscales were more useful than broadscales. • Interactions are the strongest single predictors available in each model (though we chose to present a simpler model for peer-related self-esteem). • There’s still a lot of unpredicted variance in each model.

  16. Limitations/Caveats • Preliminary findings • Correlations, not causes • Very limited generalization • Limited practical utility(for now)

  17. Key, Unanswered Question • Were the most successful match dynamics a byproduct of a general approach (i.e., meet the student where he/she needs to be met)ORWere the most successful match dynamics a product more specifically directed effort? • Even if the findings were generalizable, how should practitioners use the insight? • There may be a risk of overly prescriptive mentoring if practitioners coach mentors to strive for overly specific efforts.

  18. Next Steps • For research • Integrated methods study at larger scale • Quantitative: More fully investigate the range of variation in relationships between pre-mentoring conditions, match characteristics, and outcome. • Qualitative: Better understand what these relationships mean about how to do good mentoring. • For practice • Importance of external support • Mentoring partner • It may be more important to ask what participants are doing than how they feel about it.

More Related