1 / 17

Susan Carter Evaluation Coordinator National Indian Youth Leadership Project

Peering into the black box between pre and post: A theory of change related to interactive/ experiential delivery methods. Susan Carter Evaluation Coordinator National Indian Youth Leadership Project susanleecarter@comcast.net with J. Fred Springer, EMT Associates fred@emt.org.

perron
Download Presentation

Susan Carter Evaluation Coordinator National Indian Youth Leadership Project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Peering into the black box between pre and post: A theory of change related to interactive/ experiential delivery methods Susan Carter Evaluation Coordinator National Indian Youth Leadership Project susanleecarter@comcast.net with J. Fred Springer, EMT Associates fred@emt.org

  2. A presentation for the 6th Annual Research and Evaluation in Adventure Programming Symposium Gaithersburg, MD March 17-19, 2010

  3. 48 site quasi-experimental design • 10,500 youth • 4 point longitudinal study • Common instrument • Comprehensive process data • Hierarchical statistical design • American Evaluation Association Study of the Year award, 2000 • Largest direct service ATOD prevention evaluation ever implemented The Multi-Site High Risk Youth Study funded by SAMHSAFred Springer, EMT, Co-PI

  4. Common Factors Reducing 30-day Drug Use .18 .135 Effect Size .09 .045 Connection Building Life-skills Focused Coherent Introspective Orientation Intense Contact 4

  5. Promoting introspection / connectedness are critical common factors • Out of school more effective • Manualized programs less effective • Intensity counts • Promoting behavioral skills more effective than focus on knowledge and attitude change HRY Common Factor Findings

  6. Project Venture identified as one of eight best programs in HRY study • PV included all of the common factors • Incorporates connectedness building and introspection • Uses interactive experiential program delivery strategy Application: Project Venture

  7. Population • Middle school aged American Indian youth • High school aged AI service staff • Mix of risk levels (at risk + high risk) • Program Components • Experiential education • Connecting to the natural world • Physical and social-emotional challenge • Service learning • Focus on developing positive relationships • Positive Youth Development approach Essential Elements of Project Venture

  8. Experiential, adventure-based positive youth development program for American Indian youth • Recognized by NREPP as evidence-based • The only American Indian ATOD prevention program currently in NREPP • Widely disseminated throughout the US and Canada • Twenty year implementation and evaluation history Project Venture: Overview

  9. Outdoor, adventure activities: ropes course, rappelling, canoeing, backpacking, camping, mountain biking, etc. • Service Learning Meaningful projects that are youth centered and designed, address cultural, environmental and other community needs. In a research-supported, year long sequential program. Project Venture: Strategies

  10. Positive substance abuse prevention findings (prevent, reduce, delay onset) especially related to alcohol abuse • Increased internal assets • Increased external assets, especially community and peer domains Project Venture: Evaluation Findings

  11. We propose to advance knowledge concerning how adventure based strategies and activities can be strengthened, promoted, and replicated. Purpose of proposed study

  12. Interactive /experiential activities that clearly incorporate reflective learning skills and promote connectedness to positive people and environments will promote intended behavioral change more effectively than interactive/experiential activities that have less focused emphasis on these interim outcomes. Hypothesis

  13. Quasi-experiment • Baseline, exit, six and twelve month follow up • Assess similar participants in Project Venture programs: • 6 programs using an enhanced version of PV • 6 “regular” PV programs • (perhaps 6 groups with no PV) Method notes

  14. Behavioral outcomes (ATOD, etc.) • Connectedness (e.g., Michael Karcher’s Hemmingway scales) • Reflective learning (perhaps adapted from education literature) Measures

  15. Rigorous development and testing program to develop the measures • Rigorous statistical methodology including propensity scores for improving comparison equivalence • Hierarchical modeling to account for nesting effects, treatment interactions, treatment settings • Strong fidelity/coherence measurements Analysis notes

  16. High Risk Youth Study, SAMHSA mapped onto Project Venture

  17. Secondary data analysis of 5 years of pre/post survey data from Project Venture 7th grade participants (600) and matched comparison group (400) collected between 2005-2010 AND • Longitudinal follow-up study of PV program participants who were in 7th grade in 2006 – including original program survey (adapted for older youth cohort) AND life history interviews Other study ideas

More Related