Rebecca Stanfield McCown Daniel Laven Nora Mitchell

1. Early Attempts to Build Evaluation Capacity: Examples from National Park Service Youth Engagement Programs Rebecca Stanfield McCown Daniel Laven Nora Mitchell National Park Service Conservation Study Institute Jennifer Jewiss University of Vermont

2. The need for evaluation research of diversity programs in the NPS • Significant research has been done to articulate the issue and identify barriers with regards to diversity in the national parks • Theories for underrepresentation (Floyd 1999; Gomez 2002; Johnson, Bowker, English, and Worthen 1998) • Identified barriers to visitation (Payne, Mowen, and Orsega-Smith 2002; Solop, Hagen, and Ostergre, 2003; Tinsely, Tinsley, and Croskeys 2002)

3. The need for evaluation research of diversity programs in the NPS • Few (if any) studies have examined cases where the NPS has been successful in addressing issues of diversity and engaging traditionally underserved audiences

4. Keeping National Parks Relevant in the 21st Century • Focus areas: • - working with others to tell inclusive stories; • - engage in an ongoing dialogue with openness, sensitivity, and honesty; • - sustain community relationships; • - create a workforce reflective of society (Mitchell, Morrison, Farley, and Walter 2006)

5. Using evaluation research to understand diversity programs in the NPS Evaluation can provide a systematic assessment of a program or policy, contribute to its improvements, prioritize resources, and demonstrate effectiveness (adapted from Weiss, 1998)

6. Using evaluation research to understand diversity programs in the NPS Evaluation research can help identify barriers, opportunities, and strategies for increasing the visitation and participation of new and diverse audiences in national parks Evaluation can help inform ongoing program design

7. Overarching Research and Evaluation Questions How do programs overcome barriers and challenges with regards to increasing diversity? What are some key ingredients for successfully engaging diverse youth? Provide a foundation in the development of an evaluation tool for diversity programs Begin to build a theory of change model regarding diversity programs

8. Study Methods • Qualitative interviews with key informants and stakeholders • Phase 1: Preliminary Interviews and Model Development • Interviews with NPS personnel, partners, academics, and other key informants

9. Study Methods • Phase 2: Case Study Research • Youth engagement programs at Santa Monica Mountains National Recreation Area and Boston Harbor Islands National Park Area • Examined 7 programs run by the park and partner organizations

10. What we have learned…

11. What we have learned… • Cyclical nature of the model • Need to address all areas of the model in order to effectively address diversity issues • Implications for Phase 2 • Building off the model developed in Phase 1 • Examine how the model is made operational at the field level

12. What we have learned… • What is the larger context of the program regarding participants’ daily lives? • How does the program manage challenges within participants lives that impact program success? • Important considerations: • Mentorship within the context of the program and with regards to school • Continued contact throughout the school year • Barriers to participation such as access to park areas

13. Applying what we have learned to new research • Sharing information throughout the process with stakeholders and interested parks and program managers • Using evaluation to inform program design • SAMO Youth Program informs the development of the YIP program in Massachusetts Area Parks • Utilize similar study techniques • Stakeholder meeting to key ingredients and program logic • Key informant interviews