Kathleen J. Phillips, Ed.D. Education Consultant 307 Moultrie St. San Francisco, CA 94110

1. Kathleen J. Phillips, Ed.D. Education Consultant 307 Moultrie St. San Francisco, CA 94110 (415) 282-4904 (T) kpsf@comcast.net Making Evaluation Work for You A Case Study of St. John’s Educational Threshold Center

2. St. John’s Educational Threshold Center (SJETC) • After-school programs in academics, leadership, health and wellness, sports and recreation, and the arts • Serves 1100 youth, ages 5-19 • Low-income, mostly Latino, Asian American and African American population • Serves the local elementary school and middle school and neighborhood kids • Many stay involved for 5-8 years

3. SJETC Evaluation Strategy Criteria • Enhance agency, staff and youth control over the evaluation process • Provide comprehensive information • Realistic • Reflect agency values and mission • Provide useful information for program planning and development • Demonstrate program impact

4. SJETC Evaluation Strategies • Focus groups and surveys collecting program-specific data • Youth-led evaluation of specific aspects of SJETC programming • Longitudinal survey data on all youth across all programs • Review of relevant data gathered by public and private funders • Review of relevant data collected by other community groups and researchers

5. Using the Evaluation Data 1. Assessing and understanding the impact of programs 2. Informing program planning 3. Supporting fundraising and public relations 4. Empowering everyone involved

6. 1. Assessing and understanding the impact of programs • Consistent documentation—not just stories • Enables assessment of impact over time • Helps us know what’s working and what’s not • Provides a TRUE youth perspective

7. 2. Informing program planning • Informs mid-year program adjustments • Helps identify and clarify needs • Supports planning with collaborators • Enriches agency-wide strategic planning

8. 3. Supporting fundraising and public relations • Allows us to tell our story on our terms • Justifies our community and program needs • Provides information and materials for general education and outreach

9. 4. Empowering everyone involved • Agency: Strengthened mission, own sense of legitimacy, and funders’ respect • Youth: Took on a respected, powerful, and real role • Staff: Gained understanding of, control over and appreciation for evaluation

10. Making Evaluation Work and Become a Source of Power • Who needs to be involved? • What needs to be in place?

11. Who needs to be involved? • Organizational leader(s) • All levels of the organization: youth, staff, administrators • Someone who really understands evaluation— staff-members and/or consultants

12. What needs to be in place? • An overall strategy, including plans for using evaluation findings in meaningful ways • Dedicated funding and staff time • A culture that supports youth empowerment (if incorporating youth-led evaluation) • Skill in youth leadership training (if incorporating youth-led evaluation) • Stipends and food whenever possible