Evaluating for Impact Learning Circle Project

1. Evaluating for Impact Learning Circle Project Faculty Presentation Name: Ellen Rowe Title: Community & Leadership Development

2. Ideal Distribution of Evaluation Capacity in Public Service Organizations(Douglah, Boyd, & Gundermann, 2003)

3. Content Modules • Program Planning • Focusing an Evaluation • Evaluation Questions and Designs • Evaluation Methods • Collecting and Handling Data • Analyzing and Interpreting Data • Communicating Evaluation Results

6. How to Find the Educational Content http://www.national4-hheadquarters.gov/comm/4h_ydresearch.htm (Educational Content and Self-Assessment Tool)

7. Goals of Evaluating for Impact Project • Provide learning and practice opportunities to develop educators to the practitioner level • Provide coordinated education in program evaluation that is consistent in scope and depth across the country • Provide opportunities for learning and practice through the use of evaluation learning circles and collaborative learning environments- both on-line and on-site • Provide customized content that is useful for 4-H educators, thus facilitating the application of what is learned

8. Input: Facilitated Project in Vermont 1) Recruit eight to ten 4-HYD professionals to participate in a learning community focused on the modules 2) Describe their plan for engaging the learning community through electronic and/or in-person communication 3) Describe how they implemented the learning community 4) Describe how users perceived different parts of the modules and their understanding of concepts (I would recommend 3 or 4 main topics/concepts/etc. as selected by your Evaluating for Impact team) 5) Report back a 2-3 page brief (due May 1, 2009) and participate in a follow-up teleconference after the learning communities completed their work with the modules

9. Vermont Evaluation Mentoring Project • Complete a self-assessment • Review results: - for module areas to strengthen (ability/ capacity) - for module areas important to your job - for module areas important to the organization • Select modules to focus learning upon

11. Results & Reflections • Up to 12 hours per week for mentor • Comfortable with facilitated learningfrom YDPA participation • Independent Study method appreciated with staff busy schedules • Members were actively engaged; conference calls regularly at 100% participation • Six of the Learning Action Plan addressed module 3 (Evaluation Questions & Designs); sense of collective inquiry, shared freely

12. Goal: To create better questions for the evaluation I conduct; ultimately to get richer data that I can use to support program outcomes. “Responses to 2010 Teen Congress revealed that many kids are challenged by having to speak to people they don’t know and Teen Congress helps them to get over this. “Conducted an evaluation on the experience of Teen Board members. I used the findings from this evaluation to help launch 4-H Leadership Clubs in afterschool programs.”

13. Goal: To create a 4-H SET parent Survey to be administered at 2009 VT 4-H State Day to use as a baseline for future evaluations of VT 4-H SET programming. “I created a survey and implemented it at 2009’s State Day. As a result of the survey, 1)VT 4-H created the 4-H SET web page to inform & promote program offerings, trainings and 4-H Science resources available to 4-H volunteers, teachers and home school families; and 2) a 4-H Science “what’s new” type of article has appeared in every VT Clovergram since June 2009. “Plan to replicate the survey at State Day, May 2011.”

14. Goal: To create an evaluation for the State Hippology Contest to better assess learning related to biosecurity, life skills, and SET abilities. “Incorporated a pre-test to bring focus of participants to these important topics: do you like science? what are life skills?; what is biosecurity? “Post event assessment addressed: what did you learn that was science-like?; what biosecurity steps will you use at home?; and what life skills did you use today?” “Changing the evaluation in 2011 to make it a little more youth friendly.”

15. Goal: Create an evaluation for the Horse Clinic that will indicate the level of learning taking place in the 2 day Clinic. “Survey data was shared with the 4-H Horse Leaders and the new coordinator for the 4-H Horse Clinic. “Plans are being made for changes in the 2011 Clinic that will reflect some of the concerns identified in the 2009 survey. The Clinic has been primarily for riding lessons in the past. This year’s Clinic will include activities for 4-H members in the Horseless Horse project. There will also be more structured time for group activities designed to foster teamwork, communication and acceptance of differences.”

16. Baughman, S., Arnold, M., Boyd, H.H., Franz, N.K., Mead, J., Rowe, E., & Silliman, B. (2010). Evaluating for Impact professional development curriculum delivery through learning circles. Journal of Extension. [On-line Serial], 48 (3). http://www.joe.org/joe/2010june/tt3.php