1 / 51

An Overview of Virginia 4-H Camping Program Evaluation: Process and Outcomes

An Overview of Virginia 4-H Camping Program Evaluation: Process and Outcomes. 2002 National 4-H Camping Institute, Ashley, OH February 13-17, 2002 Barry A. Garst, Extension Specialist 4-H Staff Development and Camping Becky Gilles, Program Director Smith Mountain Lake 4-H Educational Center

cecily
Download Presentation

An Overview of Virginia 4-H Camping Program Evaluation: Process and Outcomes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An Overview of Virginia 4-H Camping Program Evaluation: Process and Outcomes 2002 National 4-H Camping Institute, Ashley, OH February 13-17, 2002 Barry A. Garst, Extension Specialist 4-H Staff Development and Camping Becky Gilles, Program Director Smith Mountain Lake 4-H Educational Center Bryan Branch, Program Director Holiday Lake 4-H Educational Center Ruben Brown, Urban 4-H Program Assistant Chesapeake Cooperative Extension

  2. Why this session? • Increasing importance of demonstrating the value and benefits of youth programs • Increasing emphasis on the development of rigorous, systematic evaluation procedures and “good” instruments • Importance of sharing information and approaches to evaluating the impacts/outcomes of youth camping

  3. What are your expectations? • What information are you hoping to gain through this session? • Why is this session important to you?

  4. Overview • Overview of Virginia 4-H Camping • 4-H camp evaluation prior to 2001 • 2001 data sources • Results and analysis • Limitations • What did we learn? • Where do we go from here? ASK QUESTIONS AT ANY TIME!

  5. Virginia 4-H Camping • Involved in 4-H camping since 1917 • Recognized as one of Virginia’s primary delivery modes for 4-H educational programs • Conducted at the unit-level (107) and 4-H Center (6) level • Approximately 30,000 served annually • From 1995-2001, approximately 4,000 more youth served

  6. Virginia’s 4-H Educational Centers

  7. 4-H Educational CentersOperation and Management • Virginia’s six 4-H Educational Centers are owned and operated by their respective Boards of Directors • Each 4-H Center has a Memorandum of Understanding with Virginia Tech • Virginia Cooperative Extension provides funding for (1.5 FTE) positions and operations • Virginia Tech provides insurance for facilities • Each 4-H Center is accredited through the American Camping Association

  8. 4-H Camp Evaluation Prior to 2001 • Parent/guardian study (1995 and 1998) • Some 4-H Centers were evaluating post-camp satisfaction (ratings, etc.), • Some 4-H Centers were not conducting regular evaluation • No state-wide standardized instruments • No state-wide standardized questions

  9. Evaluation Needs Leading into Summer 2001 • Needed to be able to collect similar data across 4-H Centers • Assess overall program quality • Data to assist resource development • Baseline data for future comparison • Comparing 4-H Centers was not a goal

  10. Data Collection in Summer 2001

  11. Data Sources • Virginia 4-H Camping Participation Form • Standardized 4-H Center Survey • Parent/Guardian Survey

  12. Virginia 4-H Camping Participation Form • Distributed to 113 units in Virginia 4-H • 107 counties/cities • Six (6) 4-H Centers • Questions • WHO is participating? • WHAT programs are being offered? • WHEN are programs taking place? • WHERE are programs being conducted?

  13. Standardized 4-H Center Survey • Population sample • Every Junior 4-H Camp participant • Post-camp survey (last day of camp) • 11,254 surveys collected (82% response) • Questions • Are youth, teen/adult volunteers, and Agents receiving a positive experience during Junior 4-H Camp? • Are youth campers (ages 9-13) receiving life skill benefits as a result of participation in Junior 4-H Camp?

  14. Parent/Guardian Survey • Random sample of youth selected • Survey mailed to parents/guardians • Post camp survey (2 weeks after camp) • Survey/Reminder/2nd Survey • 363 surveys returned (46%) • Questions • What are parents’ perceptions of Junior 4-H Camp? • Are youth campers (ages 9-13) receiving (short-term) life skill benefits as a result of participation in Junior 4-H Camp?

  15. Virginia 4-H Participation Form

  16. 30,175 servedThrough 4-H Camping/Environmental Education

  17. 4-H Camping ParticipationYouth, Teen Counselors, Adult Volunteers* * Based on a total of 22,816 participants

  18. Percentage of 4-H CampingUnit-based vs. 4-H Center based 150 programs* 148 programs* * Based on a total of 298 camping programs conducted

  19. 4-H Camp Participation By 4-H Center* * Based on a total of 22,816 participants

  20. 4-H Camp Participation By Month* * Based on a total of 298 camping programs conducted

  21. Types of 4-H Camps Offered • Cloverbud (ages 5-8) • Junior 4-H Camp (ages 9-13) • Teen Camps (ages 14-18) • Day Camps (ages 9-18) • Special Interest Camps (ages 9-18) • Forestry Camp, Shooting Education, Outdoor Adventure, Science and Technology, Family Camp, Creative Arts, Nutrition Education

  22. 4-H Camping ProgramsPercentage of 4-H Camping Programs Conducted * Based on a total of 298 camping programs conducted

  23. 4-H Camping ProgramsMost Common 4-H Camp Themes

  24. 4-H Camp ParticipationGender * Total of 22,816 participants NOTE: This is a logarithmic scale.

  25. 4-H Camp ParticipationEthnic Groups Represented * Total of 22,816 participants NOTE: This is a logarithmic scale.

  26. Previous 4-H Camping Experience * Total of 22,816 participants NOTE: This is a logarithmic scale.

  27. Standardized 4-H Camp Survey Results

  28. 2001 Junior 4-H Camp Ratings By Youth Campers Ages 9-13* * Scale of 1-5, where 1=poor and 5= excellent. (n=9,296)

  29. 2001 Junior 4-H Camp Ratings By CITs, Teen Counselors, Adult Volunteers, and Extension Agents* * Scale of 1-5, where 1=poor and 5= excellent. (n=1,958)

  30. 2001 Junior 4-H Camp Interest in Future 4-H Camp Participation*

  31. 4-H Camp Standardized Survey Life Skill Items and Related Domains

  32. 2001 Junior 4-H Camp Life Skill Rankings by Youth Campers Ages 9-13* * Youth campers ages 9-13 were asked to identify whether or not attending camp helped them on a scale of 1-4, where 1= helped me very little and 4=helped me very much.

  33. Additional Analysis of Life Skill Items • Significant difference between the life skill mean response of campers ages ages 8-11 and campers ages 12-14 • “helped me to learn more about different subjects” • “helped me to develop new skills in an area that I enjoy” • Suggests that younger campers perceive more educational benefits from 4-H camp than older campers. • This trend is noticeable across most life skill areas (except life skills related to friendship development) suggesting that the Virginia 4-H camping program may be geared towards younger campers.

  34. Parent/Guardian Survey Results

  35. Other Camps in Which 4-H Youth Campers ParticipatedSummer 2001 * This figure (43%) represents 89 other camps in which 4-H youth participated.

  36. 2001 Junior 4-H Camp Ratings By Parents/Guardians of 4-H Youth Campers* * Scale of 1-4, where 1=much worse and 4= much better. (n=363)

  37. 4-H Camp Parent/Guardian Survey Life Skill Items and Related Domains

  38. Data Analysis of Life Skill Items • Conducted paired t-test comparison of “before-camp” and “after-camp” life skill averages (means)

  39. 2001 Junior 4-H Camp Life Skill Rankings by Youth Campers Ages 9-13 Before and After Attending Junior 4-H Camp* 1 Parents were asked to rate their child’s life skill behavior before and after the Junior 4-H Camp experience on a scale of 1-4, where 1= almost never and 5= almost always. 2 Paired t-test analysis found significant differences (p<.05) between all of the before and after camp mean comparisons.

  40. Additional Analysisof Life Skill Items • Additional t-test analysis of life skill rankings among male and female campers indicated individual benefits (rather than gender-related benefits) as perceived by campers’ parents/guardians • “Gender” differences do not explain significant results • Suggests that Junior 4-H camp programs/activities are appropriate for both genders.

  41. Summary of Outcomes • Youth (ages 9-13) perceived that Junior 4-H Camp participation increased their own skills (immediate outcomes) • Developing social relationships with new friends • Developing new skills • Parents/guardians perceived that Junior 4-H Camp participation increased their child’s life skill behaviors(short-term outcomes) • “takes care of his/her own things” • “shares work responsibilities” • “takes initiative and is a self-starter”

  42. Limitations • Standardized survey • Response bias (tendency to respond favorably) • Lack of pre-test (How did campers’ perceive their life skills before camp?) • Lack of analysis of open-ended responses • Parent/guardian survey • Response bias (Who returned the survey? / What about non-respondents?) • Can outcomes be attributed to something else? • Others (What do you think??)

  43. What did we learn?

  44. Value of Systematic Evaluation • 1st step has been taken!! • Standardized evaluation allowed for state-wide reporting of 4-H camp outputs and outcomes • Standardized evaluation can provide data for meaningful comparisons • Staff/volunteers learned the importance of evaluation in the programming process • Evaluation results are assisting resource development • 4-H Centers • Virginia 4-H Foundation

  45. Where do we go from here?Impact on Virginia 4-H Camping- Programs • Examine 4-H camp components that received lower ratings • Explore the 12-14 year old camp experience

  46. Where do we go from here?Impact on Virginia 4-H Camping- Program Evaluation • Utilize methods that are less labor-intensive • Review life-skills that were assessed in 2001 • Qualitative study to explore results • CIT and teen leadership • 12-14 year old experience • Long-term outcomes of 4-H camp involvement

  47. Where do we go from here?Impact on National 4-H Camping • As a 4-H camping community, we should explore 4-H camping outcomes together • COMMUNICATE- Increase communication about our instruments, methods, and results • COLLABORATE- Work together to develop a multi-state approach to evaluating 4-H camp program outcomes

  48. Where do we go from here?Impact on National 4-H Camping • Virginia invites other states to work with us to develop a National 4-H Camp Research Consortium by 2005.

  49. Summary • Overview of Virginia 4-H Camping • 4-H camp evaluation prior to 2001 • 2001 data sources • Results and Analysis • Limitations • What did we learn? • Future evaluation • State and national levels

  50. What do you still need? • Were any of your questions unanswered? If so, what are they? • What would help you to better understand or apply the information learned in this session?

More Related