An Adventure in Assessment

1. An Adventure in Assessment Stephen “Gavin” Weiser, M.Ed stephen.weiser@gmail.com

2. A Lil About Me • Gavin Weiser • Residential Experiential Education Facilitator • Princeton-Blairstown Center • Recent M.Ed from U. South Carolina

3. Let’s Hear About You • Find someone from a similar type of program as folks are talking.

4. Outline • What is Assessment • Purposes of Assessment • Some Pitfalls • Methods • Learning Outcomes - Creation • Tool Selection & Creation

5. What is assessment? • A. What we do to people we don’t like • B. An old wooden ship • C. Something we all hate and loathe • D. A sustainable process of collecting data to improve upon services & programs

6. Purposes • To collect data in order to better understand what we are doing • What can this look like?

7. Pitfalls • “teaching to the test” • The politics • Fatigue – particular to certain methods • Some bad ways to use assessment.

8. Methods • Qualitative vs. quantities • Mixed Methods • Literature review

9. Learning Outcomes • Why are they important? • What are they? • Measureable & meaningful • Examples: • To gain leadership abilities. • Good or bad?

10. Tool Selection • Based upon what we are trying to measure • Different Types: • Surveys • Focus Groups • Interviews • Structured vs. Unstructured Protocols • Field Observation

11. Create a“Culture of Assessment” • What does this look like? • Vital to it’s and the programs success!

12. Example Project

13. Background of Outdoor Recreation at Carolina • Opened Fall 2008, Director: Katie Coley • Information from Katie, annual report, focus groups with current student leaders • Annual Report • Strengths • Weaknesses • 5 year plan

14. Programs of Outdoor Recreation • Climbing Wall • Clinics • Adventure Trips • Bike Shop • Student Leadership • Outdoor Leadership Training

15. Focus Groups • 8 Students Leaders; 2 groups of 4 • Methodology • Provided information based on experiences and goals with outdoor recreation • Information received assisted in guiding what learning outcomes should be created

16. Previous Assessments in Outdoor Recreation • Office had “learning outcomes” but they were immeasurable • Same tool was used to assess all programs • Tool focused on satisfaction

17. Purpose of Assessment • New office which allowed for flexibility • To create a process that can be used for future assessments within Outdoor Recreation • Outdoor Recreation programs nationally have a need for quality assessments • Provide feedback on some of the learning outcomes related to the Rock Wall

18. Methodology • Students that have completed climbing the rock wall • University 101 and walk-up participants • A qualitative assessment was distributed post wall climb • 2 week period in November – only one week is included in this analysis

19. Learning Outcomes • Started from focus group, Katie’s feedback and goals • Took two goals: Teambuilding and Trust and developed outcomes from them • Created measurable outcomes

20. U101 Outcomes • Recognize individual’s strengths within a working group. • Understand how to apply strengths in an existing challenge. • Demonstrate a willingness to rely on others during a challenge. • Participants will learn technical skills related to the activity in which they participate.

21. Individuals’ Outcomes • Demonstrate an increase in willingness to go beyond limitations and boundaries. • Participants will learn technical skills related to the activity in which they participate.

22. Tool • Selected a qualitative instrument based on: • feasibility, rationale, design, research questions • Technically mixed method • Local instrument – developed by us • Open-ended questionnaire • Challenge: probing but not leading and receiving the information that was necessary • Katie’s feedback to make improvements to the tool • Created a second tool

23. Analysis • Used a coding procedure • “Focused coding” • Codes were guided by outcomes and data • U101: coded as whole system • Individual: coded by question • Used SPSS to develop codebook and pie charts for representations

24. Results • Solid process of developing a survey based on targeted outcomes – Major Result • Most applicable to outcomes will be discussed • Others are informative but not directly tied to outcomes

25. Individual: Expectations and Challenges Reasons For Visiting Challenges 40% 46.70% 41% 52.9% 13.3% 5.90%

26. Skills Learned Individual U101

27. U101: How were challenges overcome? Interesting to note, this came as a surprise

28. Points of Improvement • Tool did not always provide needed data • Not a large sample size • Difficult to know if technical skills were learned during activity • Time constraints

29. Strengths • Informed by focus group data • Process for developing outcomes and tool was well documented and could be replicated • Tool design – well-sequenced • Informed support from whole office • Code and analysis were manageable and clear

30. Recommendations for Future • Use process to develop a comprehensive tool for each activity • Connect outcomes to Campus Recreation outcomes • Use satisfaction data and skills data to publicize the wall, attract non-climbers • Be intentional in educating student participants about outcomes • Expand office staff

31. Partner, Pair, Share • Find that special someone from earlier • Develop outcomes and the beginnings of a tool.

32. Thank You! • Contact: Gavin Weiser @ • stephen.weiser@gmail.com • (609) 458-2151