Tutorial 5

1. Tutorial 5 Service-Learning Assessment

2. At the end of this tutorial, you will have an understanding of: Learn about the importance of assessing service-learning Understand the assessment process Realize the various strategies and rubrics used in assessment Tutorial Goals

3. At the end of this tutorial you will be able to: Generate potential assessment measures for your service-learning course Create potential rubric for assessment Tutorial Objectives

4. Method of gathering & interpreting information to identify whether we are successful at meeting student needs & achieving learning outcomes Provides feedback to enable faculty to identify changes and make improvements in courses A process not an event Allows us to respond to the needs of students, community partners, and ourselves What is Assessment?

5. Why is Assessment Important? • Improves the quality of academic programs • Enhances student learning • Makes explicit processes and outcomes that are often implicit • Can help faculty become better teachers • Helps us provide more effective accounting of our value to our communities and public constituencies

6. Why is Assessment Important? • Provides for wiser planning, budgeting, curriculum change, staffing, programming, and student support, among others… • Creates a climate of caring and engagement that supports students’ own commitment to their learning • Helps us to be accountable for what we say we are doing **You may have been doing assessment all along; you may just not have thought of it as assessment… (Diamond, R. M. 1998; Walvoord, B. E., 2004)

7. Partners for Assessment Complete assessment should include all partners of the service-learning experience: • Student Assessment • Instructor Assessment • Institutional Assessment (may relate to program, department…) • Community Assessment This tutorial will focus mainly on student-related assessment but ideas can easily be modified

8. Steps for Effective Assessment • Begin with Clear Goals and Objectives • Plan Design of Evaluation • Select Data Collection Methods • Collect Data • Process and Analyze/Evaluate the Results • Report Feedback from Results • Make Appropriate Adjustments Based Upon Feedback; loop starts all over again

9. Begin with your clear goals and objectives of what you want the students to learn and what you want them to be able to do at the end of the project/semester Hopefully you already have thought of some goals/objectives in your initial planning of your service-learning project Goals and objectives are also covered in tutorial #2 should you like to review those items Goals and Objectives

10. Design: Two Main Types of Assessment • Formative Assessment • Assessment conducted during the activity/project • On-going as the activity/semester is taking place • Provides feedback that can be used to modify, shape, and improve as it is happening • Summative Assessment • Conducted after the program/semester…is completed or has been in operation for a length of time • Helps to make judgments of what went well, what needs to be improved for the next time • Useful for making adaptations for a future activities • Sometimes used as a before-after instrument

11. Quantitative methods: Quantitative methods are for quantifying numbers and percentages; that is, for measurement. This can be effective for some evaluation. Questions like: Was there a change? How big was the change? What was the satisfaction level? What proportion of students achieved the objective? Two Main Methods of Data Collection

12. Methods • The primary tools for quantitative data collection are: • Surveys - self administered (paper questionnaires) • Personal interviews (face-to-face or phone) • Tests • Other objective measures of outcomes (perhaps grades – careful with clearances and reporting though)

13. Methods • Qualitative methods: Qualitative methods are best for gaining a deep understanding of students’ experiences, motivation, opinions, concerns, and so on… • Questions like: • How have students civic behaviors changed based upon their experiences? • How have students’ perceptions of the elderly been shaped by their community visits? • How have the community members been impacted by students’ presence in their homes?

14. Methods • The primary tools for qualitative data collection are: • Focus groups (moderated group discussions) • Critical incident review • Reflections/journal reading • Open-ended questionnaires • Personal interviews (one-on-one interactive discussions) • Observation (in-class or field observations are used for capturing information on what is actually done and generating a qualitative sense of the experience)

15. Many times, rubrics are used to measure students’ behaviors, activities, performances… Once you know your methods and what you want for data, you can create rubrics (Gelman, S., B. Holland, et. al. 2001) Creating Rubrics

16. Determine what you want to measure – more than likely an outcome, may be via skill, knowledge, or attitude Do you simply want to divide students into two or three groupings, based on whether they have attained or exceeded the standard for an outcome?  If so, then a short scale may be adequate Do you want to use the rubric as a grading scale also? Then perhaps a 5-pt scale is more appropriate Creating Rubrics

17. Think of the best acceptable response for this item (1-5 scale) – your 5 Think of least acceptable – your 1 Think of minimal/intermediate acceptable response - your 3 Develop descriptions and labels for each item Creating Rubrics

18. Sample of Major Assessment Competencies Measures Adapted from National Communication Association criteria, http://www.natcom.org/nca/Template2.asp

19. Measure performance on project or assignment with items linked to learning objectives Ex; looking at linking concepts from class in reflection; analyzing societal implications of health care policies on the elderly via community experiences Self-assessment done by students to gauge their self-perception of some aspect of course/outcome Rubrics can be used to:

20. Process and Analyze Data • May need some statistical knowledge; or general descriptive analysis • May be looking for relevant themes • May be able to go a quick cross-check to see whether objectives have been met • Determined by data collection

21. Report Feedback from Results • With your guidance and comments, assessment can provide students with feedback on their performance and direction for improvement • Helps instructors better articulate the effectiveness of this approach to learning; Are students understanding material? Are they successful with achieving outcomes? Are you effective at getting ideas across to your students?

22. Report Feedback from Results • Keep careful track of your results so you can report and accumulate feedback on an ongoing process • Always keep in mind that feedback from assessment can be an advantage in the future (i.e., funding purposes, obtaining resources, demonstration of teaching effectiveness for tenure and promotion…) in addition to the basic intention of knowing whether learning outcomes are achieved

23. Make Appropriate Adjustments Based Upon Feedback • Once you have analyzed the results, you need to close the feedback loop with making any adjustments in your assessment plan • Do your results give you the information to link directly to your outcomes? Can you determine from your instrument whether students are meeting the outcome? If not, perhaps the assessment instrument needs to be modified • Do your results tell you that students are not meeting the intended outcome you want? Different issue---perhaps you need to modify the outcome • Once you make adjustments, begin the process again to see if your adjustments worked and continue assessing…

24. General Tips for Assessment • Do not try to re-invent the wheel • Look for those instruments and rubrics already created that you can modify for your purposes • Create a plan and try to stick to it • Always different/perhaps better ways to do assessment • Save that for the feedback look and adjustment step • Don’t be afraid to ask for help • You may not know everything about assessment • Tap into your resources on campus, websites, others texts… • Assume that assessment is “messy” • Be sensitive to people error and have a sense of humor

25. Assessment with Different Parties of Service-Learning • We covered mainly student assessment; other issues to consider with faculty, community partners and institutional/programmatic assessment • Basically same principles, but perhaps different data collection and issues to consider with providing results and feedback; different purposes

26. Final Thoughts • Learned the steps for conducting assessment • Covered some different types of assessment techniques and methods of data collection for examining your learning outcomes • Learned how to create rubrics, which can be helpful in various teaching & assessment purposes • Hopefully you are well on your way to be confident in your service-learning classroom and will be ready to determine whether your students are really benefiting from their community experiences • Various assessment samples and grids for you to example and complete on the web pages

27. Diamond, R. M. (1998). Designing & assessing courses & curricula: A practical guide. San Francisco: Jossey-Bass. Gelman, S., B. Holland, et. al. 2001. Assessing Service-Learning and Civic Engagement: Principles and techniques. Providence, RI: Campus Compact. Howard, J. (Ed.) (2001). Service-learning course design workbook. Michigan Journal of Community Service Learning. Ann Arbor, Mi: OCSL Press, The University of Michigan Walvoord, B. E., (2004). Assessment clear and simple: A practical guide for institutions, departments, and general education. San Francisco: Jossey-Bass http://www.natcom.org/nca/Template2.asp References