Kathy O’Byrne, Ph.D. UCLA Center for Community Learning May 5, 2010

1. Assessment of Service-learning Courses in Higher Education:Understanding the Rationale and Techniques Kathy O’Byrne, Ph.D. UCLA Center for Community Learning May 5, 2010

2. Agenda • Welcome and Introductions • Definitions of assessment and service learning • Learning outcomes • Research options: unit of analysis • Integration into your syllabus & examples • Resources • Questions and Discussion

3. Your campus and your experience • History of your service learning program? • Institutional resources? • Disciplines? • Campus leadership? • History with assessment? • Service learning courses or others? • Research and publications? • Faculty development experience?

4. Key components of assessment • Assessment is linked to learning goals/outcomes. • Meaningful work in service learning courses is designed to elicit higher order thinking • Multiple forms of evidence • Assessment activities should draw on the affective and cognitive domains of learning • Self-assessment is often part of the overall design • Student involvement and feedback is important data to guide future learning • Consider an examination of processes as well as outcomes • Pre-planning and integrated with instruction • Expectations apparent to students

5. Definitions and Common Language

6. Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance. Assessment…can help us focus our collective attention, examine our assumptions, and create a shared academic culture dedicated to…improving the quality of higher education.

7. What is service learning? • Equal parts “service” and “learning” • Reflection is key • Embedded in academic, credit-bearing courses • Connection to graded assignments • Reciprocal benefit • Collaborative approach to projects

8. Examples of strong service learning programs • Tulane University • University of San Diego • Kapiolani Community College • University of Washington • UCLA

9. Options for assessment: unit of analysis • Students: learning outcomes, knowledge, skills, attitudes, behaviors • Faculty: changes in teaching and instruction • Community: impact of the project, qualities of the partnership

10. Potential service-learning outcomes for students

11. Student learning and assessment issues • Define student learning outcomes: knowledge, skills, attitudes or values? • Systemic issues: what is the role of the student, faculty, sites? How will students demonstrate learning? • What data will you use? Graded assignments? Pre-post instruments? Other?

12. Service-learning outcomes for assessment • Understand and apply knowledge • Develop identity and values • Develop community awareness and practice community engagement • Demonstrate a commitment to lifelong learning and action • Awareness of social justice issues

13. Summarize overall plan • Expectations/assumptions about students • Learning outcomes: what they should know and/or know how to do • Teaching strategies in support of outcomes • Assessment Strategies: who? when? how? where? what?

14. Five Assessment techniques: Strategy #1: Critical Incidents • Describe an incident that was “critical” to you. • It can be surprising, frightening, enlightening or something that made you proud • What made it significant? • What did you learn or how did it change your worldview/assumptions?

15. Strategy #2: Concept mapping • One main concept from the course • Additional cards or post-its that students arrange in a patter • Related concepts connected or close to one another • Policy, diversity, social justice, the role of government, economic factors, etc.

16. Strategy #3: Problem based learning • Based on training from medical schools • Students write about an issue related to their service learning work. • Have to provide a potential solution for change in the community, that includes classroom/curriculum learning and connections to the service learning site/staff/meaningful work they performed.

17. Strategy #4: Focus Groups • Small groups of students who all worked at the same site answer structured, pre-planned questions about the organization, the community, the work they accomplished, and the future of the organization. • If change is needed, who would have to get involved to make that happen? Elected officials? Residents? Others?

18. Strategy #5: Pre-post instruments • Can focus on knowledge, attitudes, beliefs, values related to the work of the organization. • Can have students complete the pre-test and return it after they complete the post-test to compare their answers. • Can break down differences according to gender, majors, other characteristics including grades.

19. Other options for assessing service learning: faculty or academic departments • Departmental views for integrating service learning into a major or minor; • Faculty transformation after teaching a service learning course • Research and publication on discipline-based service learning

20. Assessment with Community Partners • Satisfaction • Impact • Visibility • Building the organization’s capacity • Outcomes for community residents or agency clients • Connection of projects to public policy • Research that informs decision-making

21. Example: Sample questions for service learning community partners • Rate student learning: cultural awareness, appreciation of diversity, communication, critical thinking, negotiation, career exploration, understanding of non-profits and public policy issues, etc. • General: understanding of their responsibilities, how well prepared were students? • What was the meaningful work that was performed?

22. Your syllabus • Definition of service learning • Learning outcomes • Community partners and criteria • Job descriptions for each site • Connection to graded assignments • Logistics

23. Some learning outcomes • Course Competencies and Learning Outcomes: • Students in this course will learn: • Definitions of civic engagement and service learning; • Current and historical national trends in higher education to promote democratic action and civic engagement; • The diverse traditions of service and the history of service movements; • How civic engagement and participation in public life contribute to overall quality of life in the community; • An overview of the issues raised by theories and empirical studies of links between individual motivation and success, access to resources and distribution of wealth; • A description of the national movement for reform of undergraduate education; • Trends in research and scholarship that are impacted by this movement; • Definitions and research on sustainable environments, including (but not limited to) issues of water usage, endangered species and native habitats, transportation, and pollution. • The role of community-based organizations that support a variety of sustainable environment programs, non-profits and governmental agencies; • The potential range of personal choices that individuals can make to promote sustainable environments in Los Angeles.

24. Principles of university/community partnerships (CCPH) • Principles of good community-campus partnerships: • Partners have agreed upon mission, values, goals, and measurable outcomes for the partnership. • The relationship between partners is characterized by mutual trust, respect, genuineness, and commitment. • The partnership builds upon identified strengths and assets, but also addresses areas that need improvement. • The partnership balances power among partners and enables resources among partners to be shared. • There is clear, open and accessible communication between partners, making it an ongoing priority to listen to each need, develop a common language, and validate/clarify the meaning of terms. • Roles, norms, and processes for the partnership are established with the input and agreement of all partners. • There is feedback to, among, and from all stakeholders in the partnership, with the goal of continuously improving the partnership and its outcomes. • Partners share the credit for the partnership's accomplishments. • Partnerships take time to develop and evolve over time.

25. A “menu” of service learning options: Based on time available, resources, and experience • An “Appetizer”: Pre-post research on knowledge, attitudes, etc. • An “Entrée”: Ongoing collection of data throughout the academic term or a comparison of service learners to non-service learners • A “Dessert”: small scale study at the end of a course.

26. Examples • Attitude toward Korean immigrant neighborhoods before and after the service learning experience • Compare the learning outcomes of two sections in a large general education course (service learning and non) • In what way would you say you have changed your mind—if only a little bit—about one issue, as a result of the service learning work?

27. Resources: Publications and Conferences • Journal of Applied Learning in Higher Education): 2009 Ash and Clayton • Generating, Deepening, and Documenting Learning: The Power of Critical Reflection in Applied Learning • This article will consider the meaning of critical reflection and principles of good practice for designing it effectively and will present a research-grounded, flexible model for integrating critical reflection and assessment. http://www.missouriwestern.edu/appliedlearning/issues.asp • 1999: Where’s the Learning in Service Learning? Janet Eyler and Dwight Giles. • National Service Learning Clearinghouse • Bonner Foundation • Campus Compact • Tenth International Research Conference on Service-learning and Community EngagementInternational Perspectives: Crossing Boundaries through ResearchOctober 28-30, 2010Indianapolis, IN. • The Indiana University-Purdue University Indianapolis (IUPUI) Center for Service and Learning, Indiana Campus Compact, and the International Association for Research on Service-learning and Community Engagement. • October 28-30, 2010 at the Crowne Plaza at Historic Union Station in Indianapolis, IN. • The featured theme for the 10th annual conference is “International Perspectives:  Crossing Boundaries through Research.”  Service-learning is valued as an active learning strategy across the globe; however, little is known about the ways that service-learning is similar or different in varied contexts. Understanding service-learning and community engagement from diverse cultural perspectives will add insight necessary for comparative research and to improve practice..

28. Questions ? • Reactions • Discussion • Relevance • Need for further information • Next steps for your service-learning course!

29. Contact information • Kathy O’Byrne, Ph.D. • UCLA Center for Community Learning • kobyrne@college.ucla.edu