Mary Allen Wayne Community College October 14, 2010

1. Assessment of College-Wide Learning Outcomes and General Education Competencies Using Signature Assignments Mary Allen Wayne Community College October 14, 2010

2. Assessment Assessment is an on-goingprocess designed to monitor and improve student learning. Faculty: • develop SLOs • verify alignment • develop an assessment plan • collect assessment data • assess the evidence and reach a conclusion • close the loop

3. Quotations from the Wise and Experienced

4. Vocabulary • Direct vs. Indirect Assessment • Embedded Assessment • Authentic Assessment • Formative vs. Summative Assessment • Triangulation

5. If you have absolute outcomes, your assessment plan should emphasize direct, authentic, summative assessment, with triangulation.

6. SLOs • Clarify what faculty want students to learn • Clarify how the assessment should be conducted

7. Articulating Learning Outcomes: • Knowledge • Skills • Values

8. Outcomes at Different Levels • Course • Program • Institution

9. GE and College-Wide Outcome Examples

10. Bloom’s Taxonomy

11. General Education Competencies • Focus on what students can do • Should be widely distributed • Should be known by major stakeholders • Guide course and curriculum planning • Focus assessment efforts and conversations about learning

12. Aspirational Outcomes vs. Learning Outcomes

13. Time to Refine Your Learning Outcomes

14. The Cohesive Curriculum • Coherence • Synthesizing Experiences • Ongoing Practice of Learned Skills • Increasing Sophistication and Application

15. Curriculum Map • I = outcomes are introduced at the basic level • D = students are given opportunities to practice, learn more about, and receive feedback to develop more sophistication • M = students demonstrate mastery at a level appropriate for graduation Is this a cohesive curriculum?

16. Entries on the Map Indicate: • Course-learning outcomes align with relevant program-learning outcomes • Assessment evidence related to the program learning outcomes is readily available in this course

17. The Curriculum Map • Focuses faculty on curriculum cohesion • Guides course planning • Allows faculty to identify potential sources of embedded assessment data • Allows faculty to identify where they might close the loop

18. Time to Draft Your Column of the Curriculum Map

19. Ensuring GE Curriculum Alignment

20. Assessment Plan • Who? • What? • When? • Where? • How?

21. We don’t have to assess every outcome in every student every year!

22. Sampling • Relevant Samples • Representative Samples • Reasonably-Sized Samples

23. Ethical Issues • Anonymity • Confidentiality • Informed Consent • Privacy

24. Properties of Good Assessment Techniques • Valid • Reliable • Actionable • Efficient and cost-effective • Engage respondents • Interest us • Triangulation

25. Embedded Assessment • Exams or parts of exams • Homework assignments and projects • In-class presentations • Group projects • Community-service learning and other fieldwork activities • Student recitals and exhibitions

26. Signature and Template Assignments

27. Indirect AssessmentStrategies • Surveys • Interviews • Focus Groups

28. Rubrics • Holistic rubrics • Analytic rubrics

29. Rubric Examples

30. Rubric Strengths • Complex products or behaviors can be examined efficiently. • Developing a rubric helps to precisely define faculty expectations. • Well-trained reviewers apply the same criteria and standards. • Rubrics are criterion-referenced, rather than norm-referenced. • Ratings can be done by faculty or others.

31. Using Rubrics for Grading and Assessment • Faculty can adapt assessment rubrics for grading. • Faculty control their own grades. • Faculty collaborate on assessment rubrics. • Rubric categories are used for assessment; numbers may be used for grading.

32. Assessment vs. Grading Concerns • Grading rubrics may include extra criteria. • Grading requires more precision. • Calibration

33. Rubrics Can: • Speed up grading • Clarify expectations • Reduce grade complaints • Improve reliability and validity • Focus faculty on important dimensions • Help faculty create better assignments

34. Using Rubrics in Courses 1. Hand out rubric with assignment. 2. Use rubric for grading. 3. Develop rubric with students. 4. Students apply rubric to examples. 5. Peer feedback using rubric. 6. Self-assessment using rubric.

35. Creating a Rubric • Adapt another’s rubric • Analytic approach

36. Drafting the Rubric

37. Managing Group Readings • One reader/document • Two independent readers/document • Paired readers

38. Before Inviting Colleagues • Develop and pilot test the rubric. • Select exemplars. • Consider pre-programming a spreadsheet for data collection.

39. Inter-Rater Reliability • Correlation between readers • Discrepancy index

40. Rubric Orientation and Calibration

41. Can we agree on a common set of rubric categories?

42. Time to Draft an Assessment Plan and Rubric for Your Outcome • Direct evidence • Rubric • Indirect evidence

43. Assessment Standards • How good is good enough?

44. What would be a reasonable standard for your outcome?

45. Closing the Loop • Celebrate! • Change pedagogy • Change curriculum • Change student support • Change faculty support • Change in equipment/supplies/space

46. Some Friendly Suggestions • Focus on what is important. • Don’t try to do too much at once. • Take samples. • Pilot test procedures. • Use rubrics. • Close the loop. • Include adjunct faculty. • Keep a written record.

47. Plans for Our Next Meeting: Tomorrow Afternoon 1:00-4:00 • Share our drafts. • Develop a three-year assessment plan. • Develop an action plan for this year.