How to Develop and/or Update A Simple and Clear Assessment Plan

1. How to Develop and/or Update A Simple and Clear Assessment Plan Office of Academic Program Assessment Spring 2013

2. Overview • Introduction • Definition of Basic Terms • Differences Between Goals and Outcomes • Criteria for Evaluating Goals and Outcomes • Using Rubrics in PLOs • The Next Step: Developing the Assessment Plan

3. Handouts • PowerPoint slides • WASC Rubric for Assessing the Quality of Academic Program Learning Outcomes • Examples of Goals and Outcomes • Curriculum map (Sociology) • Rubric (BA Nutrition and Food) • VALUE Rubric on Critical Thinking

4. Learning Goal “a high-level, very general statement of learning expected of graduates, aligned with the institution’s mission, vision, and values (more specific learning outcomes are derived from goals)”(WASC Draft 2013 Handbook of Accreditation, p. 45)

5. Learning Outcome “a concise statement of what the student should know or be able to do. Well-articulated learning outcomes describe how a student can demonstrate the desired outcome; verbs such as “understand” or “appreciate” are avoided in favor of observable actions, e.g., “identify,”“analyze.” Learning outcomes can be formulated for different levels of aggregation and analysis. Student learning outcomes are commonly abbreviated as SLOs, course learning outcomes as CLOs, program learning outcomes as PLOs, and institution-level outcomes as ILOs.”(WASC Draft 2013 Handbook of Accreditation, p. 48)

6. Assessment Plan Assessment—an ongoing, iterative process consisting of four basic steps: • defining learning outcomes • choosing a method or approach and then using it to gather evidence of learning • analyzing and interpreting the evidence • using this information to improve student learning (WASC Draft 2013 Handbook of Accreditation, p. 40)

7. Qualities of Goals(adapted from Graves, 2000) • General but not vague • Transparent to all stakeholders (no jargon) • Realistic • Simple—usually not multifaceted • Something the program addresses • Stated in terms of the learner

8. Qualities of Outcomes(adapted from Graves, 2000) • More specific than goals • Directly relate to goals • hierarchical relationship • cause-effect relationship • Focus on what students learn and/or associated processes • More objectives than goals • Stated in terms of the learner • Transparent to all stakeholders

9. More Qualities of Outcomes(WASC Rubric for Assessing the Quality of Academic Program Learning Outcomes) • Complete & well-organized • Both discipline-specific and institution-wide outcomes • Clearly distinguish between levels (e.g., undergraduate versus graduate) • Have explicit criteria statements developed by faculty (e.g., rubrics)

10. Example: BA General Communication GOAL: • Students will communicate effectively in a variety of contexts SUB-GOALS: • Demonstrate proficiency in oral communication • Demonstrate proficiency in written communication

11. Example: BA General Communication Learning Outcomes for Sub-Goal 1(Demonstrate proficiency in oral communication) • determine presentation needs in different situations • correctly use visual aids • make appropriate language choices • use proper structure • effectively deliver presentations

12. Example: BA General Communication Learning Outcomes for Sub-Goal 2 (Demonstrate proficiency in written communication) • implement a variety of style sheets • use thesis statements • use appropriate organizational strategies • apply transitions • include appropriate evidentiary support material • employ grammar conventions

13. The Importance of Verbs(Mager, 1975, cited in Brown, 1995) Multiple Interpretations Fewer Interpretations to write to recite to identify to sort to solve to construct to build to compare to contrast • to know • to understand • to really understand • to appreciate • to fully appreciate • to grasp the significance of • to enjoy • to believe • to have faith in

14. Criteria for Evaluating Goals Have program goals been specified? • Is each goal comprehensible? • Is each goal justified? • represents institutional goals • represents what faculty and/or discipline feel to be important • Is each goal stated in terms of the learner? • Will all goals actually be addressed in the program? • Can all goals be achieved by students in the program?

15. Criteria for Evaluating Outcomes Have program learning outcomes (PLOs) been developed? • Does the program specify clear PLOs? • Are there distinct differences in the level of specificity between program goals and program learning outcomes? • How will achieving program learning outcomes help to reach program goals? • Are PLOs achievable within the curriculum?

16. Criteria for Evaluating Outcomes • Have each of the following been considered in developing PLOs?: • Do PLOs focus on the key knowledge, skills and values students learn in the program? • Are the verbs in PLOs clear and unambiguous? Do they describe the observable behaviors by which students will demonstrate learning? • Are the criteria for performance on each PLO explicit and transparent? (Are there benchmarks?) • How and where can you observe each PLO? • What expectations do you have for student achievement for each PLO?

17. Rubrics for PLO Specification • Especially for skills-based outcomes • Rubrics allow faculty to agree on explicit criteria statements that • describe observable behaviors • differentiate levels of performance • e.g. , Cultural and global awareness/sensitivity including demonstrated understanding, respect and • support of multiple perspectives from other disciplines, societies, individuals, groups, and cultures

18. Example Rubric BA in Nutrition and Food Awareness of the integration of the different concentrations in FACS and their importance to the relationships between humans and their diverse environments as individuals and groups as a whole.

19. Rubric Development • Time-consuming • Look at existing rubrics e.g., VALUE rubrics can be used ... • without change • with revisions • as a heuristic

20. Group Discussion • Your program includes “critical thinking” as a learning goal and has decided to assess it this year. You have not yet specified PLOs or criteria for this goal. • Look at the VALUE rubric • How useful would this rubric be to your program for establishing PLOs and explicit criteria? • Choose one dimension. How would you need to revise the rubric to represent how that dimension is realized in your program?

21. Debrief • What did you learn about the process of specifying PLOs and criteria from doing this? • What did you learn from trying to adapt the rubric?

22. The Next Step:Sustainable Assessment Plans Good Assessment Plans: • Focus on the program (e.g., majors and concentrations) rather than courses • Have a limited number of goals and outcomes • Are developed collaboratively • Are ongoing • Are manageable • Use multiple methods of data collection • Anticipate how results will be used and lead to improvement

23. Comprehensive Assessment Planning • Include both content area knowledge and skills/competencies • Multi-year plan with 1-2 outcomes assessed each year • Direct and indirect methods • Capstone student projects (direct) • Student exit surveys: self-reporting on all outcomes

24. Writing an effective assessment Plan From UH-Manoa Assessment Template: http://http://manoa.hawaii.edu/assessment/resources/templates.htm

25. The Next Step:Sustainable Assessment Plans What should be assessed?? • IPP definition of Programs • What appears on the diploma • Undergraduate majors/concentrations • Graduate majors/concentrations • Do we have to assess all of these separately??

26. The Next Step:Sustainable Assessment Plans Issues • too many outcomes, not enough time • too many programs, not enough time

27. Too Many Programs—Not Enough Time An Example: Biological Sciences Undergraduate programs • 1,449 students (excluding pre-major) • BA Biological Sciences • BS Biological Sciences (7 concentrations) • Common curriculum for all concentrations: two LD and two UD courses How can we assess so many programs??

28. Learning Goals/Outcomes • Students will demonstrate the ability to communicate in the Biological Sciences • Written Communication • Oral Communication • Information Literacy • Students will demonstrate the ability to apply the process of science • Critical Thinking • Quantitative Reasoning

29. BA & BS-General: Curriculum Map

30. The problem • Shared courses: Bio 1, 2, 100, 184 • Other courses: problematic! • Bio 188: Evolution • Required by 4 of 7 concentrations • Will capture 70% of students

31. Five Year Assessment Plan

32. For more information • Please contact Office of Academic Program Assessment • Amy Liu (amyliuus@hotmail.com) • Julian Heather (jheather@csus.edu) • Shannon Datwyler (datwyler@csus.edu) • Elizabeth Strasser (strasser@csus.edu)