Module 2

1. Module 2 Norming Expectations and Assessments for Student Learning 2.15.05

4. NB —— note well Separating -student learning outcomes from -student learning outcomes assessment -wastes time, -confuses the coherence of programs, and -looses opportunities for collaboration and collective norming

5. Step by Step: Improving Student Learning

6. Step 1: Dialogue • Dialogue about a department’s philosophy and assumptions of teaching and learning should guide the choice of student learning outcomes: • A developmental philosophy would set increments of increasingly complex SLOs • A problem-based learning philosophy would implement complex, “messy” learning outcomes in every course

7. ACTIVITY: DIALOGUE1 1. What are the philosophies or principles about teaching and learning on which we’ve built our curriculum, instructional design, pedagogy, or use of educational tools? 2. What methods of teaching or educational experiences develop the knowledge, understanding, habits of the mind, ways of knowing, and problem solving that this discipline, profession, trade, program or institution values? 3. How do we impart those ways of thinking, knowing and problem solving to students? 4. Which students benefit from which teaching strategies, educational processes, or educational experiences? 1Maki, P.L. (2004). Assessing for Learning. American Association for Higher Education, Sterling, VA: Stylus.

8. On poster paper • Write at least one response to each bulleted question.

9. Step 2: Norm Expectations • Faculty discuss and debate what students should know and be able to demonstrate upon completion of one course or the program. • Expectations are collaboratively authored and collectively accepted. Full- and adjunct/ part-time faculty who teach a course come to consensus. 1Maki, P.L. (2004). Assessing for Learning. American Association for Higher Education, Sterling, VA: Stylus.

10. ACTIVITY: DIALOGUE ABOUT EXPECTATIONS • What are the critical minimum learning/ performance outcomes for students in this course? • What do students need to progress to the next course in this sequence? (Do students usually take more courses in this sequence?) • What outcomes do students need to meet their own goals (vocational certificate, AAS, transfer?) • What outcomes has institution defined?

11. On poster paper • Write at least one response to each bulleted question.

12. Collectively Set Expectations for a Course EXPECTATIONS • use active verbs that identify student performance: create, apply, construct, formulate • align with desired outcomes for program or degree • align with disciplinary philosophy of learning • are collaboratively authored and collectively accepted • incorporate professional, licensure, or standardized outcomes • can be assessed qualitatively or quantitatively

13. Step 3: Norm and Implement Assessment(s) • Align assessment procedures with programmatic philosophy/ assumptions about learning- What are the values of program? • Collectively identify evidence/ criteria/ performance indicators/ primary traits • Collectively set rubric for scoring

14. Assessment should fit the nature of a program’s goals using methods which reflect the type of learning that is valued, rather than methods which are most easily constructed or scored. For example, if courses emphasize open-ended problems, the assessment procedure should emphasize the same objective. Facing students with a multiple-choice test to measure open-ended problem solving would be inappropriate. <www.maa.org/past/ql/ql_part4.html>

15. Using Criteria/ Standards/ Traits • Share with students • Consistently grade each student’s work according to criteria and rubric • Guide students in self-use of criteria • Collect and analyze outcomes • Use outcomes to improve teaching • Share learning activities, outcomes, assessments, criteria, rubrics with other faculty who teach this lesson 1Adapted from Scroggins, B. (2004). Targeting Student Learning. Modesto,CA: Modesto Junior College.

16. BUILDING A RUBRIC1 1Adapted from Scroggins, B. (2004). Targeting Student Learning. Modesto ,CA: Modesto Junior College

17. ACTIVITY: MORE DIALOGUE1 What do faculty do now? • How do you determine an A from a B from a C; acceptable from not acceptable? (“I know it when you see it” versus written criteria for grading every assignment? • Are your criteria easy enough to use that you use them every time? (consistent use) • Do you share criteria and rubrics with students before assignments? Do you encourage students to self-assess their work using the criteria and rubrics? • How do you use the results of assessment, of criteria to improve your assessment methods, criteria, and/or rubrics? 1Adapted from Scroggins, B. (2004). Targeting Student Learning. Modesto,CA: Modesto Junior College

18. REVIEW: Thinking through SLOs • What outcomes do you want for your students? • From whom do you want these outcomes? (targeted ~ all) • When do you want these outcomes? (entry — middle — exit) • What evidence will be credible and agreed upon by faculty? (assessment method) • What criteria is acceptable? (criteria)

19. ACTIVITY: NORM SLOs • DRAFT SLOs components (columns), using one of the formats offered or making one to meet your local needs.

20. Step 4: Compile and share results • Gather results/ data across sections of a course and/or across courses within a program and/or across courses in a sequence • Share and discuss results; identify gaps in student learning; discuss effectiveness of current teaching and assessment methods • Use accessible terminology and formats for reporting data

21. Analyzing Results • Plotting students’ achievement along agreed-upon criteria/ standards/ traits focuses attention on patterns of strengths and weaknesses • Visually presented using comparative tables focuses attention on why some groups do well while others do not • Focus attention on WHY?

22. ACTIVITY: COMPILING RESULTS

23. Identifying Improvement Strategies

25. Collecting data ≠ Improvement in student learning outcomes

26. Step 5: Collectively set improvement plan • Collectively identify patterns of weakness in student learning • Allow time to reach consensus about “Why” and about changes/ improvements to address gaps • Collectively distribute specific tasks and locations for piloting improvements

27. Potential improvement areas • Improvements in instruction • Improvements in curriculum/ content • Improvements in student support • Changes in motivation and engagement of students???

28. Instructional Improvement • More active forms of learning, including projects, applications, simulations • Decreased use of lecturing • Alignment between values/ desired goals of program and pedagogy/ assessment procedures • • may also improve motivation and engagement of students

29. ACTIVITY: IMPROVING INSTRUCTION • How does your institution focus on instructional skills of faculty now? • Are there opportunities for faculty to observe one another, team teach, video and review their own teaching, or “practice” effective pedagogy in other ways? • What is your institutional culture surrounding teaching? (I never hear chalk anymore; I hear lots of chairs moving; I don’t see instructors in front of the room, etc) • What role does pedagogy play in hiring and personnel evaluation at your campus?

30. On poster paper • Write at least one response for each question.

31. Curricular Improvement • Interdisciplinary connections (increases relevance) • Learning communities of linked courses (increases “community”) • Infusion of applications into liberal arts or infusion of math/communication skills into career-technical courses • • interdisciplinary connections may improve student motivation and engagement

32. Appropriate improvements • Students don’t connect what they learn in one course with content/ concepts of another course: link courses • Developmental students don’t connect learning activities with their own goals: contextual courses • Even students who are placed into math or English courses can’t do the required work: re-evaluate placement instrument • Technical students don’t have necessary basic skills: contextual courses; infusion of academic into technical content

33. ACTIVITY: CURRICULAR IMPROVEMENT • Describe previous curriculum reform endeavors on your campus. What gaps in student learning were these changes to address? What data did you collect and analyze? How effective was/ were those curriculum reform(s)? • Which curriculum improvements seem promising to meet the learning gaps you’ve identified?

34. On poster paper • Write at least one response for each question.

35. Student services improvement • Student services decentralized: connected to disciplines, departments • Student services centralized: one stop shop • Student services faculty engaged in teaching • How does student services contribute to student learning outcomes?

36. ACTIVITY: IMPROVING LEARNING VIA STUDENT SERVICES • Describe previous reforms in student services on your campus. • What gaps in student learning were these changes to address? What data did you collect and analyze? How effective was/ were those student services reform(s)? • Which student services revisions seem promising?

37. On poster paper • Write at least one response for each question.

38. Improvement in Student Motivation and Engagement • Difficult to address directly • Usually results from improvements in instruction, curriculum or student services • • Many efforts to “fix” students contradict the CCC mission

39. Moving from data to improvement • Where will your campus target its initial efforts? • target population • content area • program • Which strategy(ies) are aligned with those improvement goals? • Which strategy(ies) appear to offer the best cost/benefit?

40. Reporting Out • Prioritize your efforts. • Where will you put your first efforts to improve student learning: improvements in instruction, curriculum, student support, or?? • Why?

41. Evaluating Improvement Strategies

43. Making and Evaluating Improvement Strategies • Improvement strategies are aligned with student learning gaps • Baseline data provides comparison • Collaboratively author and collectively accept new outcomes, learning activities, curriculum/ student services reform • Collect data on effectiveness of improvement strategies

44. Step 6: Continue SLOA Cycle • Reassess how well students improve based on implemented changes • Periodically update assessment plans that build in a cycle of inquiry

45. Mapping Backwards from Outcomes to Learning Activities • Establish desired outcome • Then credible assessment • Then criteria/ primary traits • Then rubric for scoring • Then content/ learning activities that align with outcome and assessment

46. Confusion: Where to Start?? • Some colleges start with institutional outcomes: Gen Ed • Some colleges start with Perkins/ VTEA funded occupational programs • WHERE is not nearly as important as STARTING because this is continuous professional learning to improve student learning • Ready, Fire, Aim as you gain experience

47. Homework Report • How does Program Review compare to SLOACs? • What benefits/ disadvantages would there be for revising Program Review to incorporate some/all SLOAC data-driven decision making?

48. Norena Norton Badway, Ph.D. Principal Phone 209-951-7477 home office 209-946-2168 University office 209-601-7121 Email badway@aol.com nbadway@pacific.edu