Student Learning Outcomes Assessment More on Measures

1. Student Learning Outcomes Assessment More on Measures Assessment Brown Bag Session 2 March 16-17, 2009

2. Developing Measures How will we know if students have learned what we wanted them to learn or have achieved the skills we wanted them to achieve?

3. Indicators What behaviors or performances will indicate that students have achieved a certain outcome?

4. Middlestates Guidelines • Two valid and reliable indicators per outcome • At least one direct measure • Indirect measures • Specific criteria outlined for each indicator • Rubrics and/or rating scales

5. Appropriate Measures • Course-embedded assessment measures • Existing student work products/performances • Professional organizations

6. Criteria/Standards What level of student performance will we accept as evidence of course or program success?

7. Suskie on Setting Benchmarks or Standards • Midterm class averages was 55 points. • Did the class do well on the midterm? • Need to compare 55 to something else • The “something else” depends on what we want the test to tell us Suskie 2008

8. Five Ways to View Success • Standards-based perspective • Are your students meeting your standards? • Benchmarking perspective • How do your students compare to peers? • Best-practice perspective • How do your students compare to the best of their peers? • Value-added perspective • Are your students improving? • Longitudinal perspective • Is your program improving?

9. Suppose 35 is passing and 80 is a perfect score • Local standards • Aka competency-based • Aka criterion-referenced • Question answered: • Are our students meeting our standards? • Challenge: • Establishing sound performance standards Suskie 2008

10. Suppose 35 is passing and 80 is a perfect score on a published exam • External standards • Aka competency-based • Aka criterion-referenced • Question answered: • Are our students meeting external standards? • Challenge: • Do the standards match what we think is important? Suskie 2008

11. Suppose the class average is 65 • Internal peer benchmark • Aka norm-referenced • Question answered: • How do our students compare to peers in their class, program, or institution? • Challenge: • Identifying appropriate peers and collecting information from them Suskie 2008

12. Suppose the national average is 70 • External peer benchmark • Aka norm-referenced • Question answered: • How do our students compare to peers at other colleges? • Challenge: • Identifying appropriate peers and collecting information from them Suskie 2008

13. Suppose class average was 25 a year ago • Value-added benchmark • Aka grown, change, pre-post • Question answered: • Are our students improving? • Challenge: • Transfers in and out • Motivating students on pre-test • Is growth due to us? • Imprecise assessments mask growth • Is this question relevant? Suskie 2008

14. Suppose class average is 65 now and 40 three years ago • Historical trends benchmark • Aka improvement • Aka (incorrectly) longitudinal • Question answered: • Are our teaching and curricula improving? • Are we getting better • Challenge: • Using the same assessment Suskie 2008

15. Suppose class average is 65 for knowledge and 45 for real-world application • Strengths and weaknesses perspective • Question answered: • What are our students’ relative strengths and areas for improvement? • Challenge: • Getting “sub-scores” that are truly comparable Suskie 2008