Assessing Essential Learning Outcomes with the “VALUE” METARUBRICs

1. Assessing Essential Learning Outcomes with the “VALUE” METARUBRICs Using Rubrics for Teaching, Learning, & Assessment

2. Introductions • Office of Academic Assessment 101 Hillcrest Hall (0157) • Ray Van Dyke, 231-6003, rvandyke@vt.edu • Steve Culver, 231-4581, sculver@vt.edu • Kate Drezek, 231-7534, kmdrezek@vt.edu • Others here today

3. Today’s agenda • Rubrics as an assessment tool • Definition and Overview of Rubrics • Using Rubrics for Program Assessment • Pros/Challenges • VALUE METARUBRICS – Assessment toolkit • Background • Purpose • Application at Virginia Tech

4. Overview: What is Assessment of Learning Outcomes? • “Assessment of student learning is the systematic gathering of information about student learning, using the time, resources, and expertise available, in order to improve the learning.” – Walvoord • A student learning outcome states a specific skill/ability, knowledge, or belief/attitudestudents are expected to achieve through a course, program, or college experience. • Example: Upon completion of a B.A. degree in English, a student will be able to read critically and compose an effective analysis of a literary text.

5. What is The Process for Assessing Student Learning Outcomes?

6. Rubric as an Assessment Tool • A rubric is an explicit summary of the criteria for assessing a particular piece of student work, plus levels of potential achievement for each criterion.  • Not only can rubrics can be used to assess an individual student’s performance for a course grade, rubrics can be used to assess the achievement of course, program, or college/ institutional-level student learning Outcomes

7. Rubric as an assessment tool • To the extent that a given student assessment measures or relates to a programmatic outcome, it can potentially be aggregated and used for program assessment (course-embedded assessment) • The same student product (e.g., essay, capstone project) is used twice: once for individual student assessment and again for program

8. Rubric as an assessment tool • Pros • Course-embedded assessment helps to overcome student motivation problem • Inherent efficiencies – “two birds, one stone” • Authenticity of student learning “artifact”

9. Rubric as an assessment tool • Challenge – the rubric! • Think-pair-share • Take a moment and jot down as many challenges associated with rubrics as you can • Pair up with a neighbor and share your list

10. Rubric as an assessment tool • Challenge – the rubric! Needs to be well-designed • Time – to create, to test, to implement, to assess • Reliability/validity – needs to be well-designed

11. Well-designed rubrics increase an assessment’s reliability by setting criteria that raters can apply consistently and objectively. Well-designed rubrics increase an assessment’s construct and content validity by aligning evaluation criteria to standards, curriculum, and instruction tasks. Rubric as an assessment tool

12. If, in theory, rubrics can be a useful assessment tool… AAC&U’s VALUE METARUBRICS ARE A TANGIBLE “TOOL KIT” TO FACILITATE THE APPROPRIATE AND MEANINGFUL ASSESSMENT OF THE ESSENTIAL LEARNING OUTCOMES THAT ARE THE HALLMARK OF A 21st CENTURY LIBERAL EDUCATION

13. Assessing essential learning outcomes VALUE METARUBRICS: Assessment toolkit

14. VALUE METARUBRICS: Assessment toolkit Background • AAC&U • Organization • role in assessment, accountability • “LEAP” project – Liberal Education & America’s Promise

15. VALUE METARUBRICS: Assessment toolkit Liberal Education is an approach to learning that: • Empowers individuals and prepares them to deal with complexity, diversity, and change. • Provides students with broad knowledge of the wider world (e.g. science, culture, and society) as well as in-depth study in a specific area of interest. • Helps students develop a sense of social responsibility, as well as strong and transferable intellectual and practical skills such as communication, analytical and problem-solving skills, and a demonstrated ability to apply knowledge and skills in real-world settings • usually includes a general education curriculum that provides broad learning in multiple disciplines and ways of knowing, along with more in-depth study in a major. http://www.aacu.org/leap/What_is_liberal_education.cfm

16. VALUE METARUBRICS: Assessment toolkit AAC&U's LEAP Campaign has defined a robust set of "Essential Learning Outcomes" that students develop during an excellent contemporary liberal education. Beginning in school, and continuing at successively higher levels across their college studies, students should prepare for twenty-first-century challenges by gaining all these outcomes http://www.aacu.org/leap/What_is_liberal_education.cfm

17. VALUE METARUBRICS: Assessment toolkit VALUE project: PURPOSE • Valid Assessment of Learning in Undergraduate Education • seeks to contribute to the national dialogue on assessment of college student learning. • builds on a philosophy of learning assessment that privileges multiple expert judgments of the quality of student work over reliance on standardized tests administered to samples of students outside of their required courses. • The assessment approaches that VALUE advances are based on the shared understanding of faculty and academic professionals on campuses from across the country. http://www.aacu.org/value/

18. VALUE METARUBRICS: Assessment toolkit VALUE project: ASSUMPTIONS • to achieve a high-quality education for all students, valid assessment data are needed to guide planning, teaching, and improvement; • colleges and universities seek to foster and assess numerous essential learning outcomes beyond those addressed by currently available standardized tests; • learning develops over time and should become more complex and sophisticated as students move through their curricular and co-curricular educational pathways toward a degree; http://www.aacu.org/value/

19. VALUE METARUBRICS: Assessment toolkit VALUE project: ASSUMPTIONS • good practice in assessment requires multiple assessments, over time; • well-planned electronic portfolios provide opportunities to collect data from multiple assessments across a broad range of learning outcomes while guiding student learning and building self-assessment capabilities; • e-portfolios and assessment of work in them can inform programs and institutions on progress in achieving expected goals. http://www.aacu.org/leap/What_is_liberal_education.cfm

20. VALUE METARUBRICS: Assessment toolkit VALUE project: Connections to VT • Essential Learning Outcomes • Incorporated into purpose statement for cle: http://www.cle.prov.vt.edu/purpose.html • Charge to QEP from Provost • Alignment with SCHEV Core Competencies • VT “Partner Institution” in VALUE project http://www.aacu.org/value/

21. VALUE METARUBRICS: Assessment toolkit VALUE project: Connections to VT • VT “Partner Institution” for Value Project • Opportunity to use, provide feedback, shape rubrics • Join Google Group with access to rubrics as they are revised • Final rubrics – fall 2009 http://www.aacu.org/value/

22. VALUE METARUBRICS: Assessment toolkit VALUE project: Why META????? • Best thinking of higher ed faculty professionals • Specific enough to be useful, broad enough to be useful, meaningful in multiple contexts, at multiple levels for assessment http://www.aacu.org/value/

23. To the METARUBRICS! • Civic engagement • Creative thinking • Critical thinking • Ethical reasoning • Foundations and skills for lifelong learning • Information literacy • Inquiry and analysis • Integrative learning • Intercultural knowledge and competence • Oral communication • Problem solving • Quantitative reasoning • Teamwork • Written communication

24. To the METARUBRICS! • Identify learning outcome(s) of interest to program? • Which essential learning outcome(s) are in alignment with program goals/outcomes? • Which rubric(s)/elements of the rubric(s) can be incorporated into assessment efforts?

25. The Big Message • Avoids Re-Inventing the wheel, staring at a blank page • Potential connections to other campuses, best practices for using metarubrics • More sophisticated, authentic approach to capturing information on our students’ learning

26. Rubrics help us to Reflect Before we React to our work “We are being pummeled by a deluge of data and unless we create time and spaces in which to reflect, we will be left with only our reactions.” – Rebecca Blood

27. Questions/Comments? (Please take a Few minutes to complete the session feedback form. Thanks!)