Scoring Rubrics

1. Scoring Rubrics Margaret Kasimatis, PhD VP for Academic Planning & Effectiveness

2. What is a Scoring Rubric? • A scheme for evaluating student work along certain dimensions • Specific skills or aspects of a general learning outcome • Concrete descriptors of levels performance • Can be applied to a variety of student products or performances (e.g., written work, presentations, efforts, etc.) • Good for measuring higher-order skills or outcomes not easily measured by tests(e.g., oral communication, integration)

3. EXAMPLE OF ONE DIMENSION OF AN ORAL PRESENTATION SCORING RUBRIC

4. What is a Scoring Rubric? • How is this different from grading? • Multidimensional vs. holistic • Concerned more with skills than “right answer” • Results are aggregated across students

5. Constructing Rubrics • Select learning outcome or competency • Identify specific dimensions/skills • Develop concreted descriptors of levels of performance • Need to keep in mind the nature of the work product • Ask yourself, “What level of performance should a graduating senior in my department have?” • Best you can expect is highest rating • Unacceptable product is lowest rating • Worst acceptable product is lowest acceptable rating

6. EXAMPLE LEARNING OUTCOME:“Can…analyze and interpret data” Student Product: Lab Report

7. EXAMPLE LEARNING OUTCOME:“Understanding the impact of engineeringsolutions in a global and societal context” Student Product: Final Paper in Integrative Experience

8. Applying Rubrics • Ideal Case: • Train a pair of raters (from outside course or department) to use the rubric (test inter-rater reliability) • Each rater independently scores the work • This can be done “live” (e.g., with oral presentations), or copies of student products could be made and retained for later scoring

9. Using Rubrics for Program Assessment • Where in curriculum is outcome addressed and at what level? (refer to curriculum map) • Ideally, look for courses that “Introduce” as well as those that “Emphasize” (to test improvement across program) • Or look at required senior-level courses • Identify student work products/performances that should demonstrate the outcome • E.g., written assignments, research reports or posters, oral presentations, creative performances or products • If you want to test improvement, look for similar products in lower and upper division courses • Cautionary note about apples and oranges

10. Using Rubrics for Program Assessment • To analyze and interpret the data: • Average scores across raters (if you used two raters) • Aggregate those scores across students for each rubric dimension • Present data in user-friendly way and have discussion of what it means • It helps to already have a criterion/standard in mind

12. Advantages of Scoring Rubrics • Direct evidence of student learning • Good for measuring higher-order skills or evaluating complex tasks • Summaries of results can reveal patterns of student strengths and areas of concern • Can be unobtrusive to students • Can generate great discussions of student learning among faculty, especially regarding expectations

13. QUESTIONS?