250 likes | 269 Views
Good Rubric, Bad Rubric What ’ s the Difference?. Kathleen Norris, MFA, EdD Plymouth State University. Good, or Emerging?. Rubrics are a phenomenon of the profession of education.
E N D
Good Rubric, Bad RubricWhat’s the Difference? Kathleen Norris, MFA, EdD Plymouth State University
Good, or Emerging? Rubrics are a phenomenon of the profession of education. They have been embraced by some educators, tolerated by others, and loathed by some. Some educators are afraid of them, or don’t understand them.
Good Rubrics • Do what you want them to do, assess what you want to assess. • Are pleasing to look at, are easy on the eyes, are clear to everyone, are not too big or small for their purpose. • Are reliable, valid, fair, and completely connected to what you are assessing. • Everybody understands the same thing when they read one, but they can still be scary.
Emerging Rubrics • Assess something, but not necessarily what you want them to assess. • Could be clearer, easier on the eyes; might need to be smaller or larger. • Are not completely connected to the standards, the performance (task), or both, so are not reliable, valid, or fair. • Not everyone understands them the same way and they can be very scary.
Authentic Assessment “A move toward more authentic tasks and outcomes thus improves teaching and learning: students have greater clarity about their obligations (and are asked to master more engaging tasks), and teachers can come to believe that assessment results are both meaningful and useful for improving instruction.” Wiggins, Grant (1990). The case for authentic assessment. Practical Assessment, Research & Evaluation, 2(2). Retrieved January 22, 2007 from http://PAREonline.net/getvn.asp?v=2&n=2.
Good Rubrics • Assess the performance of a student in the completion of a task. • Complement, but do not completely replace, traditional assessments. • Communicate expectations clearly. • Communicate the level of success a student is having. • Inform future instruction.
Rubric Development • Questions that need to be answered, and should be answered collaboratively: • What should the students know and be able to do? (These are the standards.) • What products indicate that students have met these standards? (Describe the authentic task(s) or project(s) that will be scored.)
Rubric Development • What does “perfect” performance on these tasks, or a “perfect” project look like, and what does an unsuccessful project look like? (These are the top and lowest score criteria for judgment.) • To communicate the expectations for the task, and, in the end, how well students performed, a rubric is developed that contains • the elements of the task or project that meet the established standards, • the criteria for the evaluation of each element, and • ultimately the student’s “score” on each element.
Good Rubric Development • Collaboration is essential in this work. • Define the standards to be met. • Describe the task(s) or project(s) to be completed. • Describe the criteria for judging the task or project on each required element. • Set the scale from “perfect” to unsuccessful on each required element.
Designing the Rubric • Creating a table with scores across the top and criteria down the side and language within each box is easy, but you might be creating an “emerging dog” instead of your desired, and good, “dog.” • The language in each of the boxes is the key to a “good rubric.” • Rubrics that are “emerging” can be rehabilitated and become “good” ones!
Language Within a Rubric • Each “box” will have a description of what that score value means relative to the criteria. • The descriptions will be written in language that is appropriate for the grade level of the students and that avoids any jargon or language that would be unclear to them or their parents.
Language Within a Rubric • The language will be agreed upon by the developers of the rubric, at a minimum. Best practice would be to have at least a sample of the students and parents involved in revising a draft of the rubric to raise any questions of clarity about standards, criteria or the task, and to assist in revising the language before it becomes the “final” draft.
Testing a Rubric • The “final draft” of a new rubric should be “tested” by having a sample of tasks provided to readers for scoring on the rubric. • Ideally, teachers, students and a sample of parents would be involved in this, but at a minimum the teachers would have collaborated on this rubric.
Validity and Reliability • The rubric can be considered valid if the different readers assign the same score values for each sample, • and it can be considered reliable if a set of “perfect” projects are all given “perfect” scores by the various readers, and a set of unacceptable projects also get unacceptable scores by the readers, etc.
Fairness • The task, or project, assigned must be designed as an authentic assessment of how well students have achieved the standards. • The students should be given the rubric with the assignment, and should be taught how to use the rubric to create their best work.
Fairness • The rubric must be applied equally to all tasks or projects; the identity of the student should not affect the scoring of the project. • The connection of the rubric to a final “grade” on the assignment must be made clear from the beginning.
FAQ’s • Choose to run scores from left to right, or right to left? How big should the matrix be? • We naturally read English documents from left to right. Some suggest starting on the left with the “perfect” score column so students see that first, but this can feel awkward to traditional learners. • The matrix should contain no more than 6-7 score points and no fewer than 3. Five seems to be enough for clarity for most rubrics. You can put +/- signs in between the columns if you need finer discrimination in scoring. • The number of criteria should be the minimum necessary to assess the project or task.
FAQ’s • Can a good rubric be created by a person all on their own? What about all the “rubric maker” software programs? • The best rubrics are developed and tested collaboratively. • If there are predefined standards and established criteria, an individual can create a good rubric. There are many free online rubric creating programs as well as websites and books with sample rubrics that can be adjusted. • The best rubrics are tested for validity and reliability and this requires collaboration. Authentic assessment in general will be used more effectively to inform instruction when teachers collaborate on the testing of rubrics.
FAQ’s • If you don’t have a collection of samples to use to test the rubric, how can you be sure that it’s reliable and valid? • You really can’t be sure. • If you have a “new” task or project then the rubric is being “tested” with it as the students complete the project or task. The “untested” rubric will have to be handed out with the assignment, but even at that stage it can be improved based on student feedback. • The particular challenge with reliability and validity comes with getting inter-rater agreement on the scoring and you need to have samples and more than one reader to establish the “goodness” of the rubric.
FAQ’s • Should the points given “add up” to a grade, or be averaged for a grade? • This really depends on the task or project and what the grade is meant to represent. If the task itself is getting a grade, then the rubric can be scored to produce a grade. However, many tasks are part of a larger graded unit, so there might be a rubric created for the group of assignments that all work together to produce a grade. • A writing assignment, for example, can be scored on a rubric for the quality of the writing, but there may be a number of elements to the written piece that must be included in order for the paper to be successful at meeting content standards that are outside of actual writing standards.
Rubrics • They are an integral part of the authentic assessment of worthy intellectual tasks and projects. • They can be used effectively to communicate with students and parents about what’s expected and how well a student is meeting those expectations. • They can inform instruction more effectively than traditional assessments.
Good Rubrics • They can be time consuming to produce, but valid, reliable and fair rubrics will last for as long as the standards are assessed by a task or project. The criteria are based on the standards, so they can be adjusted, if necessary, for different authentic assignments, without being rewritten. • They need to be introduced systematically for the educators, students and parents who may be “afraid” of them. Don’t let them “jump up.”
Student Voices; What’s Best • "In the end they help make things fair so that a student can't complain about his grade, and it also gives the teacher back up for the grade he/she gave. A student can also use it for them if they feel they have been unfairly graded.” • "I think rubrics can help teachers grade better than just thinking of a grade in their heads.” • "I like the way they word the scores and also how they show you to get that score. The guidelines are right down to the point and are precise to what you need to get that score. Transforming Middle Schools and School Systems: What students say about rubrics.http://www.middleweb.com/CSLB2CubRub.html
Thank You As you implement best practices in Curriculum, Instruction and Assessment, you’ll find that the wise use of good rubrics will contribute greatly to the improvement of instruction and measurable gains in student learning. Take the time you need to build good rubrics!