Evaluating Student Work in a Standards-based Framework

1. Evaluating Student Work in a Standards-based Framework

2. In a good assessment task there is an effective interplay of three simple elements • What is taught/learnt (curriculum intent) • What is assessed (knowledge/skills/etc in the domain being sampled) • What is rewarded (performance on the criteria set down in the marking scheme) IMPLICATIONS FOR TEACHERS: unit plans are designed simultaneously with summative assessment tasks and their accompanying marking criteria

3. Types of assessment structures Rubrics or matrices: used to measure student performance against a pre-determined set of criteria Grading masters: similar to a rubric but contain the necessary features to make an holistic judgement from ‘rich’ or ‘authentic’ tasks. GTMJ or “guide to making judgments: a combination of the above two that is deigned to ensure greater alignment with curriculum

4. Essential elements of any assessment structure Criteria (‘what counts’): clear and explicit for the types of evidence students will produce (note: syllabus outcomes are not assessment criteria) Levels of performance/achievement Standards descriptors: sufficient and clear enough to provide advice to students (and other assessors) for making judgements

5. What to avoid in a grading tool Beware atomism and complicated matrices that result in • fine grained outcomes statements • marking schema with lots of cells containing too many superfluous words …as that tend to reduce the multidimensionality of complex tasks. Increasing the number of criteria and levels of achievement (LoA) increases error because judgements become finer.

6. Basic Rubric/Matrix CRITERIA LoA STANDARDS DESCIPTORS

7. Positive aspects of rubrics Rubrics help ensure that assessment is criteria-based and that standards are transparent and explicit and, the format is familiar for most intended audiences. However, the simplicity of the matrix format can disguise real complexities in its design and use

8. Problems with rubrics • The traditional matrix format requires the number of significant and discernible differences used in judging quality be the same for all criteria ‘manufacturing’ distinctions in quality where they may not exist obfuscating standards, biasing grades and making discussion of standards more difficult • formats require the ‘quantum’ of achievement between adjacent standards descriptors is also the same (or close to it), i.e. the gap between each LoA is the same results are biased if standards descriptors do not have this quantum property

9. Problems with rubrics (cont’d) 3. A matrix requires teachers to commit to a single description for each cell. This can be seen as counterproductive in that it tends to reduce the multidimensionality of complex tasks designed to measure a range of knowledges, skills and dispositions. Removing the matrix frees up the psychology of the assessment process and teacher’s interest shifts from the grade per se.

10. An Alternative Model THE GRADING MASTER

13. THE GRADING MASTER Standards descriptors are positioned along a continuum (‘pole’) – this allows the number of standards and their placements to vary for each criterion Teachers can plot a student’s performance on each of the criteria before making an on balance judgement to arrive at the overall grade (it is also possible give feedback to students and report on performances on each pole separately).

14. THE GRADING MASTER Is a preferred tool for making holistic judgements of students work. This is due to its fundamental structure: Each pole captures a desired quality of performance (criteria). Each pole represents a continuum of standards (and then assessors’ judgements) Positioning standards descriptors along a continuum allows the number of standards to vary from criterion to criterion , as can their relative placements.

15. USING THE GRADING MASTER Teacher-assessors plot a student’s performance on each of the criteria on the corresponding pole before making an on-balance judgement to arrive at the overall grade.

16. USING THE GRADING MASTER When making an ‘on-balance’ judgement teachers consider each pole but also look across the poles to ascertain the region or vicinity where the student work is best placed. They are not looking for a precise location, instead they are using the regions to reach a holistic judgement. The final consideration is then given to what grade to award the student work.

17. MAKING AN HOLISTIC JUDGEMENT • This is reached when teachers make judgements across several criteria, trading off inconsistencies. • They are looking for evidence of quality of performance, as they are manifested in the tasks, and then award a grade. • Teachers use their professional knowledge in decision making. They must be well informed of curriculumintent and the desirable intellectual strategies used in the completion of the task (pedagogy).

18. The next evolution THE GUIDE TO MAKING JUDGMENTS

20. SOCIAL MODERATION Apart from improving consistency in evaluating student work, social moderation enhances quality assurance of teacher built assessment tasks. Social moderation of student work increases teachers’ understanding of standards and provides a professional learning forum for designing and evaluating quality assessment tasks. .