Assessment Standards: A Manifesto for Change

Assessment Standards Knowledge exchange Assessment Standards:A Manifesto for Change Dr Chris Rust, Deputy Director ASKe CETL Directorate: Margaret Price, Jude Carroll, Berry O’Donovan and Chris Rust

Origin • Weston Manor Group, November 07 • 40 National and International Experts in Assessment • Triggered by the Burgess Report, NSS Results • Two days of discussions Outcome • Six tenet manifesto for change to assessment practice related to standards • Sent to HEFCE, HEA, QAA, UUK, GuildHE, NUS • Lead article in THES – April 24, 08

Why assessment ? Assessment: a key driver of student learning “Assessment is at the heart of the student experience” (Brown, S & Knight, P., 1994) “From our students’ point of view, assessment always defines the actual curriculum” (Ramsden, P.,1992) “Assessment defines what students regard as important, how they spend their time and how they come to see themselves as students and then as graduates.........If you want to change student learning then change the methods of assessment” (Brown, G et al, 1997)

But…… • QAA subject reviews • National Student Satisfaction Survey • “the Achilles’ heel of quality” (Knight 2002a, p. 107) • Summative assessment practices “in disarray” (Knight 2002b, p. 275 • “Broken” (Race 2003, p. 5) • “There is considerable scope for professional development in the area of assessment” (Yorke et al, 2000, p7) • Rising concern about cheating and plagiarism

Why change is needed (1) “The types of assessment we currently use do not promote conceptual understanding and do not encourage a deep approach to learning………Our means of assessing them seems to do little to encourage them to adopt anything other than a strategic or mechanical approach to their studies.” (Newstead 2002, p3) “Even when lecturers say that they want students to be creative and thoughtful, students often recognise that what is really necessary, or at least what is sufficient, is to memorise” (Gibbs, 1992, p. 10) Many research findings indicate a declining use of deep and contextual approaches to study as students’ progress through their degree programmes (Watkins & Hattie, 1985; Kember et al, 1997; Richardson, 2000; Zhang & Watkins, 2001)

Tenet 1 “The debate on standards needs to focus on how high standards of learning can be achieved through assessment. This requires a greater emphasis on assessment for learning rather than assessment of learning.”

Why change is needed (2) Our current systems focused on marks and grades aren’t working • Belief that it is possible to distinguish the quality of work to a precision of one percentage point (Elander & Hardman, 2002) • Belief that double-marking will ensure fairness and reliability (Laming (1990) • Belief that consistency can be achieved through conformity, and simple numerical rules (e.g. level 1 essay 3,000 words, level 3 essay 5,000; or no more than two pieces of assessment per module) • The combination of scores, which obscures the different types of learning outcome represented by the separate scores • The distortion of marks by the type of assessment (e.g. coursework c.f. examination) and the actual subject discipline/s studied (Yorke et al, 2002; Bridges et al, 2002) • The distortion of resulting degree classifications by the application of idiosyncratic institutional rules (e.g. Armstrong et al, 1998) (Rust, 2007)

Why change is needed (2) contd. ‘This quest for reliability tends to skew assessment towards the assessment of simple and unambiguous achievements, and considerations of cost add to the skew away from judgements of complex learning’ (Knight 2002 p278) “…students become more interested in the mark and less interested in the subject over the course of their studies.” (Newstead 2002, p2) “…summative judgement itself is the problem” (Burgess, 2007, p. 8)

Tenet 2 “When it comes to the assessment of learning, we need to move beyond systems focused on marks and grades towards the valid assessment of the achievement of intended programme outcomes.”

Why change is needed (3)Some aspects of quality cannot be communicated through explicit criteria alone • Regulative and logical criteria “standards can be defined in terms of well-defined outcomes” (Sadler, 1987, p. 70) • Prescriptive and constitutive criteria refer to matters of degree and “It would be difficult or impossible to guess the educational level at which they are applicable… ” (Sadler, 1987, p. 70) Such types of criteria are often interdependent and can only be assessed using holistic/professional judgement (Sadler, 2008) • Such criteria are socially constructed requiring the sharing of tacit knowledge over time (O’Donovan et al, 2004; Rust et al, 2005)

Limitations of explicit articulation • Meaningful understanding of standards requires both tacit and explicit knowledge (O’Donovan et al. 2004) • “we can know more than we can tell” (Polanyi, reprinted 1998, p.136). • Verbal level descriptors are inevitably ‘fuzzy’ (Sadler 1987) • There is a cost (in terms of time and resources) to codifying knowledge which increases the more diverse an audience’s experience and language (Snowdon, 2002).

Tenet 3 “ Limits to the extent that standards can be articulated explicitly must be recognised since ever more detailed specificity and striving for reliability, all too frequently, diminish the learning experience and threaten its validity. There are important benefits of higher education which are not amenable either to the precise specification of standards or to objective assessment.”

Why change is needed (4)Assessment standards applied to high-level complex learning can only be understood through active engagement with members of a disciplinary community • ‘making sense of the world’ is a social and collaborative activity (Vygotsky, 1978) • Tacit knowledge is experience-based and can only be revealed through the sharing of experience – socialisation processes involving observation, imitation and practice (Nonaka, 1991) • An indispensable condition for improvement in student learning is that “the student comes to hold a concept of quality roughly similar to that held by the teacher” (Sadler, 1989) • Passive receipt of feedback has little effect on future performance (Fritz, et al., 2000) • Dialogue and participatory relationships are key elements of engaging students with assessment feedback (ESwAF FDTL, 2007)

Why change is needed (4) contd. • The most significant factor in student academic success is student involvement fostered by student/staff interactions and student/student interactions (Astin, 1997) • The only common factor in a study of departments deemed excellent in both research and learning and teaching is high levels of student involvement (Gibbs, 2007) • “participation, as a way of learning, enables the student to both absorb, and be absorbed in the culture of practice” (Elwood & Klenowski, 2002, p. 246)

Tenet 4 “Assessment standards are socially constructed so there must be a greater emphasis on assessment and feedback processes that actively engage both staff and students in dialogue about standards. It is when learners share an understanding of academic and professional standards in an atmosphere of mutual trust that learning works best.”

Why change is needed (5) Important aspects of complex, high-level learning outcomes can only be achieved when students are allowed time to ‘come to know’ the standards in use by the community. • Slowly learnt academic literacies require rehearsal and practice throughout a programme (Knight & Yorke, 2004) • The achievement of high-level learning requires integrated and coherent progression based on programme outcomes • Where there is a greater sense of the holistic programme students are likely to achieve higher standards than on more fragmented programmes (Havnes, p. 2007) • Students need to engage as interactive partners in a learning community, relinquishing the passive role of ‘the instructed’ within processes controlled by academic experts (Gibbs et al, 2004)

Tenet 5 “ Active engagement with assessment standards needs to be an integral and seamless part of course design and the learning process in order to allow students to develop their own, internalised, conceptions of standards and monitor and supervise their own learning.”

Why change is needed (6) • Changes in higher education (e.g. massification, reduced unit of resource, expectations of increased productivity in staff) threaten the ‘health’ of disciplinary communities and their ability to share and exemplify professional judgement. • There has been slow progress in the professionalisation of university teachers • There has been limited attention paid to professional assessment practice • Reliance on the external examiner system to mediate standards within the system is misplaced (Newstead and Dennis,1994) • “…it cannot be assumed students graduating …. will have achieved similar standards” (QAA, 2007) If some aspects of high-level learning can only be assessed using professional judgement then we need to ensure that judgement is indeed professional

Tenet 6 “ Assessment is largely dependent upon professional judgement and confidence in such judgement requires the establishment of appropriate forums for the development and sharing of standards within and between disciplinary and professional communities.”

More…. For more about the background arguments behind the manifesto, go to: Price, M., O’Donovan, B., Rust, C. & Carroll, J(2008), 'Assessment standards: a manifesto for change', Brookes eJournal of Learning and Teaching, Vol.2, No. 3. (Online December 2008) Available at:  http://bejlt.brookes.ac.uk/article/assessment_standards_a_manifesto_for_change If you would like to be a personal signatory to the manifesto please visit: http://www.business.brookes.ac.uk/learningandteaching/aske/

Assessment Standards: A Manifesto for Change