Assessment Practice: A Manifesto for Change

1. Assessment Standards Knowledge exchange Assessment Practice:A Manifesto for Change Dr Chris Rust, Deputy Director ASKe CETL Directorate: Margaret Price, Jude Carroll, Karen Handley, Berry O’Donovan and Chris Rust

2. Importance of assessment Assessment: a key driver of student learning “Assessment is at the heart of the student experience” (Brown, S & Knight, P., 1994) “From our students’ point of view, assessment always defines the actual curriculum” (Ramsden, P.,1992) “Assessment defines what students regard as important, how they spend their time and how they come to see themselves as students and then as graduates.........If you want to change student learning then change the methods of assessment” (Brown, G et al, 1997)

3. But (in the UK) • QAA subject reviews • Burgess Report - “system no longer fit for purpose” (p5) • QAA - “…it cannot be assumed students graduating …. will have achieved similar standards” (2007) • Media accusations of dumbing down & grade inflation • National Student Satisfaction Survey • “the Achilles’ heel of quality” (Knight 2002a, p. 107) • Summative assessment practices “in disarray” (Knight 2002b, p. 275) • “Broken” (Race 2003, p. 5) • “There is considerable scope for professional development in the area of assessment” (Yorke et al, 2000, p7)

4. And let’s remember… “Students can, with difficulty, escape from the effects of poor teaching, they cannot (by definition if they want to graduate) escape the effects of poor assessment. Assessment acts as a mechanism to control students that is far more pervasive and insidious than most staff would be prepared to acknowledge. It appears to conceal the deficiencies of teaching as much as it does to promote learning. If, as teachers, we want to exert maximum leverage over change in higher education we must confront the ways in which assessment tends to undermine learning.” (Boud, 1995, p35)

5. And at least once, in the USA… “In Pasadena, California, student Edward Mulrooney was arrested after he tossed a bomb at his psychology teacher’s house and left a note that said: ‘If you don’t want your home bombed or your windows shot out, then grade fairly and put your assignments on the board – or is this asking too much?’” Time magazine, 16th April 1956 [Quoted by Bill Bryson in The Life and Times of the Thunderbolt Kid, 2006, p153]

6. Origin of the manifesto • Weston Manor Group, November 07 • 40 National and International Experts in Assessment • Two days of discussions Outcomes so far… • Six tenet manifesto for change to assessment practice related to standards • Sent to HEFCE, HEA, QAA, UUK, GuildHE, NUS • Lead article in THES – April 24, 08 • Submissions, both written (08) and oral (09), to Parliamentary IUSS select committee • Working with four UK institutions

7. Focus on standards “Assessment standards lie at the heart of the assessment process underpinning assessment strategy, design and marking. For students, assessment standards provide guidance for their learning and allow them to monitor their progress, and ultimately, the standards will be used to judge their performance.” (Price et al, 2008) “It is unacceptable for the sector to be in receipt of departmental spending of £15 billion but be unable to answer a straightforward question about the relative standards of the degrees of students, which the taxpayer has paid for” (IUSS committee, 2009, p 153)

8. Why change is needed (1) “The types of assessment we currently use do not promote conceptual understanding and do not encourage a deep approach to learning………Our means of assessing them seems to do little to encourage them to adopt anything other than a strategic or mechanical approach to their studies.” (Newstead 2002, p3) “Even when lecturers say that they want students to be creative and thoughtful, students often recognise that what is really necessary, or at least what is sufficient, is to memorise” (Gibbs, 1992, p. 10) Many research findings indicate a declining use of deep and contextual approaches to study as students’ progress through their degree programmes (Watkins & Hattie, 1985; Kember et al, 1997; Richardson, 2000; Zhang & Watkins, 2001)

9. Tenet 1 “The debate on standards needs to focus on how high standards of learning can be achieved through assessment. This requires a greater emphasis on assessment for learning rather than assessment of learning.”

10. Why change is needed (2) Our current systems, focused on marks and grades, aren’t working • Belief that it is possible to distinguish the quality of work to a precision of one percentage point (Elander & Hardman, 2002) • If the scale is not quantitative (i.e. 60 = 1.5 x 40), standard arithmetic operations on the grades are illegitimate (Dalziel, 1998) • The combination of scores, which obscures the different types of learning outcome represented by the separate scores • Transactional and bestowed credits & debits (Sadler, 2009) • Belief that double-marking will ensure fairness and reliability (Laming,1990; Brooks, 2004)

11. Why change is needed (2) contd. • “…students become more interested in the mark and less interested in the subject over the course of their studies.” (Newstead 2002, p2) • Belief that consistency can be achieved through conformity, and simple numerical rules (e.g. level 1 essay 3,000 words, level 3 essay 5,000; or no more than two pieces of assessment per module) • The distortion of marks by the type of assessment (e.g. coursework c.f. examination) and the actual subject discipline/s studied (Yorke, 1997; Yorke et al, 2000, Bridges et al, 2002) • The distortion of resulting degree classifications by the application of idiosyncratic institutional rules (e.g. Armstrong et al, 1998) (Rust, 2007)

12. Tenet 2 “When it comes to the assessment of learning, we need to move beyond systems focused on marks and grades towards the valid assessment of the achievement of intended programme outcomes.”

13. Why change is needed (3)Some aspects of quality cannot be communicated through explicit criteria alone • Regulative and logical criteria “standards can be defined in terms of well-defined outcomes” (Sadler, 1987, p. 70) • Prescriptive and constitutive criteria refer to matters of degree and “It would be difficult or impossible to guess the educational level at which they are applicable… ” (Ibid) • Such types of criteria are often interdependent and can only be assessed using holistic/professional judgement (Sadler, 2008)

14. Limitations of explicit articulation • Verbal level descriptors are inevitably ‘fuzzy’ (Sadler 1987) • “we can know more than we can tell” (Polanyi, reprinted 1998, p.136). • Meaningful understanding of standards requires both tacit and explicit knowledge (O’Donovan et al. 2004)

15. Tenet 3 “ Limits to the extent that standards can be articulated explicitly must be recognised since ever more detailed specificity and striving for reliability, all too frequently, diminish the learning experience and threaten its validity. There are important benefits of higher education which are not amenable either to the precise specification of standards or to objective assessment.”

16. Why change is needed (4)Assessment standards applied to high-level complex learning can only be understood through active engagement with members of a disciplinary community • ‘making sense of the world’ is a social and collaborative activity (Vygotsky, 1978) • Criteria are socially constructed requiring the sharing of tacit knowledge over time (O’Donovan et al, 2004; Rust et al, 2005) • Tacit knowledge is experience-based and can only be revealed through the sharing of experience – socialisation processes involving observation, imitation and practice (Nonaka, 1991) • An indispensable condition for improvement in student learning is that “the student comes to hold a concept of quality roughly similar to that held by the teacher” (Sadler, 1989)

17. Why change is needed (4) contd. • Passive receipt of feedback has little effect on future performance (Fritz, et al., 2000) • Dialogue and participatory relationships are key elements of engaging students with assessment feedback (ESwAF FDTL, 2007) • It is not enough to make it a better monologue; feedback must be seen as a dialogue (Nicol, 2009) • “participation, as a way of learning, enables the student to both absorb, and be absorbed in the culture of practice” (Elwood & Klenowski, 2002, p. 246) • The most significant factor in student academic success is student involvement fostered by student/staff interactions and student/student interactions (Astin, 1997) • The only common factor in a study of departments deemed excellent in both research and learning and teaching is high levels of student involvement (Gibbs, 2007)

18. Tenet 4 “Assessment standards are socially constructed so there must be a greater emphasis on assessment and feedback processes that actively engage both staff and students in dialogue about standards. It is when learners share an understanding of academic and professional standards in an atmosphere of mutual trust that learning works best.”

19. Why change is needed (5) Important aspects of complex, high-level learning outcomes can only be achieved when students are allowed time to ‘come to know’ the standards in use by the community. • Slowly learnt academic literacies require rehearsal and practice throughout a programme (Knight & Yorke, 2004) • The achievement of high-level learning requires integrated and coherent progression based on programme outcomes • Where there is a greater sense of the holistic programme students are likely to achieve higher standards than on more fragmented programmes (Havnes, p. 2007) • Students need to engage as interactive partners in a learning community, relinquishing the passive role of ‘the instructed’ within processes controlled by academic experts (Gibbs et al, 2004) • Assessment should also be “about equipping students for the learning and assessing they will need to do after completing their course and the challenges they will face after graduation” (Assessment Futures website, 2009)

20. Tenet 5 “ Active engagement with assessment standards needs to be an integral and seamless part of course design and the learning process in order to allow students to develop their own, internalised, conceptions of standards and monitor and supervise their own learning.”

21. Why change is needed (6) • Changes in higher education (e.g. massification, reduced unit of resource, expectations of increased productivity in staff) threaten the ‘health’ of disciplinary communities and their ability to share and exemplify professional judgement. • There has been slow progress in the professionalisation of university teachers • There has been limited attention paid to professional assessment practice • Reliance on the external examiner system to mediate standards within the system is misplaced (e.g. Newstead and Dennis,1994; IUSS, 2009) • “…it cannot be assumed students graduating …. will have achieved similar standards” (QAA, 2007) If some aspects of high-level learning can only be assessed using professional judgement then we need to ensure that judgement is indeed professional

22. Tenet 6 “ Assessment is largely dependent upon professional judgement and confidence in such judgement requires the establishment of appropriate forums for the development and sharing of standards within and between disciplinary and professional communities.”

23. More about the manifesto… For more about the background arguments behind the manifesto, go to: Price, M., O’Donovan, B., Rust, C. & Carroll, J(2008), 'Assessment standards: a manifesto for change', Brookes eJournal of Learning and Teaching, Vol.2, No. 3. (Online December 2008) Available at: http://bejlt.brookes.ac.uk/article/assessment_standards_a_manifesto_for_change If you would like to be a personal signatory to the manifesto please visit: http://www.business.brookes.ac.uk/learningandteaching/aske/