Streamlining assessment: giving feedback effectively and efficiently UEA 20 th January 2012

1. Streamlining assessment: giving feedback effectively and efficientlyUEA 20th January 2012 Professor Sally Brown Emeritus Professor, Leeds Metropolitan University Adjunct Professor, University of the Sunshine Coast and James Cook University Visiting Professor University of Plymouth

2. By the end of this workshop, you will have had opportunities to: • Discuss the importance of feedback as part of the learning process; • Review how feedback can be used as part of a cycle; • Consider how you can enable students to learn from each assignment cumulatively; • Make feedback fit-for-purpose.

3. Why does assessment matter so much? “Assessment methods and requirements probably have a greater influence on how and what students learn than any other single factor. This influence may well be of greater importance than the impact of teaching materials” (Boud 1988)

4. What students think about assessment Student evaluations frequently reveal poor assessment practices that: • Lack authenticity and relevance to real world tasks; • Make unreasonable demands on students; • Are narrow in scope; • Have little long-term benefit; • Fail to reward genuine effort; • Have unclear expectations and assessment criteria; • Fail to provide adequate feedback to students; • Rely heavily on factual recall rather than on higher-order thinking and problem-solving skills. (Flint and Johnson, 2011, p.2)

5. Interpreting NSS scores • NSS scores are an unreliable indicator as they can be skewed by all kinds of extraneous factors, but do give us some hints on what we should be doing; • Attempts to massage NSS scores usually fail; • It is much more sensible and worthwhile to concentrate on improving the student experience of assessment and feedback, since these are so central to enhancing the student experience.

6. Boud et al 2010 propose that assessment has most effect when...: • It is used to engage students in learning that is productive. • Feedback is used to actively improve student learning. • Students and teachers become responsible partners in learning and assessment. • Students are inducted into the assessment practices and cultures of higher education. • Assessment for learning is placed at the centre of subject and program design. • Assessment for learning is a focus for staff and institutional development. • Assessment provides inclusive and trustworthy representation of student achievement. (Assessment 2020)

7. Formative and summative assessment • Formative assessment is primarily concerned with feedback aimed at prompting improvement, is often continuous and usually involves words. • Summative assessment is concerned with making evaluative judgments, is often end point and involves numbers.

8. Learning from Royce Sadler “The indispensable conditions for improvement are that the student comes to hold a concept of quality roughly similar to that held by the teacher, is able to monitor continuously the quality of what is being produced during the act of production itself, and has a repertoire of alternative moves or strategies from which to draw at any given point. In other words, students have to be able to judge the quality of what they are producing and be able to regulate what they are doing during the doing of it” (Sadler 1989, my italics).

9. More from Sadler “For students to be able to apply feedback, they need to understand the meaning of the feedback statements. They also need to identify, with near certainty, the particular aspects of their work that need attention. For these to occur, students must possess critical background knowledge. [Students] must appropriate for themselves three fundamental concepts - task compliance, quality and criteria - and also develop a cache of relevant tacit knowledge”. (Sadler 2010 ‘Beyond feedback: developing student capability in complex appraisal’)

10. Nicol and Macfarlane-Dick Good feedback practice: 1. Helps clarify what good performance is (goals, criteria, expected standards); 2. Facilitates the development of self-assessment (reflection) in learning; 3. Delivers high quality information to students about their learning; 4. Encourages teacher and peer dialogue around learning; 5. Encourages positive motivational beliefs and self-esteem; 6. Provides opportunities to close the gap between current and desired performance; 7. Provides information to teachers that can be used to help shape the teaching.

11. Assessment for learning 1 1. Tasks should be challenging, demanding higher order learning and integration of knowledge learned in both the university and other contexts; 2. Learning and assessment should be integrated, assessment should not come at the end of learning but should be part of the learning process; 3. Students are involved in self assessment and reflection on their learning, they are involved in judging performance; 4. Assessment should encourage metacognition, promoting thinking about the learning process not just the learning outcomes; 5. Assessment should have a formative function, providing ‘feedforward’ for future learning which can be acted upon. There is opportunity and a safe context for students to expose problems with their study and get help; there should be an opportunity for dialogue about students’ work.

12. Assessment for learning 2 6. Assessment expectations should be made visible to students as far as possible; 7. Tasks should involve the active engagement of students developing the capacity to find things out for themselves and learn independently; 8. Tasks should be authentic; worthwhile, relevant and offering students some level of control over their work; 9. Tasks are fit for purpose and align with important learning outcomes; 10. Assessment should be used to evaluate teaching as well as student learning. (Sue Bloxham, unpublished paper for HEA)

13. Enhancing learning: how can we improve assessment and feedback to maximise student achievement?

14. Graham Gibbs: tactics to improve student learning • Capture student time and effort, distributing that effort appropriately across topics and weeks. • Generate high-quality learning effort, oriented towards clear and high standards. • Provide sufficient feedback, often enough, and in enough detail. • Focus feedback on students’ performance, on actions under their control, rather than on students themselves or their characteristics. • Make feedback timely, while it still matters to students, in time for them to use it towards further learning, or to receive further assistance. • Link feedback to what students believe they are supposed to be doing. • Ensure that feedback is not only received, but is attended to, so that students act on it to change their future learning and performance.

15. To improve assessment we should realign it by: • Exploring ways in which assessment can be made integral to learning. • Constructively aligning (Biggs 2003) assignments with planned learning outcomes and the curriculum taught: • Providing realistic tasks: students are likely to put more energy into assignments they see as authentic and worth bothering with; • Providing faster, more effective and more detailed feedback.

16. Providing faster, more effective and more detailed feedback is hard • But numerous studies (and the NSS, plus other measures of student satisfaction) tell us that students really want to see improvements in this area above all others; • There are clear links between good feedback and effective learning; • Universities in tough times need a competitive edge: supporting learning through good feedback makes good business sense.

17. The feedback cycle • To what extent have we designed systems to enable students to benefit from feedback before they complete the next assignment? • To what extent can we build in tasks relating to feedback on earlier assignments within subsequent assignments (‘feedforward’)? • How can we help students to build on what they have learned through assessment between modules and academic years?

18. Some ways in which we can give feedback faster • Use ‘exploded’ model answers; • Give whole cohort feedback in the form of an oral or written report in the classroom; • Harness statement banks; • Use assignment return sheets; • Involve students in their own and each other’s assessment (inter & intra peer group plus self assessment); • Make better use of computer-aided assessment.

19. Assessment matters • Effective assessment significantly and positively impacts on student learning, (Boud, Mentkowski, Knight and Yorke and many others). • Assessment shapes student behaviour (marks as money) and poor assessment encourages strategic behaviour (Kneale). Clever course developers utilise this tendency and design assessment tools that foster the behaviours we would wish to see (for example, logical sequencing, fluent writing, effective referencing and good time management). • Feed-forward (Gibbs et al) shapes student behaviour by focussing information and advice for students on future performance in assignments, not just ‘correciones’.

20. Setting good patterns • Students rarely respond positively to exhortation or vague threats of poor marks: we need to change the assessment practices so that they make routine these behaviours very early on in a their HE career. • Yorke (1999) encourages us to believe that the first six weeks of the first semester of the first year are crucial and that how we assess within that period can make a difference to student success or failure. • Avoidance of assessment in Semester One doesn’t solve the problem. Designing a really coherent first six weeks for students, which includes assessment opportunities can be very helpful.

21. Diverse and innovative assessment helps • Traditional assessment methods tend to reinforce rather limited approaches to learning by students, by encouraging memorisation, unproductive rote learning and attitude to knowledge acquisition that are reminiscent of the language of eating disorders (stuffing in and regurgitation of facts). We need to utilise a wide range of assessment methods and approaches. • Innovative assessment approaches can foster a spirit of enquiry, encourage curiosity and promote autonomy where they encourage students to become closely involved with evaluating their own and each others’ learning. (Falchikov, Pickford and Brown, 2006).

22. Sound and frequent assessment • Students need regular feedback so they can improve performance at a time when this can make a difference; • Students need to see examples of good work so they can evaluate their own work effectively; • Good assessment is valid, reliable, practical, developmental, manageable, cost-effective, fit for purpose, relevant, authentic, inclusive, closely linked to learning outcomes and fair; • We should explore whether it is possible also to make it enjoyable for staff and students; • Incremental assessment has more value in promoting student learning than end-point ‘sudden death’ approaches.

23. Helping students understand and use feedback • Frequent, formative feedback impacts positively on student learning and we need to re-engineer practices to make this possible; • The UK Open University inter alia is keen to promote feed-forward as well as feedback, prompting students to use advice from one assignment to inform their actions prior to the next one. • Students need convincing that assignments are not just ‘make work’ or punishing tasks. We need to win their hearts and minds to recognise that assessment is integral to their learning and is a crucial part of it.

24. Students giving feedback to peers • Can be hugely beneficial if managed effectively (but there are no quick fixes!); • Students will need training or refreshing in purposes and practices of peer feedback; • Work on language use is crucial since students can be very harsh on one another: training may be necessary to help them build a repertoire of formative feedback responses; • Building students’ expertise in giving peer feedback helps them get more from the feedback they receive.

25. Conclusions • Feedback impacts significantly on student learning, so designing good feedback mechanisms is highly important; • A systemic approach to feedback and ‘feedforward’ needs to be adopted by programme teams holistically to have best effect; • To make time and resources available for giving effective feedback, we are likely to need to reengineer other aspects of the curriculum, for example, content delivery.

26. References (1) Assessment Reform Group (1999) Assessment for Learning : Beyond the black box Cambridge UK, University of Cambridge School of Education. Biggs, J. and Tang, C. (2007) Teaching for Quality Learning at University Maidenhead: Open University Press. Bloxham, S. and Boyd, P. (2007) Developing assessment in Higher education: a practical guide. Maidenhead: Open University Press. Bloxham, S. Assessment for Learning, Unpublished paper for HEA Boud, D. (1995) Enhancing learning through self-assessment London: Routledge. Boud, D. and Associates (2010) Assessment 2020: seven propositions for assessment reform in higher education Sydney: Australian Learning and Teaching Council. Brown, G. with Bull, J. and Pendlebury, M. (1997) Assessing Student Learning in Higher Education London: Routledge. Brown, S. and Glasner, A. (ed.) (1999) Assessment Matters in Higher Education, Choosing and Using Diverse Approaches, Maidenhead: Open University Press. Brown, S. and Knight, P. (1994) Assessing Learners in Higher Education, London: Kogan Page. Brown, S. Rust, C. & Gibbs, G. (1994) Strategies for Diversifying Assessment Oxford Centre for Staff Development.

27. References (2) Crooks, T. (1988) Assessing student performance HERDSA Green Guide No 8 HERDSA (reprinted 1994) Falchikov, N. (2004) Improving Assessment through Student Involvement: Practical Solutions for Aiding Learning in Higher and Further Education, London: Routledge. Flint, N. R. and Johnson, B. (2011) Towards fairer university assessment – recognising the concerns of students, London: Routledge. Gibbs, G. (1999) Using assessment strategically to change the way students learn, In Brown S. & Glasner, A. (eds.), Assessment Matters in Higher Education: Choosing and Using Diverse Approaches Maidenhead: SRHE/Open University Press. Gibbs, G. (2010) Using assessment to support student learning, Leeds: Leeds Met Press. Kneale, P. E. (1997) The rise of the "strategic student": how can we adapt to cope? in Armstrong, S., Thompson, G. and Brown, S. (eds) Facing up to Radical Changes in Universities and Colleges, pp.119-139 London: Kogan Page. Knight, P. and Yorke, M. (2003) Assessment, learning and employability Maidenhead, UK: SRHE/Open University Press.

28. References (3) Mentkowski, M. and associates (2000) p.82 Learning that lasts: integrating learning development and performance in college and beyond, San Francisco: Jossey-Bass. Nicol, D. J. and Macfarlane-Dick, D. (2006) Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, Vol 31(2), 199-218 Pickford, R. and Brown, S. (2006) Assessing skills and practice, London: Routledge. Race, P. (2001) A Briefing on Self, Peer & Group Assessment in LTSN Generic Centre Assessment Series No 9 LTSN York. Race, P. (2006) The lecturer’s toolkit (3rd edition) London: Routledge. Rust, C., Price, M. and O’Donovan, B. (2003) Improving students’ learning by developing their understanding of assessment criteria and processes Assessment and Evaluation in Higher Education. 28 (2), 147-164. Sadler, R. (2008) Assessment of Higher Education, in International Encyclopaedia of Education Yorke, M. (1999) Leaving Early: Undergraduate Non-completion in Higher Education, London: Routledge.