Making a difference through assessment NCI Learning and Teaching Development workshop

1. Making a difference through assessmentNCI Learning and Teaching Development workshop http://sally-brown.net Emeritus Professor, Leeds Metropolitan University, Adjunct professor, University of the Sunshine Coast, Central Queensland and James Cook University Queensland. Visiting Professor: University of Plymouth and Liverpool John Moores University. .

2. Why does assessment matter so much? “Assessment methods and requirements probably have a greater influence on how and what students learn than any other single factor. This influence may well be of greater importance than the impact of teaching materials” (Boud 1988)

3. What students think about assessment Student evaluations frequently reveal poor assessment practices that: • Lack authenticity and relevance to real world tasks; • Make unreasonable demands on students; • Are narrow in scope; • Have little long-term benefit; • Fail to reward genuine effort; • Have unclear expectations and assessment criteria; • Fail to provide adequate feedback to students; • Rely heavily on factual recall rather than on higher-order thinking and problem-solving skills. (Flint and Johnson, 2011, p.2)

4. Assessment matters • Effective assessment significantly and positively impacts on student learning, (Boud, Mentkowski, Knight and Yorke and many others). • Assessment shapes student behaviour (marks as money) and poor assessment encourages strategic behaviour (Kneale). Clever course developers utilise this tendency and design assessment tools that foster the behaviours we would wish to see (for example, logical sequencing, fluent writing, effective referencing and good time management). • Feed-forward (Gibbs et al) shapes student behaviour by focussing information and advice for students on future performance in assignments, not just ‘correciones’.

5. To improve assessment we should realign it by: • Exploring ways in which assessment can be made integral to learning. • Constructively aligning (Biggs 2003) assignments with planned learning outcomes and the curriculum taught: • Providing realistic tasks: students are likely to put more energy into assignments they see as authentic and worth bothering with; • Providing faster, more effective and more detailed feedback.

6. A fit-for-purpose model of assessment: the key questions Why are we assessing? What is it we are actually assessing? How are we assessing? Who is best placed to assess? When should we assess?

7. Boud et al 2010 propose that assessment has most effect when...: • It is used to engage students in learning that is productive. • Feedback is used to actively improve student learning. • Students and teachers become responsible partners in learning and assessment. • Students are inducted into the assessment practices and cultures of higher education. • Assessment for learning is placed at the centre of subject and program design. • Assessment for learning is a focus for staff and institutional development. • Assessment provides inclusive and trustworthy representation of student achievement. (Assessment 2020)

8. Formative and summative assessment • Formative assessment is primarily concerned with feedback aimed at prompting improvement, is often continuous and usually involves words. • Summative assessment is concerned with making evaluative judgments, is often end point and involves numbers.

9. Learning from Royce Sadler “The indispensable conditions for improvement are that the student comes to hold a concept of quality roughly similar to that held by the teacher, is able to monitor continuously the quality of what is being produced during the act of production itself, and has a repertoire of alternative moves or strategies from which to draw at any given point. In other words, students have to be able to judge the quality of what they are producing and be able to regulate what they are doing during the doing of it” (Sadler 1989, my italics).

10. Nicol and Macfarlane-Dick Good feedback practice: 1. Helps clarify what good performance is (goals, criteria, expected standards); 2. Facilitates the development of self-assessment (reflection) in learning; 3. Delivers high quality information to students about their learning; 4. Encourages teacher and peer dialogue around learning; 5. Encourages positive motivational beliefs and self-esteem; 6. Provides opportunities to close the gap between current and desired performance; 7. Provides information to teachers that can be used to help shape the teaching.

11. Diverse and innovative assessment helps • Traditional assessment methods tend to reinforce rather limited approaches to learning by students, by encouraging memorisation, unproductive rote learning and attitude to knowledge acquisition that are reminiscent of the language of eating disorders (stuffing in and regurgitation of facts). We need to utilise a wide range of assessment methods and approaches. • Innovative assessment approaches can foster a spirit of enquiry, encourage curiosity and promote autonomy where they encourage students to become closely involved with evaluating their own and each others’ learning. (Falchikov, Pickford and Brown, 2006).

12. Assessment for learning 1 1. Tasks should be challenging, demanding higher order learning and integration of knowledge learned in both the university and other contexts; 2. Learning and assessment should be integrated, assessment should not come at the end of learning but should be part of the learning process; 3. Students are involved in self assessment and reflection on their learning, they are involved in judging performance; 4. Assessment should encourage metacognition, promoting thinking about the learning process not just the learning outcomes; 5. Assessment should have a formative function, providing ‘feedforward’ for future learning which can be acted upon. There is opportunity and a safe context for students to expose problems with their study and get help; there should be an opportunity for dialogue about students’ work.

13. Assessment for learning 2 6. Assessment expectations should be made visible to students as far as possible; 7. Tasks should involve the active engagement of students developing the capacity to find things out for themselves and learn independently; 8. Tasks should be authentic; worthwhile, relevant and offering students some level of control over their work; 9. Tasks are fit for purpose and align with important learning outcomes; 10. Assessment should be used to evaluate teaching as well as student learning. (Sue Bloxham, unpublished paper for HEA)

14. Graham Gibbs: tactics to improve student learning • Capture student time and effort, distributing that effort appropriately across topics and weeks. • Generate high-quality learning effort, oriented towards clear and high standards. • Provide sufficient feedback, often enough, and in enough detail. • Focus feedback on students’ performance, on actions under their control, rather than on students themselves or their characteristics. • Make feedback timely, while it still matters to students, in time for them to use it towards further learning, or to receive further assistance. • Link feedback to what students believe they are supposed to be doing. • Ensure that feedback is not only received, but is attended to, so that students act on it to change their future learning and performance.

15. Providing faster, more effective and more detailed feedback is hard • But numerous studies (and the UK NSS, plus other measures of student satisfaction) tell us that students really want to see improvements in this area above all others; • There are clear links between good feedback and effective learning; • Universities in tough times need a competitive edge: supporting learning through good feedback makes good business sense.

16. Making the most of feedback • Plan to maximise the impact of formative feedback. Release extra time helping students to understand the importance of feedback and the value of spending some of their time after receiving work back to learn from the experience. • Provide opportunities for students to respond to our feedback, for example, by giving students follow-up tasks or give them ‘feed-forward’ comments to improve their next piece of work. • Think about the means by which we deliver feedback, since this can be vital in determining how much notice students take of what you say.

17. The feedback cycle • To what extent have we designed systems to enable students to benefit from feedback before they complete the next assignment? • To what extent can we build in tasks relating to feedback on earlier assignments within subsequent assignments (‘feedforward’)? • How can we help students to build on what they have learned through assessment between modules and academic years?

18. Some ways in which we can give feedback faster • Use ‘exploded’ model answers; • Give whole cohort feedback in the form of an oral or written report in the classroom; • Harness statement banks; • Use assignment return sheets; • Involve students in their own and each other’s assessment (inter & intra peer group plus self assessment); • Make better use of computer-aided assessment.

19. Setting good patterns • Students rarely respond positively to exhortation or vague threats of poor marks: we need to change the assessment practices so that they make routine these behaviours very early on in a their HE career. • Yorke (1999) encourages us to believe that the first six weeks of the first semester of the first year are crucial and that how we assess within that period can make a difference to student success or failure. • Avoidance of assessment in Semester One doesn’t solve the problem. Designing a really coherent first six weeks for students, which includes assessment opportunities can be very helpful.

20. Sound and frequent assessment • Students need regular feedback so they can improve performance at a time when this can make a difference; • Students need to see examples of good work so they can evaluate their own work effectively; • Good assessment is valid, reliable, practical, developmental, manageable, cost-effective, fit for purpose, relevant, authentic, inclusive, closely linked to learning outcomes and fair; • We should explore whether it is possible also to make it enjoyable for staff and students; • Incremental assessment has more value in promoting student learning than end-point ‘sudden death’ approaches.

21. Helping students understand and use feedback • Frequent, formative feedback impacts positively on student learning and we need to re-engineer practices to make this possible; • The UK Open University inter alia is keen to promote feed-forward as well as feedback, prompting students to use advice from one assignment to inform their actions prior to the next one. • Students need convincing that assignments are not just ‘make work’ or punishing tasks. We need to win their hearts and minds to recognise that assessment is integral to their learning and is a crucial part of it.

22. Students giving feedback to peers • Can be hugely beneficial if managed effectively (but there are no quick fixes!); • Students will need training or refreshing in purposes and practices of peer feedback; • Work on language use is crucial since students can be very harsh on one another: training may be necessary to help them build a repertoire of formative feedback responses; • Building students’ expertise in giving peer feedback helps them get more from the feedback they receive.

23. Conclusions • Assessment and feedback impact significantly on student learning, so designing good systems, and making sure they work well, is crucial; • A systemic approach to assessment, feedback and ‘feedforward’ needs to be adopted holistically to have best effect; • To make time and resources available for doing better assessment and giving more effective feedback, we are likely to need to reengineer other aspects of the curriculum, for example, content delivery.

24. References (1) Assessment Reform Group (1999) Assessment for Learning : Beyond the black box Cambridge UK, University of Cambridge School of Education. Biggs, J. and Tang, C. (2007) Teaching for Quality Learning at University Maidenhead: Open University Press. Bloxham, S. and Boyd, P. (2007) Developing assessment in Higher education: a practical guide. Maidenhead: Open University Press. Bloxham, S. Assessment for Learning, Unpublished paper for HEA Boud, D. (1995) Enhancing learning through self-assessment London: Routledge. Boud, D. and Associates (2010) Assessment 2020: seven propositions for assessment reform in higher education Sydney: Australian Learning and Teaching Council. Brown, G. with Bull, J. and Pendlebury, M. (1997) Assessing Student Learning in Higher Education London: Routledge. Brown, S. and Glasner, A. (ed.) (1999) Assessment Matters in Higher Education, Choosing and Using Diverse Approaches, Maidenhead: Open University Press. Brown, S. and Knight, P. (1994) Assessing Learners in Higher Education, London: Kogan Page. Brown, S. Rust, C. & Gibbs, G. (1994) Strategies for Diversifying Assessment Oxford Centre for Staff Development.

25. References (2) Crooks, T. (1988) Assessing student performance HERDSA Green Guide No 8 HERDSA (reprinted 1994) Falchikov, N. (2004) Improving Assessment through Student Involvement: Practical Solutions for Aiding Learning in Higher and Further Education, London: Routledge. Flint, N. R. and Johnson, B. (2011) Towards fairer university assessment – recognising the concerns of students, London: Routledge. Gibbs, G. (1999) Using assessment strategically to change the way students learn, In Brown S. & Glasner, A. (eds.), Assessment Matters in Higher Education: Choosing and Using Diverse Approaches Maidenhead: SRHE/Open University Press. Gibbs, G. (2010) Using assessment to support student learning, Leeds: Leeds Met Press. Kneale, P. E. (1997) The rise of the "strategic student": how can we adapt to cope? in Armstrong, S., Thompson, G. and Brown, S. (eds) Facing up to Radical Changes in Universities and Colleges, pp.119-139 London: Kogan Page. Knight, P. and Yorke, M. (2003) Assessment, learning and employability Maidenhead, UK: SRHE/Open University Press.

26. References (3) Mentkowski, M. and associates (2000) p.82 Learning that lasts: integrating learning development and performance in college and beyond, San Francisco: Jossey-Bass. Nicol, D. J. and Macfarlane-Dick, D. (2006) Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, Vol 31(2), 199-218 Pickford, R. and Brown, S. (2006) Assessing skills and practice, London: Routledge. Race, P. (2001) A Briefing on Self, Peer & Group Assessment in LTSN Generic Centre Assessment Series No 9 LTSN York. Race, P. (2006) The lecturer’s toolkit (3rd edition) London: Routledge. Rust, C., Price, M. and O’Donovan, B. (2003) Improving students’ learning by developing their understanding of assessment criteria and processes Assessment and Evaluation in Higher Education. 28 (2), 147-164. Sadler, R. (2008) Assessment of Higher Education, in International Encyclopaedia of Education Yorke, M. (1999) Leaving Early: Undergraduate Non-completion in Higher Education, London: Routledge.