Marking Guides and Effective Feedback

1. Marking Guides and Effective Feedback Frances Chetwynd EATING 2011

2. Talk outline • My COLMSCT Project • Proposal for TU100 new Assessment and Tutor Guides • A TU100 example EATING 2011

3. COLMSCT Project • Assessment feedback – a new taxonomy • The survey: tutor attitudes to feedback and marking guides • The current T175 assignments, marking guides and feedback – a brief analysis: EATING 2011

4. The Taxonomy EATING 2011

5. A possible taxonomy of feedback Content Concepts Theories etc. Retrospective-on-content Future-altering-on-content (Brown & Glover 2006) Skills Verbal Numerical Planning Study etc. Retrospective-on-skills Future-altering-on-skills Future-altering Retrospective (Walker 2009) Refer to assignment just completed Points out omissions, misconceptions, etc. Explain how answer could have been improved How concepts and skills can be applied in future Clarify concepts and avoid future errors EATING 2011

6. The survey T175 tutors – 140 distributed, 70 respondents EATING 2011

7. The survey Background • 91% OU tutor for 5+ years, 29% 10+ years experience Tutors’ methods • Primary aim of PT3 comments to improve grade on next TMA (60%) • On-script comments most effective future-altering device (71%) • Additional home-made templates (54%) • Duplication and triplication of comments (35%) EATING 2011

8. The survey • Relative importance of feedback categories EATING 2011

9. Marking guides – a brief analysis EATING 2011

10. Feedback – Retrospective on Content • Mainly checklists • Ideal minus approach • Excludes other possible, or arguable answers • Generally no help for tutors in closing the gap, correcting misapprehensions, clarifying concepts • Used principally for mark allocation • Students are simply asked to list these components so bullets are acceptable here. Award 1 mark per ICT component listed below: • The touch screen • RFID reader • RFID tag • Computer terminal • Library computer network • Library database • Sensor gates • Alarm EATING 2011

11. Feedback – Retrospective on Skills • Tend to fall back on model answers • Underlying concepts may be implied only • No specific help on corrections and typical errors Award up to 4 marks. 1 mark for correct use of scientific notation, 2 for correct conversions of KB and Mbps and 1 for answer. File size = 42KB = 42 x 1024 x 8bits = 344 064 bits Download speed = 0.9Mbps = 0.9 x 106 bps Time to download = file size/ speed = 344 064/(0.9 x 106) = 382 293 x 10-6 s = 3.82 x 10-1 s EATING 2011

12. Feedback – Future altering on Skills • Again, tutors are left very much on their own • Language skills?? • To what extent should we expect tutors to be tutors in basic skills? Award high marks for clear and well structured notes suitable for intended audience. Give fewer marks if students use jargon, acronyms or unexplained technical concepts, but also use this as a teaching point about writing for specific audiences. Also do not award full marks to students that use full sentences rather than notes, as required. Do not deduct any marks if students go over the suggested 200 word limit but explain the importance of word limits in TMAs and generally. EATING 2011

13. Analysis of feedback opportunities • T175 – 1 Marking Guide EATING 2011

14. TU100 – the proposal EATING 2011

15. TU100 TMAs – general principles • Assessment for learning not assessment of learning • Support tutors in giving highest quality feedback • TMAs have a clear thread of progression and achievement • Promote re-use by CT • Can grow to match developments in the digital world. EATING 2011

16. TU100 Tutor Guides - general principles • Skills criteria properly flagged • Overall mark and comment on skills • Generic advice and documentation • Should look forward and back EATING 2011

17. Detailed TMA principles • TMAs need to explicitly develop skills, with skills requirements clearly stated • Should offer an opportunity to showcase achievement • Questions on SENSE should be, initially, a set of independent tasks • TMAs should focus on tutor feedback for learning; • All retrospective-on-skills marks should be gathered together into single mark EATING 2011

18. Detailed Tutor Guide principles • Advice on future-altering feedback presented separately from the retrospective-on-content and marks • Information on how each question links to other TMAs and module skills development • Granularity of marks reduced and simplified • Clear notes included on language skills EATING 2011

19. TMA01 & TMA02 • Now an opportunity to discuss the first two TMAs and Tutor Guides. • Have we achieved our aims? • Has it worked?? • How can we make the skills strands more explicit? EATING 2011

20. Future plans • eSTEeM project led by Chris Dobbyn: Transforming retention and progression in a new Level 1 course • Staged interviews with group of 15 tutors • Feedback on assessment and tutor guides • Establish leakage points and threshold concepts http://www.open.ac.uk/about/teaching-and-learning/esteem/p3.shtml EATING 2011

21. References Brown, E. and Glover, C. (2006) ‘Evaluating written feedback’, in Bryan, C. & Clegg, K. (Eds) Innovative Assessment in Higher Education Abingdon, Routledge, pp 81 – 91 Gibbs, G & Simpson, C. (2004) ‘Conditions under which assessment supports students’ learning’. Learning and Teaching in Higher Education 1 pp3 – 31  Hounsell, D. (2007)‘Towards more sustainable feedback’, in Boud, D. & Falchikov, N. (Eds) Rethinking Assessment in Higher Education. Abingdon, Routledge, pp 101 - 113  Huxham, M. (2007) ‘Fast and effective feedback: are model answers the answer?’ Assessment and Evaluation in Higher Education, 32:6 pp 601 – 611  Nicol, R. & Macfarlane-Dick, D. (2006) ‘Formative assessment and self-regulated learning: a model and seven principles of good feedback practice.’ Studies in Higher Education. 31:2 pp 199 – 218 Nicol, R. (2008) ‘Transforming assessment and feedback: Enhancing integration and empowerment in the first year.’ Quality Assurance Agency, Scotland. Sadler, D. R. (1989) ‘Formative assessment and the design of instructional systems’. Instructional Science 18 pp 119 - 144  Walker, M. (2009) ‘An investigation into written comments on assignments: do students find them usable?’ Assessment & Evaluation in Higher Education. 34:1 pp 67 - 78 EATING 2011