1 / 20

Assessment Careers: towards a vision of post-modularisation

Assessment Careers: towards a vision of post-modularisation . Dr Gwyneth Hughes Dr Martin Oliver Dr Holly Smith SRHE Conference 2012. Some current problems with feedback. Feedback rarely scrutinised, when it is practice is inconsistent

elden
Download Presentation

Assessment Careers: towards a vision of post-modularisation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessment Careers: towards a vision of post-modularisation Dr Gwyneth Hughes Dr Martin Oliver Dr Holly Smith SRHE Conference 2012

  2. Some current problems with feedback • Feedback rarely scrutinised, when it is practice is inconsistent • Lack of learner engagement with and understanding of feedback (Lizzio & Wilson, 2008) • Inefficiency in terms of effort and impact • Transmitted feedback creates dependency on teacher (Carless et al. 2011)

  3. Praise is not helpful to students unless accompanied by detail on why the praise is deserved. Encourages dependency on others (Hattie & Timperley, 2007). • Praise-critique does not encourage dialogue and self-reliance (Nicol & Macfarlane-Dick, 2005) • Lack of information about progress which could be motivating (Hughes, 2011) • Critique and advice are for the current assignment. Encourages focus on short-term grade improvement rather than longer-term learning.

  4. Assessment Careers Project: a response to modularisation • Year 1: Baseline reporting and feedback analysis • Year 2 (2012/13): Five pilot programmes • Year 3 Institutional implementation • Assessment Careers: Institute of Education website: • www.ioe.ac.uk/assessmentcareers and video clip: • http://youtu.be/VSaGbPoXPh0

  5. Tool to identify feedback categories • P1 Giving praise • P2 Recognising progress or ipsativefeedback • Critique • C1 Correction of errors • C2 Factual criticisms • C3 Criticism of approach • Giving advice • A1 Specific to content current assignment • A2 General skills in current assignment • A 3 For future assignments • Q Clarifications and questions • O Other unclassified statements • Adapted from Orsmond & Merry, 2011 including Hughes, 2011. • Adapted from Orsmond, & Merry (2011) using Hughes, (2011)

  6. Analysing feedback categories • The score is the number of times a classification appears in the feedback • The default unit for analysis was the sentence • Where a sentence contains clauses that make distinct points, it was split into separate clauses, each of which was classified separately. • Neutral comments that for example describe the piece of work, but do not make any judgement are unclassified.

  7. Data • Analysed formative and summative assessment feedback for modules on 5 postgraduate programmes at the IOE (total 228 pieces) • Recorded the total number of comments in each category and the average per script • Ranked the categories to obtain a feedback profile at programme level as well as an aggregate profile of the 5 programmes.

  8. Baseline feedback profile

  9. Baseline feedback profile

  10. What do students do with feedback?

  11. Overview of interventions • Pilot 1 MA Education, Health Promotion and International Development • Pilot 2 MRes in Educational and Social Research • Pilot 3 MA Clinical Education  • Pilot 4 MA/MSc Psychology of Education • Pilot 5 PGCE Primary • Approx. 400 students and 30 staff in total.

  12. Assignment cover sheets • Thinking about the feedback on your draft of this essay, please indicate what the key points were and what action you took to respond to this feedback to help you prepare for and write this essay I would like tutor feedback on: 1. 2.  3.

  13. Learner reflections on assessment careers • An early intervention, to inform pilots • Postgraduate module, 18 students • Peer assessment element: critical review of a reading • Week 1: feedback from tutor on reviews • Week 2: feedback from peers on reviews, tutor provides feedback on feedback • Week 3-7: feedback from peers • Post-experience reflective discussion about the process

  14. Examples of peer feedback • I think you presented this quite clearly. For me online feedback has its place, however, its is less personal and sometimes this makes it easier to critique someone. I do believe that it could suffer from 'general feedback' where teachers, lecturers offer general feedback having assessed a few pieces. Personally I believe it could be quite beneficial and effective. Peer assessment, computer assisted assessment together with adaptive release grades will definitely become part of our practice, it is only a matter of time. • What disadvantages, if any did you find in any of these? • If these are not the future, what alternatives do you suggest?

  15. Examples of peer feedback • Overall you have written a really comprehensive and detailed review of the document. Your strength lies in your fantastic ability to contextualise the document with your political and pedigogical references. […]What do you understand by the term disruptive in this context? […] You seem to suggest ( or I am interpreting wrongly) that disruptive in this context is bad? […]Finally, you quote and cite almost in every sentence- but what is your personal view of 'the rise of K-12 blended learning'in primary and secondary education. Where is your personal voice? It scares me - or at least having a purely distance online learning component for such students worries me. What about you?

  16. Experience varied: some not positive • I share others opinions who indicate that it would be beneficial to have more frequent feedback from our instructor on our postings rather than from our classmates […] it is much more difficult to give usefull feedback on a critical review that you have not even read.

  17. Others were more positive • Training us to give feedback to each other was a great idea. Granted, it didn't always work - it showed how hard it is to get the right balance between constructive criticism vs motivating encouragement. It's not really helpful if someone just focuses on being encouraging, but sometimes I really worried if I'd been demotivating by giving too many suggestions for improvement. Sometimes I wanted to give feedback to someone, but just couldn't work out what to say! But giving feedback to others made me develop a list of things to look for, which I immediately realised I wasn't doing in my own summaries!

  18. Conclusion • Need to move beyond modularised assessment • Pre-pilot showed that some students (not all) can be supported in developing their capacity to give and act on feedback • Pilots set up to facilitate longitudinal feedback in diverse ways • The Assessment Careers project will evaluate the success of pilot interventions in enabling students and staff to develop feedback beyond the module level

  19. References • Carless, D. Slater, D.Yang, M. and Lam, J. 2011. Developing sustainable feedback practices. Studies in Higher Education 36, no.4: 395-407. • Gibbs, G. & Simpson, C. (2004) Conditions Under Which Assessment Supports Students’ Learning. Learning and Teaching in Higher Education, 1: 3-31 • Hattie, J. & Timperley, H. 2007. The Power of Feedback. Review of Educational Research 77 no. 1: 81-112. • Hughes, G. (2011) Aiming for Personal Best: a Case for Introducing Ipsative Assessment in Higher Education Studies in Higher Education 36 (3): 353 – 367

  20. Lizzio, A. & Wilson, K. 2008. Feedback on assessment: student’s perceptions of quality and effectiveness. Assessment & Evaluation in Higher Education 33 no.3:263-275. • Nicol, D. & Macfarlane,-Dick, D. 2006. Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education 31 no. 2: 199-218. • Orsmond, P. & Merry, S. 2011. Feedback alignment: effective and ineffective links between tutors’ and students’ understanding of coursework feedback. Assessment & Evaluation in Higher Education. 36(2): 125-126.

More Related