Peer feedback in formative assessment to aid learning:

1. Peer feedback in formative assessment to aid learning: Annie Cushing Angela Hall Olwyn Westwood Centre for Medical Education s CETL 2008

2. s CETL 2008 Drivers • Professional Skill – Multisource Feedback as part of professional practice • NSS – students want feedback • Formative feedback as a support for learning • Large student numbers • Proximity of practice/feedback/practice

3. Study 1 in Nov 2008Graduate Entry Students: Year 1: n=93Medical & Nursing [Adult, Child, Mental Health] 3 Stations Based on PEBL cases 5-minute station with 3-min feedback Actors play patient/carer

4. Study 1: Nov 2008 ( 78 attended = 84%) Students: • Generate the marking criteria* • Undertake all OSCE stations as ‘candidates’ • Undertake ‘examiner’ role in 1 station. • Actors give verbal feedback Evaluation by: Questionnaire*: 20 statement items Likert response scale Focus Group: nominal group technique. * Adapted from FAST project Open Univ/Sheffield Hallam (Brown et al 2003)

5. Q1.Being the examiner was helpful in providing clear instructions on what was expected ofme(N=77)

6. Q5 The feedback was rapid and helpfulN=78

7. Q13. I found it difficult giving feedback to my colleagues. (p <0.05)

8. Q20. I felt that I was able to be honest in my feedback

9. Focus Group (7)Helpful to Learning Seeing examples of: good and bad communications (content & delivery) (5 votes) differences between people in the same situation (5 votes) others’ behaviour helps you understand what you should do yourself (3 votes)

10. Unhelpful to Learning • Checklist is too long (7 votes) • Peers are ‘amateurs’, both at the skills being role-played and at feedback (5 votes) • Some people are not brave enough in giving critical feedback (4 votes)

11. Anxiety about giving Negative Feedback “I thought, ‘You really should never say that in front of somebody’, and I didn’t, to be honest, have the guts to tell them.” “You want to say something positive but also you want to get across what they need to improve. ………. But it can be quite difficult, I think, because if you’re peers, it's harder than if you're a tutor.”

12. Feedback Sargeant J et al (2008) J Contin Educ Health Prof Three inter-related factors for acceptance of negative feedback: • Credibility – specificity, personal observation, explicit standards of performance • Emotion-when inconsistent with self perceptions of being a ‘good doctor’ • Reflection – a good mediator of assimilation, acceptance and application

13. Incorporating Feedback • “I thought I'd done something quite well, and someone pointed out that actually I could have done it this way and it might have been better. And then seeing someone else do it, I thought, ‘Oh well, yeah, that’s a much better way of doing it’. But it hadn't even occurred to me that I could improve on the way I was doing it. And I don't think it's until someone points it out to you in quite a specific way that you think, ‘Well actually, yeah, I’ll try that’. And I've done it since, and it is better.”

14. Learning Transfer “With a patient who wanted a long conversation, I had to draw on the OSCE session. ‘OK, how do I make sure that I'm actually quite interested in what you're saying now?’ Because I've got three other patients that I really need to deal with.’” (Nurse)

15. Study 2: May 2009Changes in response to Study 1 Teaching on ‘Giving Feedback’ (video CETL). What would you give feedback on? - generate a marking scheme How does the facilitator do with her feedback? - generate principles of feedback Behavioural, specific, balanced +ve with areas to improve.

16. Changes in Study 2In response to Study 1 More feedback time: 4-mins interview and 4-mins feedback Actor to give written rather than verbal feedback Empathy Clarity Cases to be circulated in advance.

17. Feedback

18. Responses to Questionnaire Statements [1]

19. Responses to Questionnaire Statements [2]

20. Free Text Comments “I understand it was part of the research but it definitely would have been valuable to have been given verbal feedback from the actor as well as written” “Really good – nice to see that we have improved since the last one, both in interviewing and giving feedback to colleagues” “Peer feedback: I think it is very helpful, although we are relatively at the same level so limited in depth of feedback” “Love the OSCE sessions with the actors. They are invaluable and a great learning tool”.

21. IIntroduced into Curriculum’08 MBBS 2009-10 Core Curriculum in Year 3 • Both Communication (3) and Clinical (3) stations: • Faculty supported, supervised and guided sessions on peer feedback engage learners actively • Learners need training on giving constructive feedback • Learners value these sessions • Resourcing – cost effectiveness

22. Spin-offs • www.bbc.co.uk/health • BSc Med Ed Effect of intervention on student performance in end of year summative OSCE marker station. Sample: GEP yr 1 cohort (formative OSCE) Graduates in Year 2

23. Thank you Any Questions?

24. References Sargeant J et al. 2007 Challenges in multi-source feedback: Intended and unintended outcomes. Med Educ. 41:583-591 Nicol & MacFarlane (2004) Rethinking Formative Assessment in HE: a theoretical model and seven principles of good feedback practice Boud, D. (2000). Sustainable assessment: rethinking assessment for the learning society. Studies in Continuing Education. 22 (2),151-167. Kurtz S, Silverman J. 2005. Teaching & Learning Communication Skills in Medicine. Radcliffe Pub Oxford. (Calgary-Cambridge Guide) Weaver M. (2006) Do students value feedback? Student perceptions of tutors’ written responses, Assessment and Evaluation in Higher Education 31(3) pp. 379-394 Bing-You R.G., Paterson, P., Levine, M.A. (1997) Feedback falling on deaf ears: residents’ receptivity to feedback tempered by sender credibility, Medical Teacher, 19, pp. 40-44

25. Cost effectiveness 120 students All got chance to: • Examine 6 peers • Practice 6 stations • Be a patient in 2 stations Tutors = 8 ‘Bodies’ = 12 Actor patient = 4