Evidence based feedback in supervision, R2C2

1. Evidence based feedback in supervision, R2C2 Joan Sargeant PhD Professor, Medical Education Dalhousie University, Halifax, Nova Scotia, Canada National LIS Conference, Oslo, Norway November 5, 2018

2. Lunenburg, Nova Scotia DATE, ADDITIONAL DETAILS (set this text using “Header & Footer)

3. My goals for the presentation • Review evidence for effective feedback and coaching in clinical supervision • Describe a culture/ learning environment supportive of effective feedback, coaching and learning • Describe the R2C2 feedback and coaching model, and identify phrases to use at each stage • Consider the case of Chris, a resident, and how to have a feedback and coaching conversation with him DATE, ADDITIONAL DETAILS (set this text using “Header & Footer)

4. Consider Chris, a resident… • Chris is a resident at the end of his first year in Internal Medicine. He has been on your service for the past week and will be there for several weeks. • He seems to have a good knowledge base, but you wonder about his ability to integrate his knowledge and come up with a diagnosis and treatment plan. • He seems hesitant. He also missed a diagnosis of diabetic ketoacidosis earlier in the week for a sick patient. • You are wondering about how to have a feedback conversation with him, and how to coach him to improve. DATE, ADDITIONAL DETAILS (set this text using “Header & Footer)

5. Feedback evidence- what do we know? DATE, ADDITIONAL DETAILS (set this text using “Header & Footer)

6. What is feedback? An older definition - “information” • Specific information about the comparison between a trainee’s observed performance and a standard, given with the intent to improve the trainee’s performance” (van de Ridder, 2008) • It’s about a gap

7. More recent thinking about feedback – “a process” • A 2-way conversationwith a learner/colleague about their performance data, with the goal of enabling them to • develop, • improve, and • become the very best they can. *(Archer 2010, Telio 2012, Sargeant 2017)

8. Feedback in clinical settings is complex

9. 1. Supervisor: Characteristics of effective feedback • Specific • Timely • On observed behaviour • About behaviour that can be changed • Related to a standard or competency level • Seeks the learner’s views • Demonstrates interest in the learner • Coaches for improvement

10. Supervisor: Criteria for effective feedback • Is based on observation • For credibility and for usefulness to the resident: • “I don’t pay attention to that supervisor’s feedback as he’s never actually observed me” (resident) • To be able to coach for improvement, as in sports or music “Observations are the currency of feedback, and without them the process becomes “feedback” in name only.” Jack Ende(1983) DATE, ADDITIONAL DETAILS (set this text using “Header & Footer)

11. Criteria for effective feedback(continued) 2. Is provided in relation to a standard • The outcome, expected competence level • Shared between supervisor and resident • “I don’t know what ‘good’ looks like, I don’t have a picture in my mind of how I should be doing this” (resident) DATE, ADDITIONAL DETAILS (set this text using “Header & Footer)

12. Criteria for effective feedback (continued) • Is specific, uses words and description, not numbers • “Being a 4/5 tells me nothing, it doesn’t tell me how to improve” • Provide specifics of what was observed and what needs to be improved, and what was done well; coach • “You normally get very vague responses like, ‘You did that OK’, or, ‘You did that very well’. But no one ever takes it that extra step to describe what you did well”—Resident 5 Think of a sports or music coach DATE, ADDITIONAL DETAILS (set this text using “Header & Footer)

13. 2. Resident considerations • Residents want to appear confident and competent • Therefore may not ask for feedback • And may not accept constructive feedback very well • Yet residents feel they don’t get enough feedback “I’ve never tried to ask for feedback because…you want to portray confidence.”—R3 “ Sometimes I think the reason why I take bad feedback so personally, is because we get so little feedback. So you feel like…you must be awful, like the worst doctor ever.”—R2 DATE, ADDITIONAL DETAILS (set this text using “Header & Footer)

14. 3. Culture • Cultures, contexts and systems which often do not support assessment and feedback for learning DATE, ADDITIONAL DETAILS (set this text using “Header & Footer)

15. The culture for effective feedback – “a way of thinking” • Assessment and feedback for learning and development • Not just of learning • How to promote such a developmental culture within Competency-based Education (CBME)? DATE, ADDITIONAL DETAILS (set this text using “Header & Footer)

16. Culture: CBME is about promoting increasing levels of development and competence

17. Developmental Stages (Dreyfus 1980)

18. Ericsson: Deliberate Practice… • Has well defined, specific goals • Put steps together to achieve a well-defined goal • Is focused • Involves feedback • Requires getting out of one’s comfort zone • …However, “trying hard and pushing yourself to the limit isn’t enough.” Need feedback/ guidance • Ericsson & Lehmann, 1996

19. Competency-based Medical Education (CBME) and the Role of Feedback and Coaching Feedback and Coaching Expert Proficient Dreyfus and Dreyfus, 1980 Carraccio et al., 2002 Competent Advanced Beginner Novice

20. What is coaching in CBME? • Why are we beginning to think about coaching in CBME, and not just assessment and feedback? DATE, ADDITIONAL DETAILS (set this text using “Header & Footer)

21. Coaching is focused on outcomes • And – • Needs the feedback /performance data from observation • Is longitudinal • Just think of sports coaches

22. “Coaching requires a shift in thinking and philosophy… ….It isn’t about the finite exercise of teaching something to someone, but rather about the infinite exercise of continuous improvement. The best athletes and musicians trust in the power of coaching to push them to be better than they imagined they could be.” Watling 2018 DATE, ADDITIONAL DETAILS (set this text using “Header & Footer)

23. Competency- based education requires a shift in philosophy • Focus on learning and growth • Observe learners with the goal of sharing that observation to help them get better/ grow (i.e., coaching) • Engage learners in their own development • Creation of culture and relationships to support this: • Safe • Respectful • Humanistic • Positive DATE, ADDITIONAL DETAILS (set this text using “Header & Footer)

24. Development of the R2C2 feedback model (Sargeant et al) • Arose from this evidence about challenges in feedback • Influenced by CBME • Developed through research to answer the question - “How can we promote the provision of feedback in a way that will enhance it’s acceptance and use for improvement, and promote ongoing development? “ DATE, ADDITIONAL DETAILS (set this text using “Header & Footer)

25. 4 Stage Facilitated R2C2 Feedback Model • Sargeant et al, 2015, 2016 2017, 2018

26. Purpose of the R2C2 feedback model • Overall purpose: Promote feedback use and performance improvement – • Ask questions to • Encourage the learner to reflect on their performance and feedback • Seek the learner’s self-assessment, and their perspectives and ideas • Promote feedback acceptance • Foster self-direction for improvement (and guide as needed) • Coach for change

27. Stage 1. Build relationships Goal: To engage the resident, build respect and trust, understand their context: • “How is this rotation going for you? Tell me about what you are enjoying, what challenges you?” • “What are your goals for this week?” • “What are you trying to work on?” • “Tell me how you think you did in performing X” ? • “What didn’t go as well as you hoped?” • “What went well?”

28. Stage 2. Explore reactionsto the feedback and emotional responses Goal: For resident to feel understood and that his/her views are heard and respected. • “What do you think of my observations/ feedback?“ • “What were your initial reactions?” • “Did anything surprise you? Tell me more about that...” • “How do my observations/ feedback compare with how you think you were doing? Any surprises?” • “It’s difficult to hear feedback that disconfirms how we see ourselves”

29. Stage 3. Explore understanding of the content Goal: For the resident to be clear about what the feedback mean for his/her performance and the opportunities identified for change and development. • “What are your thoughts now?” • “Was there anything that I said that didn’t make sense to you?” • “Anything you’re unclear about?” • “What struck you as something to focus on?” • “From this, what do you see to work on?”

30. Stage 4. Coach for performance change Goal: For the resident to engage in developing an achievable learning/change plan • “If there was one thing that you would target for immediate action, what would it be?” • “In X time, is there anything you would like to see changed?” • “What would be your goal?” • “What actions will you have to take?” • “What might help you with this change?” • “What might get in the way?” • “What resources do you need? • “Do you think it’s achievable?”

31. Stage 4: Coach—Learning/Change Adapted from Wakefield J, Herbert CP, Maclure M Dormuth C, Wright JM, Legare J, Brett-MacLean P, Premi J, Commitment to change statements can predict actual change in practice, J ContEduc in Health Prof 2003; 33:81-93.

32. Feedback facilitation goal • Internal control • my data to use • my opportunity • confidence • “I’m in control” • External control • external feedback • lack of control • threat • emotion - • “bad me” Facilitator: listen, accept, motivate, coach

33. Feedback in clinical settings is complex R2C2 Feedback And Coaching Model

34. Summary and take-home messages Coaching promotes improved future performance Feedback cultivates insight into current performance

35. Summary: Again, consider Chris, a resident… • Chris is a resident at the end of his first year in Internal Medicine. He has been on your service for the past week and will be there for several weeks. • He seems to have a good knowledge base, but you wonder about his ability to integrate his knowledge and come up with a diagnosis and treatment plan. • He seems hesitant. He also missed a diagnosis of diabetic ketoacidosis earlier in the week for a sick patient. • How would you conduct the feedback conversation with him, and coach him to improve? DATE, ADDITIONAL DETAILS (set this text using “Header & Footer)

36. Thank you! • Your comments and questions? Joan.sargeant@dal.ca DATE, ADDITIONAL DETAILS (set this text using “Header & Footer)

37. R2C2 Research program: Assessment, feedback, coaching 2002- 2007: Doctors did not accept negative feedback as it disconfirmed their own self-assessments of how they thought they were doing. They reacted emotionally to it. 2008-2010:Self-assessment: However, students, resident and doctors said they needed feedback to accurately self-assess. Led to a conundrum. 2011- 2017:R2C2: Testing a feedback model that will enhance feedback acceptance and use, and promote coaching. 2018-?: Understanding specific coaching skills and their use at the bedside R2C2 DATE, ADDITIONAL DETAILS (set this text using “Header & Footer)

38. R2C2 Research Team and Funding Partners: • Dalhousie University, Halifax, Nova Scotia • University of Toronto, Toronto, Ontario • University of Alberta, Calgary, Alberta • Colleges of Physicians and Surgeons of NS and ON, Canada • Maastricht University, Maastricht, The Netherlands • Hackensack University Medical School, New Jersey, USA • American Board of Internal Medicine, Philadelphia, USA • Accreditation Council for Graduate Medical Education, Chicago, USA Research grants:

39. References • R2C2 and self-assessment • Sargeant J, Lockyer J, Mann K, Holmboe E, Silver I, Armson H, Driessen E, MacLeod T, Yen W, Ross K, Power M. Facilitated reflective performance feedback: Developing an evidence and theory-based model. Acad Med, 2015;90(12):1698-706 • Sargeant J, Armson H, Driessen E, Holmboe E, Könings K, Lockyer J, Lynn L, Mann K, Ross K, Silver I, Soklaridis S, Warren A, Zetkulic M, Boudreau M, Shearer C. Evidence-Informed Facilitated Feedback: The R2C2 Feedback Model. MedEdPORTAL. 2016;12:10387. https://doi.org/10.15766/mep_2374-8265.10387 • Sargeant J, Lockyer JM, Mann K, Armson H, Warren A, Zetkulic M, Soklaridis S, Könings KD, Ross K, Silver I, Holmboe E, Shearer C, Boudreau M. The R2C2 Model in Residency Education: How Does It Foster Coaching and Promote Feedback Use?AcadMed. 2018;93(7):1055-1063 • Sargeant J, Mann K, Manos S, Epstein I, Warren A, Shearer C, Boudreau M. R2C2 in action: testing an evidence-based model for facilitating feedback and coaching for improvement in residency education. Journal of Graduate Medical Education 2017; 9(20): 165-170. • R2C2 You-Tube Videos (developed for Post Graduate Medical Education): • Video 1 uses competency based language: https://youtu.be/_cSDQYjUEok • Video 2 uses generic language: https://youtu.be/-ljhCWYujks • Sargeant J, Armson H, Chesluk B, Dornan T, Eva K, Holmboe E, Lockyer J, Loney E, Mann K, van der Vleuten C. The processes and dimensions of informed self-assessment: a conceptual model. Acad Med. 2010 Jul;85(7):1212-20.

40. References • Feedback • Boud D, Malloy E (eds). Feedback in higher and professional education: understanding it and doing it well. London and New York: Routledge; 2013 • Watling C, Driessen E, van der Vleuten CPM, Vanstone M, Lingard L. Beyond individualism: professional culture and its influence on feedback. Med Educ. 2013;47(6):585–94. • Lefroy J, Watling C, Teunissen PW, Brand P. Guidelines: the do's, don'ts and don't knows of feedback for clinical education. Perspect Med Educ. 2015 Dec;4(6):284-99. doi: 10.1007/s40037-015-0231-7. If you go into pubmed—you can access it directly. • Ajjawi R, Regehr G. When I say … feedback. Med Educ. 2018 Oct 21. doi: 10.1111/medu.13746. [Epub ahead of print] • Coaching • Heen S, Stone D. Managing Yourself- Finding the coaching in criticism: The right way to receive feedback. HBR. Jan–Feb 2014; 108-111. • Gawande A. Personal best: Top athletes and singers have coaches – should you? The New Yorker;2011, Oct 2. http://www.newyorker.com/reporting/2011/10/03/111003fa_fact_gawande • Van Neiuwerburgh C, An introduction to coaching skills: A practical guide, 2nd ed, London: Sage, 2017.  • Watling C. Coaching for CBME: Lessons form sport and music. Jan, 2018 http://www.schulich.uwo.ca/about/competencybased_medical_education/school_news_information_and_leadership_messages/coaching_for_cbme_lessons_from_sport_and_music.html

41. References • Planning for change • Mazmanian PE, Daffron SR, Johnson RE, Davis DA, Kantrowitz MP. Information about barriers to planned change: a randomized controlled trial involving continuing medical education lectures and commitment to change. Acad Med. 1998;73(8):882-6. • Armson H, Elmslie T, Roder S, Wakefield J. Is the Cognitive Complexity of Commitment-to-Change Statements Associated With Change in Clinical Practice? An Application of Bloom's Taxonomy. J ContinEduc Health Prof. 2015;35(3):166-75. • Overton GK, MacVicar R. Requesting a commitment to change: conditions that produce behavioral or attitudinal commitment. J ContinEduc Health Prof. 2008;28(2):60-6 • Implementation Science and enabling change • Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci. 2016. 11:33. • Kitson AL, Harvey G. Methods to Succeed in Effective Knowledge Translation in Clinical Practice. J NursScholarsh. 2016;48(3):294-302. • Person-Centred Psychological and Motivational Theories • Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A, et al. Making psychological theory useful for implementing evidence based practice: a consensus approach. Quality and Safety in Health Care 2005;14(1):26-33. • Rogers C. Freedom to learn: studies of the person. Columbus, OH: Charles E. Merrill Publishing Company; 1969.