390 likes | 537 Views
Experience Examined. Feedback and Evaluation in Clinical Learning Catherine Lucey, MD. Goal. Increase the comfort with which participants assess, provide feedback to, and complete evaluations on their learners. “Clinical Learning is Experience Examined”. Ende and Davidoff, 1992.
E N D
Experience Examined Feedback and Evaluation in Clinical Learning Catherine Lucey, MD
Goal • Increase the comfort with which participants assess, provide feedback to, and complete evaluations on their learners
“Clinical Learning is Experience Examined” Ende and Davidoff, 1992
Two Critical Roles • Coach: Feedback/ Formative Evaluation • Information provided during the rotation, describing performance, with the intent to guide future performance. • Judge: Summative Evaluation • Judgement provided at the close of a rotation assessing whether the learner met performance standards.
Residents : Help! How often do you get?: Isaacson et al. Cleveland Clinic Foundation, 1995
Programs : Help! • 2/3s of Program Directors feel that accuracy of faculty evaluations are major concerns • APDIM Precourse, April 1999 • 50% of Program Directors had a recent troubling feedback/eval session • APDIM Plenary, October 1999
What are the Barriers? Brainstorming Session
Barriers: Fairness, Uncertainty, Time • Will my evaluation be reliable? • “ I don’t know what learners today are supposed to be able to do at this level...” • Will my evaluation be valid? • “I didn’t see them do ‘x’ enough to be able to judge fairly…” • “I can’t remember …”
Barriers: Sensitivity Concerns • Will my feedback be well received? • “what if they cry?” • “There goes my teaching award….” • How will my feedback and evaluation be used? • “I don’t want to ruin a promising career…”
Overcoming Barriers • Knowledge • Expectations of the rotation • Understanding of the tools • Attitudes • Faith in the process and the learner • Skills • Targeted observation and • Constructive Feedback
Road Map for A Successful Rotation Expectations Formative Eval Summative Eval
Case Study Program: University of Anywhere Learner: Terry, a PGY-1 Preceptor: Dr. Andersen, a general internist in practice on the eastern shore of Maryland. S/he knows Terry’s program director from residency. The program director remembers Dr. Andersen as a smart doctor and good teacher.
Beginning of the Month • Choose Your Focus • Choose The Right Tools • Announce your feedback intentions
1. Choose your Focus: Rotation Specific Objectives Overall Clinical Competence
Examples • Knowledge: • Offer a prioritized differential diagnosis for common ambulatory complaints such as CP, SOB, Dizziness, Edema……. • Skills: • Use and document telephone encounters for dx and thx follow-up…...
2. Choose Your Evaluation Tools • Standard Tools For Assessment • Supervised Direct Patient Care • Case Presentations • Standard Tool for Measurement: • Global Rating Scales • Behaviorally Anchored Scales • Developmentally Anchored Scales
BARS *: The Final Pathway *Behaviorally Anchored Rating Scales • ABIM: 123 456 789 unsatisfactory satisfactory superior • Dartmouth poor fair good excellent Must be improved Room for improvem Adequate Skills Superior Skills
Developmental Scales Difficulty despite Does with assistance Often does w/o assist Usually does w/o assist Always expert, w/o assist
Other Tools to Consider • Medical Record Audit • Chart stimulated recall • Mini CEX • Standardized Patients • Nursing and Patient Evaluations
3. Announce Your Feedback Intentions • Why will feedback be given? • Be all you can be... • When will feedback be given? • Micro-feedback • Macro-feedback or Formative Evaluation • How will feedback be given?
Heart of the Rotation • Use Micro-feedback skills to coach the learner: • What did they do well? • Where do they need to improve? • What is the next step for them?
Micro-feedback shapes behavior • Timely: Case by Case, 1st Hand • Focused: Choose your pearls • Expected: Part of the culture • Applied: Next step, next patient, goal directed • Documented: Progress Notes
How to give effective micro-feedback • Look for behaviors to reinforce as well as correct • Be specific about what you saw • Avoid speculating on intentions • Use nonjudgmental language • Encourage self evaluation • Identify next steps
Reinforcing Feedback • “I was impressed that you gave Mrs.. Jones plenty of time to tell her story.” (specific and reinforcing) • When you are patient like that, you often get a more accurate history (nonjudgmental and reinforcing) • The next step for you is to practice using clarifying questions for symptoms such as chest pain.
Corrective Feedback • “I found a very large liver on Mr. Smith, why do you think you underestimated its size?” (specific, encourages self evaluation) • “When you examine for a liver, it is important to undress the patient.” (nonjudgmental, forces self evaluation) • “As a next step, let’s go over how you position your hands”
Embarrassment and Humiliation • Humiliation: Externally driven • generally negative experience • avoidable with skilled feedback techniques • Shame/Embarrassment: Internally driven • neither entirely avoidable nor entirely negative
Feedback Progress Notes • Action: Document case by case impressions • Goal: Identify patterns of strengths and weaknesses in multiple encounters • Outcome: More specific formative and summative evaluations
Group Exercise • Using the scenarios in your handout, role play micro-feedback and progress note completion • Switch roles to allow each individual to experience the role of feedback provider
Midpoint Formative Evaluation= Macro Feedback Coaching Role Continues ?
Macro-Feedback • Use patterns identified during micro-feedback • Concretely compare to the agreed upon goals • Plan of action to achieve the end goals • more specific = more achievable
Skill Set: Communicating Evaluations • Ask: How am I doing as a teacher? • Ask: What do you see as your strengths; then weaknesses? • Tell: Summarize your observations and ideas • Ask: What is the take home message, next steps?
Group Exercise • Practice the Ask, Ask,Tell, Ask format of leading an evaluative session • Check your action plans for specificity
End of the Month Summative Evaluation: Did they meet standards? Met Exceeded Failed
Skill Set: Pitfalls in Evaluation • Range Restriction • Hawk/Dove • Halo Effect
Group Exercise • Complete the Attached evaluation form, using the notes that you have taken during the role play sessions.
Conclusions • Feedback and Evaluation are critical components of successful clinical learning • Optimal performance of these skills requires collaboration between the program and preceptor
Conclusions • Less is Frequently More: • Choose your focus for evaluation • Provide feedback frequently, in small bits of manageable information • Next steps should be measurable • Feedback is a two way street