Effective (and Efficient) Evaluation and Feedback: An Approach and Some Tips for Outpatient Teaching Louis Pangaro MD, F

1. Effective (and Efficient) Evaluation and Feedback: An Approach and Some Tips for Outpatient Teaching Louis Pangaro MD, FACP Educational Programs Medicine, USUHS

2. some definitions • Assessment = making the observation • “sitting next to” • Evaluation = assigning value • not grading; uses words • based on goals • diagnoses what “step” they’re at

3. some definitions • Feedback = “course correction” • not feeling good • based on evaluation • get them to “next step”

4. Community Goals(“ends”) Curriculum(“means”, “objectives”) Evaluation individual programmatic Feedback

5. Evaluation vs.. Grading Grading “gradus” steps (F, D, C, B, A) Evaluation “value” strength

6. Teacher Tasks • Diagnose the Patient • Diagnose the Learner • Teach One Thing the Learner Needs to get to “the Next Step” • Diagnose the Teacher (reflection) • at the end of clinic with feedback from the student

7. How do you decide what the “next step” is for a learner?

8. Diagnosing the Learner: goals (“analytic”) • attitudes • skills • knowledge

9. AAMC Med School Objectives Project (MSOP) • Goals*: a Physician is: • Dutiful • Altruistic • Skillful • Knowledgeable * each goal has multiple objectives

10. Knowing the goalsfor the learner’s level of training • A “synthetic” approach R.I.M.E. System - Reporter - Interpreter - Manager/Educator

11. R.I.M.E. SystemUSUHS Medicine Reporter: reliably get the facts, works with patients, identifies problems, Interpreter: prioritizes, reasonable differential diagnosis, more confidence Manager: works with patients on Diagnostic and therapeutic Plan Educator: a plan for the doctor: gets to the next level of knowledge and evidence

12. What would you expect of the learner? • clinic patient : a 45 year old woman with hypertension; routine lab studies showed a total cholesterol of 340 md/dl. • if the learner is a third year student? • an intern in August? • a senior resident?

13. Efficient: a question of “how” • “doing the thing right” • Effective: a question of what • “doing the right thing”

14. Teaching Tip • One thing at a time • With “beginners”, Don’t try to do it All • What’s the agenda for feedback?

15. Kinds of Feedback • Minimal • “good”, “ugh!”, a shrug or nod • Behavioral • “that was good because…” • “you can improve by…” • Interactive • let them react, or better yet, self-evaluate after Stanford Faculty Dev Program

16. HOW to Give Effective Feedback • be specific (what they did, not who they are) • be timely, but watch the setting • “positive” too; a “feedback sandwich” (catch the learner doing something right) • label it “feedback” • have an action plan • use “interactive” method after SFDP

17. What To Give Feedback On • do you agree with their diagnosis/management • making the transitions ? (RIME) • their self-evaluation (is it similar to yours?) • an action plan for their time (give it now or after clinic)

18. How to Assess a learner Quickly student: mainly observation/listening • pertinent positives/negatives • problem list (semantic competence) resident: observation but mainly questions • “How can I help?

19. “Semantic Competence” • Bordage’s term for active use of correct terms that show pattern awareness. • e.g. “This patient has a distal, symmetric, polyarthritis.” implies a student has the knowledge and skill to move to “interpreter” • options: listen to summary statement, or ask for “problem” list

20. Expecting complete problem lists • Problem lists quickly reveal: • comprehensiveness • level of semantic competence • ability to synthesize • movement toward “interpreter” level

21. The “heart” of our method • Internal Medicine teaches a method to embrace complexity, so one can act with simplicity. • The Problem List reveals the learner’s ability to analyze a case (semantic competence) and synthesize it (embrace complexity) [residents, too?]

22. Internal Medicine Clerkships • Different from others in third year: embedding the present problem in larger context (necessary for High Pass and certainly for Honors level) • Dy’s “cogito”: I think, therefore I.M.

23. Teaching TipThe “One Minute” Preceptor • Five “Microskills” for Clinical Teaching A disciplined approach to pacing from the University of Washington, Seattle

24. The “One Minute” Preceptor • Start with a Commitment from the Learner • Probe for Supporting Evidence • Teach General Rules • Positive Feedback for what was right • Correct Mistakes

25. Intern and High Cholesterol • Intern: This lady should be started on pravastatin • Teacher: Why? • Intern: She has a risk of heart disease • Teacher: What else would you do? • Intern: Bring her back in three months

26. Intern and High Cholesterol • Teacher: Generally, be sure of your diagnosis first. You’re right that cholesterol is a risk for heart disease, but you need to be sure about causes like hypothyroidism. • Teacher: Let me tell you what tests and therapy I’d like you to order today, and what you should read about tonight.

27. Risks in complete problem lists • we try to cover it all • our compulsiveness wins out • we bring up all the interesting, fascinating, aspects of a great case • we try to do the inter-relations for the student • we start to lecture the student/resident

28. Teaching Tiplet the student do the work • Transferring incentive and initiative to the student • can we incentivize efficiency in the student??

29. What motivates students to do the work? MotivationsBarriers

30. Teaching Tip Let the Learner Do the Work • Don’t lecture about what can be learned by reading • Save Extensive Knowledge Transfer for the Learner’s Own Time • give the student a chance at High Pass

31. let the student do the work • Preparation : • in advance identify patients to see • how was the diagnosis made, how treated • what is our agenda for coming visit? • write down what to ask, to look for, labs to check (index card)

32. Working with the Patient • Student memorizes the lines: “Mrs. J----------, I’ve had a chance to review your record and I believe you’re here to follow-up on your diabetes. .. Is there anything else you’d like us to go over in the 20 minutes we have together?”

33. “reporter” stage interview/exam show only basic knowledge and skill student excused (or observer) and attending offers A&P to the patient student given homework and tasks “interpreter” stage interview/exam show good focus on pertinent info student allowed to offer assessment presence of patient? attending offers plan student excused or observer managing the educational goals

34. Teaching Tip : “How Can I Help?” • Force residents to move to Manager/ Educator Level (first, diagnose the learner) • Ask them to self-evaluate and determine their educational needs • Let them Know their Growing Independence is Important to You • “How Can I Help?”

35. Summary: Five Tips 1. Know the “next step” for this learner (RIME); 2. “One thing at a Time” 3. Practice the “One Minute Preceptor” 4. For students: let them do the work 5. For residents: “How can I help?”