Physical Examination Education in Graduate Medical Education – A Systematic Review of the Literature

1. Physical Examination Education in Graduate Medical Education – A Systematic Review of the Literature Somnath Mookherjee, MD Lara Pheatt, MA Sumant R. Ranji, MD Calvin Chou, MD, PhD SDRME Summer Meeting 2012

2. Importance of PE skills? • Fletcher RH, Fletcher SW. Has medicine outgrown physical diagnosis? Ann Intern Med. 1992;117(9):786-7. • Flegel KM. Does the physical examination have a future? CMAJ. 1999;161(9):1117-8. • Jauhar S. The demise of the physical exam. N Engl J Med. 2006;354(6):548-51. • CXR from “The Practioner, 1904” • "Laennec examines a consumptive patient with a stethoscope in front of his students at the Necker Hospital". Painting by ThéobaldChartran. • Physical diagnosis: a guide to methods of clinical investigation. George Alexander Gibson, William Russell. D. Appleton & Co., 1891. New York. • A pocket book of physical diagnosis: for the student and physician. Edward Tunis Bruen. P. Blakiston, 1881. Philadelphia

3. (Un)importance of PE skills? ACGME Internal Medicine program requirements

4. Resurgence of PE • Physical examination is still important • Hypothesis based • Evidence based • Verghese, A., Culture shock--patient as icon, icon as patient. N Engl J Med, 2008. 359(26): p. 2748-51. • Verghese, A., A touch of sense. Health Aff (Millwood), 2009. 28(4): p. 1177-82. • Yudkowsky, R., et al., Residents anticipating, eliciting and interpreting physical findings. Med Educ, 2006. 40(11): p. 1141-2. • Yudkowsky, R., et al., A hypothesis-driven physical examination learning and assessment procedure for medical students: initial validity evidence. Med Educ, 2009. 43(8): p. 729-40. • McGee, S., Evidence Based Physical Diagnosis. Second Edition ed2007, St. Louis: Saunders - Elsevier. • David L. Simel and D. Rennie, The Rational Clinical Examination: Evidence Based Clinical Diagnosis. 2009, New York: McGraw-Hill Professional.

5. But skills are inadequate ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓

6. Why focus on GME? • Different than students • Little “protected time” for didactics • Constant time pressure • Mainly experiential learning • Skills needed for imminent practice • New program requirements

7. Developmental Milestones for Internal Medicine Training – Patient Care Green ML, Aagaard EM, Caverzagie KJ, Chick DA, Holmboe E, Kane G, et al. Charting the road to competence: developmental milestones for internal medicine residency training. Journal of graduate medical education. 2009;1(1):5-20.

8. Research Questions • What teaching methods are used? • What assessment methods are used? • What teaching methods are effective? GME Program Curricula Individual practices of teachers

9. Search strategy • Search 1 = "physical examination"[MESH] AND "Education, Medical"[MESH] • Search 2 = physical examination AND "Education, Medical"[MESH] • Search 3 = physical examination AND (resident OR intern OR graduate OR residents OR interns OR graduates) AND (teaching OR learning OR education OR teach OR learn) • Search 4 = "physical examination"[MESH] AND (resident OR intern OR graduate OR residents OR interns OR graduates) AND (teaching OR learning OR education OR teach OR learn) • Personal files

10. Eligibility • English language • Inclusion of a description of the study population; number of participants and level of training • Description of an educational intervention  intention of improving PE skills • Inclusion of assessment of efficacy • Inclusion of a clear comparison group • Report of data analysis (descriptions of outcomes without statistical analysis were not included) • Study subjects enrolled in GME

11. Data Extraction • Study Quality • Medical Education Research Study Quality Instrument (MERSQI) • Based on Best Evidence Medical Education Collaboration protocol • Nation • Type of physical examination • Level and numbers of learners • Summary of intervention • Human examinees • Deliberate practice • Summary of outcomes • Benefit to learner • Assessment methods

12. Deliberate Practice? • Repetitive performance of skills by the learner. • Assessment of skills by the teacher. • Specific feedback to the learner by the teacher. • Observation of improved performance in a controlled setting. • Papers independently scored for the presence of these elements • 0 = not reported • 1 = reported • Global deliberate practice score • 0 = no use of deliberate practice or unable to determine • 1 = possible use of deliberate practice • 2 = definite use of deliberate practice • Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Academic medicine : journal of the Association of American Medical Colleges. 2004;79(10 Suppl):S70-81. • McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Medical education featuring mastery learning with deliberate practice can lead to better health for individuals and populations. Academic medicine : journal of the Association of American Medical Colleges. 2011;86(11):e8-9. • McGaghieWC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Academic medicine : journal of the Association of American Medical Colleges. 2011;86(6):706-11. • DuvivierRJ, van Dalen J, Muijtjens AM, Moulaert VR, Van der Vleuten CP, Scherpbier AJ. The role of deliberate practice in the acquisition of clinical skills. BMC medical education. 2011;11(1):101.

13. Educational Outcomes • Kirkpatrick Level • Level 0 = no assessment of impact • Level 1 = assessment of reaction to the intervention • Level 2a = assessment of attitudes or perceptions • Level 2b = assessment of knowledge or skills • Level 3 = assessment of changes in behavior • Classification • “X” = not measured, or not compared to a control group • “0” = not better than control group • “1” = beneficial (intervention group with significantly better outcome than control)

14. Statistics and Analysis • Inter-rater reliability • Individual elements of the MERSQI scores • Individual elements of deliberate practice • Global deliberate practice score • Study quality • Average and median MERSQI scores with standard deviations using consensus scores • Narrative synthesis • Group review of tabulated summaries of studies

15. Search and Selection of Articles 7250 Articles Identified and Screened 1543 "physical examination"[MESH] AND "Education, Medical"[MESH] 1943 physical examination AND "Education, Medical"[MESH] 2038 physical examination AND (resident OR intern OR graduate OR residents OR interns OR graduates) AND (teaching OR learning OR education OR teach OR learn) 1695 "physical examination"[MESH] AND (resident OR intern OR graduate OR residents OR interns OR graduates) AND (teaching OR learning OR education OR teach OR learn) 31 Hand search and expert review 7095 Rejected Based on Initial Screening Criteria (is the study about PE education in GME?) 155 Citations Meeting Initial Screening Criteria 141 Citations Removed 109 Duplicates 32 Not Meeting Full Inclusion Criteria 14 Articles Included in Systematic Review

16. Study characteristics

17. Study Quality - MERSQI Score • Perfect inter-rater agreement [kappa = 1.0 (95% CI = 1.0, 1.0)] for all but two items. • “Sampling” [kappa = 0.44 (95% CI = -0.16, 1.0)] • “Content validity” [kappa = 0 (95% CI = -0.52, 0.52)]. Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research. JAMA. 2007;298(9):1002-1009.

18. Interventions 14 Studies No resident interaction w/ human examinee, n = 7 • Criley 2008: Web-based cardiac tutorial • Horiszny 2001: Multimedia cardiac lecture • Iversen2006: Multimedia cardiac lecture • Mangione 1994: PE elective, lectures • Keren 2005: Brief cardiac audio tutorial • Oddone 1993: Cardiology simulator • Smith 2005: MSK small group session Resident interaction w/ a human examinee, n = 7 Interaction w/ patients in a clinical context Teaching associates Patient volunteers • Branch 1999: Arthritis patient educator. • Herbers 2003: Pelvic exam patient educator • Leder 2005: Pelvic exam during clinical rotation • Smith 2006: Cardiac exam, bedside rounds • Freund 1998: Breast exam, breast care clinic • Rabinovitz 1987: Pelvic exam, adolescent medicine rotation • Houck 2002: Thyroid exam on volunteer patient in workshop Single session

19. Assessments Multi-media No interaction w/ human examinee, n = 6 • Criley 2008: Computer program to assess cardiac exam • Horiszny 2002: Recorded heart sounds • Keren 2005: Recorded heart sounds 14 Studies • Freund 1998: Chart review for frequency of breast exam • Leder 2005: Chart review for genital exam Chart review Resident interaction w/ a human examinee, n = 8 Survey • Rabinovitz 1987: Survey for pelvic exam confidence Objective Structured Clinical Examinations Teaching associates Patient Volunteers • Branch 1999: MSK exam, patient educators • Herbers 2003: Pelvic exam, patient educators • Oddone 1993: Cardiac exam testing, also used simulator • Iversen 2006: Cardiac exam testing, patient volunteers • Houck 2002: Thyroid exam OSCE • Mangione 1994: Multiple PE type OSCEs • Smith 2006: Cardiac exam OSCE • Smith 2005: MSK exam OSCE Previously published assessment tool

20. Outcomes • What teaching methods are used? • What assessment methods are used? • What teaching methods are effective?

21. Outcomes – KL 3 • No evident pattern for the superiority of one educational setting over another • Bedside teaching, simulator, lecture, workshop

22. Outcomes – KL 2b • No evident pattern that spending more time results in better outcomes • Elective rotation, multiple lecture series, single workshop, single lecture

23. Outcomes – KL 2b • Little clarity or consistency in what PE competence entails and how to measure it • Technique vs. accuracy • Detection vs. diagnosis

24. Deliberate Practice and Outcome • Inter-rater reliability of components of DP score • Repetitive performance of skills by the learner • [kappa=0.86 (95% CI=0.59, 1.0)] • Assessment of skills by the teacher • [kappa=0.72 (95% CI=0.38, 1.0)] • Specific feedback to the learner by the teacher • [kappa=0.71 (95% CI=0.33, 1.0)] • Observation of improved performance in a controlled setting • [kappa=0.81 (95% CI=0.46, 1.0)] • Global deliberate practice score • [kappa=0.76 (95% CI=0.46, 1.0)]

25. Limitations • Deliberate practice assessment non-validated • Effect sizes neither calculated nor meta-analyzed • Small number of studies with heterogeneous outcome measurements

26. Findings • No convincing evidence for one setting: “going to the bed-side” vs. “going to the simulator-side” • Not just the “time spent,” but the “time well spent”

27. Findings • Deliberate practice is well-suited to teach PE in GME • Interaction with human examinees may be beneficial

28. Recommendations • UME PE education systematic review underway • Evaluate the use of deliberate practice and human examinees to teach PE • Develop a GME PE blueprint • Specific PE skills graduating residents should have • What competence in these skills entails • How these skills are best taught and evaluated

29. Acknowledgements • Society of Directors of Research in Medical Education • UCSF Division Of Hospital Medicine • UCSF – OME – Teaching Scholars Program

30. Outcomes – KL 1

31. Outcomes – KL 2a