Evaluating the Principal Clinical Experience: A Progress Report

1. Evaluating the Principal Clinical Experience: A Progress Report Ed Krupat, PhD Medical Education Grand Rounds May 18, 2007

2. The PCE Pilots • Cambridge Integrated Clerkship--begun July 2004 • Brigham & Women’s Hospital--begun July 2005 • Beth Israel Deaconess Medical Center--begun July 2005 • Massachusetts General Hospital--begun July 2006

3. The Principles • Longitudinality • with faculty and house staff • with patients • of feedback • of curriculum • Mentoring • Interdisciplinary perspectives • Integration of basic and clinical sciences • Patient-centered approach • Student-centered learning

4. Plans & Methods • Mentoring programs • Planned curriculum • Tutorials • Students present cases • PCC (with physicians who admit to the hospital) • PD III at site • Writing/reflection

5. The Evaluation Plan • Comprehensive • Experiences • Knowledge & knowledge retention • Skills • Attitudes/perceptions • Self-assessment • Multi-method • Quantitative • Qualitative • Scientifically rigorous • Tied directly to pilot program goals

6. The Design • Comparison/control group recruited from students doing traditional rotations • Pre-clerkship measures demonstrated no differences on • MCAT scores • Step I scores • PD II OSCE scores • Career preferences • Attitudes toward patient care

7. Characterizing the Experience

8. “How well would you say the following adjectives describe your clerkship experience:Frustrating”(1= Not At All & 6= Perfectly) ’05-’06 Clerkships

9. “How well would you say the following adjectives describe your clerkship experience:Confidence-building”(1= Not At All & 6= Perfectly) ’05-’06 Clerkships

10. “How well would you say the following adjectives describe your clerkship experience:Humanizing” ’05-’06 Clerkships

11. “How satisfied are you with:The overall quality of your clerkship experience”(1= Extremely Dissatisfied & 6= Extremely Satisfied) ’05-’06 Clerkships

12. Performance: Knowledge

13. Shelf Scores: CIC vs. ControlYear 1 (2004-2005)

14. Shelf Exam

15. Shelf Exam

16. Shelf Exam

17. Shelf Exam

18. Performance: Skills

19. Comprehensive OSCE

20. Comprehensive OSCE

21. Comprehensive OSCE

22. ComprehensiveOSCE Skill ScoreHistory ’05-’06 Clerkships

23. ComprehensiveOSCE Skill ScorePhysical Exam ’05-’06 Clerkships

24. Attitudes & Perceptions about the Year

25. “To what extent have your experiences prepared you:To have the knowledge base necessary to be a competent practitioner”(1= Very Poorly & 6= Very Well) ’05-’06 Clerkships

26. “To what extent have your experiences prepared you:To integrate basic sciences and clinical practice”(1= Very Poorly & 6= Very Well) ’05-’06 Clerkships

27. “To what extent have your experiences prepared you:To practice evidence-based medicine”(1= Very Poorly & 6= Very Well) ’05-’06 Clerkships

28. “To what extent have your experiences prepared you:To see how the social context affects patients and their problems”(1= Very Poorly & 6= Very Well) ’05-’06 Clerkships

29. “To what extent have your experiences prepared you:To relate well to a diverse patient population”(1= Very Poorly & 6= Very Well) ’05-’06 Clerkships

30. “To what extent have your experiences prepared you:To know your strengths and limitations”(1= Very Poorly & 6= Very Well) ’05-’06 Clerkships

31. Attitudes & Perceptions about Patient Care and Life on the Wards

32. Patient-Practitioner Orientation Scale(PPOS) ’05-’06 Clerkships

33. Patient-Practitioner Orientation Scale (PPOS) Pre Post Fall ’05 Spring ‘06 Site 1 5.14 5.13 Site 2 4.89 4.83 Site 3 5.14 5.12 Control 4.90 4.57

34. “During this past clerkship year, I observed residents encouraging patients’ participation in their own care.”(1= Never & 7= Always) ’05-’06 Clerkships

35. “Your team is rounding on a patient in his hospital room when one of the consulting services arrives for the patient. Your attending and the consulting attending proceed to talk about the patient’s case as if the patient weren’t there.”(1= Very Often & 5= Never) ’05-’06 Clerkships

36. “In general, when I made an effort to develop rapport with my patients, my instructors _____ me.”(1= Completely Discouraged & 7= Completely Encouraged) ’05-’06 Clerkships

37. “In general, when I made an effort to get to know patients as unique persons, my instructors _____ me.”(1= Completely Discouraged & 7= Completely Encouraged) ’05-’06 Clerkships

38. Focus Group Findings • Valued-added for the PCE students • Continuity with faculty • Access to hearing how faculty think about clinical issues • Mentoring • Peer group support • Sense of belonging to something larger than self

39. “Today I was in a public restroom at _____,and I noticed how there was paper all over the floor. For a moment I was disgusted and I thought about all the residents and physicians who work so hard to give top notch care. I wouldn’t want patients or their families to use the bathroom at ____ and feel this hospital is dirty when hospitals are supposed to be clean! So I picked up all the trash. At that moment, I realized that I had taken ownership of this hospital.”

40. Value-Added for the Faculty • Contact and meaningful cross-discipline discussion • Ability to get to know students well • Personally satisfying • Ability to identify strengths and weaknesses • Ability to provide feedback and assist in student growth and development

41. Remaining Challenges Paradox of the Pullout Longitudinal, interdiscipinary curriculum Attendings vs residents?? More observation & better feedback

42. Challenges (cont.) • Patient contact pre-workup • Longitudinal contact with patients • Quality of the individual clerkships • Consistency • Across disciplines within site, within discipline across sites, and across PCEs