Medical Licensure in the United States Assessment of Clinical Skills

1. Medical Licensure in the United States Assessment of Clinical Skills Ann C. Jobe, MD, MSNExecutive DirectorClinical Skills Evaluation Collaboration(CSEC)

2. Raising the Bar in Geriatrics Assessment 2010 Reynolds Grantee 9th Annual Meeting October 25-27, 2010 St. Louis, Missouri (sample of slides from presentation)

3. Clinical Skills Evaluation CollaborationCSEC • A Collaborative Partnership, established in 2003, between the Educational Commission for Foreign Medical Graduates (ECFMG) and the National Board of Medical Examiners (NBME)

4. A Look Back in Time • USMLE - introduced in early 1990’s • Single examination pathway for initial medical license (graduates of US and international medical schools) • A national standardized series of exams to assure minimal competency

5. United States Medical Licensing Examination (USMLE) • Each of the three Steps of the USMLE complements the others • No Step can stand alone in the assessment of readiness for medical licensure.

6. United States Medical Licensing Examination (USMLE) • Step 1 • understanding and application of important concepts of the sciences basic to the practice of medicine • Multiple choice exam • Step 2 • application of medical knowledge, skills, and understanding of clinical science essential for the provision of patient care under supervision • Clinical Knowledge (CK) • Multiple choice exam • Clinical Skills (CS)

7. United States Medical Licensing Examination (USMLE) • Step 2CS • Standardized patients used to assess an examinee’s ability to • gather information from patients, • perform physical examinations, • communicate their findings to patients and colleagues

8. United States Medical Licensing Examination (USMLE) • Step 3 • application of medical knowledge and understanding of biomedical and clinical science essential for the unsupervised practice of medicine • Multiple choice exam and computerized case simulations (CCS)

9. Life Cycle of a Physicianin the United States American Board of Medical Specialties (ABMS) Specialty Boards Board Certification Exams Recertification q 7-10 yrs NRMP Match Medical School Maintenance of Certification Postgraduate Training 3-7 yrs Fellowship Practice Year 1 Year 2 Year 3 Year 4 Maintenance of Licensure Step 1 Step 2 CK Step 2 CS Step 3 Licensure Relicensure q 1-3 yrs United States Medical Licensing Examination (USMLE) State Licensing Authorities

10. Miller’s Pyramid Action Performance Competence Knowledge DOES SHOWS HOW KNOWS HOW KNOWS

11. Kirkpatrick Criteria • Results Change in organizational practice Benefits to patients/clients • Behavior Transfer learning to workplace Learners apply new knowledge and skills • Learning Change attitudes/perceptions Change knowledge/skills • Reaction Customer satisfaction related to participation in educational activities

12. COMPETENCY • “Core Competencies” Accreditation Council for Graduate Medical Education (ACGME) American Board of Medical Specialties (ABMS) • Patient Care • Knowledge • Communication and Interpersonal Skills • Professionalism • Systems-Based Practice • Practice-Based Learning and Improvement

13. Correlations among Step scores

14. USMLE Step 2 Clinical Skills • Mastery of clinical and communication skills, as well as cognitive skills, by individuals seeking medical licensure is important to the protection of the public. (from USMLE Bulletin of Information)

15. Step 2 CS Examinees

16. Step 2 CS Fees • USMG – increase of $100(10%) over 6 year period • 2004-2006: $975 • 2007: $1,005 • 2008: $1,025 • 2009: $1,055 • 2010: $1,075 • IMG • 2004-2009: $1,200 • 2010: $1,295

17. USMLE Step 2 Clinical Skills • The cases cover common and important situations that a physician is likely to encounter in clinics, doctors’ offices, emergency departments, and hospital settings in the United States.

18. Presentation categories • Include, but are not limited to, cardiovascular, constitutional, gastrointestinal, genitourinary, musculoskeletal, neurological, psychiatric, respiratory, and women's health. • Examinees will see cases from some, but not all, of these categories. • The selection of cases is also guided by specifications relating to acuity, age, gender, and type of physical findings presented in each case.

19. Each Step 2 CS “session” includes 12 twenty-five minute patient encounters. • 15 minutes with patient • 10 minutes for patient note • The examination lasts approximately 8 hours. Two breaks are provided: • 1st break is 30 minutes long (lunch) • 2nd break is 15 minutes long (snack).

20. Step 2 CS Components • Communication and Interpersonal Skills (CIS) • Spoken English Proficiency (SEP) • Integrated Clinical Encounter (ICE) • Data gathering (DG) • History & PE • Patient note (PN)

21. Step 2 CS Components • USMLE Step 2 CS is a Pass/Fail examination • Each of the three subcomponents (CIS, SEP, ICE) must be passed in a single administration in order to achieve a passing performance on Step 2 CS

22. Assessment of Communication and Interpersonal Skills (CIS) • CIS performance is assessed by the standardized patients • a global rating of these skills using a series of generic rating scales • same CIS scale for all 12 encounters • 3 sub-components: • Information gathering (questioning skills) • Information sharing • Professional manner and rapport

23. Assessment of Communication and Interpersonal Skills (CIS) Questioning skills - examples include: • use of open-ended questions, transitional statements, facilitating remarks • avoidance of leading or multiple questions, repeat questions - unless for clarification, medical terms/jargon unless immediately defined, interruptions when the patient is talking • accurately summarizing information from the patient

24. Assessment of Communication and Interpersonal Skills (CIS) Information-sharing skills - examples include: • acknowledging patient issues/concerns and clearly responding with information • avoidance of medical terms/jargon unless immediately defined • clearly providing • counseling when appropriate • closure, including statements about what happens next

25. Assessment of Communication and Interpersonal Skills (CIS) Professional manner and rapport - examples include: • asking about • expectations, feelings, and concerns of the patient • support systems and impact of illness, with attempts to explore these areas • showing • consideration for patient comfort during the physical examination • attention to cleanliness through hand washing or use of gloves

26. Assessment of Communication and Interpersonal Skills (CIS) Professional manner and rapport - examples include: • providing opportunity for the patient to express feelings and/or concerns • encouraging additional questions or discussion • making • empathetic remarks concerning patient issues/concerns • patient feel comfortable and respected during the encounter

27. Assessment of Spoken English Proficiency (SEP) • SEP performance is assessed by the standardized patients • using rating scales; same scale for all 12 encounters • based upon • frequency of pronunciation or word choice errors that affect comprehension • amount of listener effort required to understand the examinee's questions and responses • clarity of spoken English communication within the context of the doctor-patient encounter (e.g., pronunciation, word choice, and minimizing the need to repeat questions or statements)

28. Scoring of the Step 2 Clinical Skills Subcomponents The ICE subcomponent includes assessment of: • Data gathering (DG) - patient information collected by history taking and physical examination • Documentation (PN) - completion of a patient note summarizing the findings of the patient encounter, diagnostic impression, and initial patient work-up

29. Scoring of the Step 2 Clinical Skills Subcomponents • Data gathering (DG) performance is assessed by the standardized patients • using checklists developed by committees of clinicians and medical school clinical faculty • checklists comprise the essential history and physical examination elements for each specific clinical encounter

30. Scoring of the Step 2 Clinical Skills Subcomponents • The patient note is rated/scored by trained physician raters • The patient note (PN) consists of three areas • Medical History and Physical Examination • Differential Diagnosis • Diagnostic Workup

31. Scoring of the Step 2 Clinical Skills Subcomponents • Scored holistically • Relevant and correct information • Congruency/consistency with specific case –based scoring guidelines • Integration/synthesis of information • Organization, coherence, cohesiveness, flow, legibility

32. Performance on Step 2 CS • Failure rate for USMGs • 2004-2005: 4% • 2005-2006: 2% • 2006-2007: 3% • 2007-2008: 3% • 2008-2009: 3% • Failure rate for IMGs • 2004-2005: 17% • US Citizens: 11% • Foreign Citizens: 18% • 2005-2006: 15% • 2006-2007: 23% • 2007-2008: 28% • 2008-2009: 27%

33. Performance on Step 2 CS • Passing rate for USMGs (first takers) • 2005-2006: CIS >99%; SEP >99%; ICE 98% • 2006-2007: CIS 99%; SEP 100%; ICE 97% • 2007-2008: CIS 99%; SEP >99%; ICE 98% • 2008-2009: CIS 99%; SEP >99%; ICE 98%

34. Performance on Step 2 CS • Passing rate for IMGs (first takers) • 2005-2006: CIS 93%; SEP 98%; ICE 89% • 2006-2007: CIS 87%; SEP 99%; ICE 85% • 2007-2008: CIS 81%; SEP 92%; ICE 86% • 2008-2009: CIS 84%; SEP 94%; ICE 84%

35. For more information about USMLE policies and procedures, and about specific exams – please go to: • www.usmle.org