1 / 35

Contemplating the Assessment of Competencies Beyond Medical Knowledge (and Patient Care)

Contemplating the Assessment of Competencies Beyond Medical Knowledge (and Patient Care) John (Jack) R. Boulet, PhD FSSH Vice President, Research and Data Resources ECFMG/ FAIMER Fall 2018 Coalition for Physician Enhancement (CPE) Meeting October, 2018 Washington, DC. Disclosures.

tedg
Download Presentation

Contemplating the Assessment of Competencies Beyond Medical Knowledge (and Patient Care)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Contemplating the Assessment of Competencies Beyond Medical Knowledge (and Patient Care) John (Jack) R. Boulet, PhD FSSHVice President, Research and Data ResourcesECFMG/ FAIMERFall 2018 Coalition for Physician Enhancement (CPE) MeetingOctober, 2018Washington, DC

  2. Disclosures • I’m not a physician (but have taken, or been involved in creating, many of the assessments required for licensure) • I’m a “difficult” patient • Evidence-based medicine should be supported by evidence-based assessment • “It’s tough to make predictions, especially about the future” • Yogi Berra

  3. Objectives • Explore how the required medical competencies have changed over time • Implications for the curriculum/ practice • Propose some “new” competencies • Discuss how current assessments are not well-aligned with the evolving competencies • Validity issues • Suggest a (partial) way forward • Technology

  4. Movement Towards Competency-Based EducationWhat was it before?

  5. The Needs of the System Should Drive Curriculum

  6. The Problem is Figuring out the “Competencies”(and how to assess them)

  7. Competencies of Entering Medical Students

  8. ACGME Competencies • Many of these competencies are difficult to define … and even more difficult to assess • We believe in a model (perhaps unrealistically) that competence is property of the individual • What about the system? • Teams?

  9. Other Frameworks

  10. The Problem of Competency Contamination • Competencies overlap • Definitions vary • CANMEDs • ACGME • GMC • Tomorrow’s Doctors • Need to be very explicit about what you want to measure • Professionalism, Situational Awareness, etc.

  11. The Past and the Future (of Medicine)

  12. Knowledge, Skills, and Attitudes that are Needed?

  13. The Practice of Medicine • Competencies that will be critical to assess in medical education (AMA) • Inquiry and improvement (“dealing with uncertainty with big data”) • Interdependency (“working in teams”) • Information management (“informatics”) • Interest and insight (“patient-centered care”) • Involvement (“adding life to years and not just years to life”) • How well do we assess these competencies?

  14. Comfort with Technology

  15. Adaptability • Employ unique analyses • Develop innovative methods for obtaining or using resources • Entertain new ideas • Actively seek out and carefully consider the merits of new approaches • Deal with ambiguity • Change plans, goals, actions or priorities to deal with changing situations

  16. Adaptable … and Technologically Savvy • Artificial intelligence (AI) will not replace physicians. Physicians who use AI will replace those who do not • Medical students will need to be able adopt new innovations • Need to be able to deal with “information overload” • There are over 30 million medical articles in PubMed

  17. Curiosity • “I have no special talents. I am only passionately curious” • Albert Einstein • For doctors, curiosity is fundamental to understanding each patient’s unique experience of illness • Common practices in medical education may inadvertently suppress curiosity

  18. Teamwork • The provision of quality care depends on teams • More than 70% of medical errors are attributable to dysfunctional team dynamics • Health Care Management Review (2014) • Teamwork is rarely assessed in medical school

  19. Knowledge Access • Medical knowledge doubles every few months • How can clinicians keep up? • Inadequate access to information limits clinicians’ ability to deliver healthcare • Ability to go and get the (correct) information • Use of ‘smart’ clinical search engines

  20. Communication (with informed patients) • Patients are much more informed • They can contribute to their own care • Discordance between patient’s and physicians’ diagnosis could create mistrust (for patient) and frustration (for physician) • Positive interrelationships between physicians and patients will be much more difficult to maintain • How we communicate in the future could be quite different

  21. Communication (with other providers) • Communication with patients may be quite different from communication with peers/ other providers • Communication is not a “singular” competency • Inter-disciplinary communication skills will be become even more important

  22. Kindness, Compassion, Empathy • Regardless of technology, practitioners will still need to be kind • Patients look to physicians for expertise, guidance and compassion • Unkind treatment negatively impacts patient care • Medical students need to develop compassion and empathy • How to measure?

  23. Current Problems with Assessment • Assessments have not kept pace with evolving competencies • How much “stored” knowledge must a practitioner have? • Often, validity evidence lacking • We often measure what is easy to measure • Scoring models

  24. Purpose(s) of Assessment often not Clear • Identification of poor performers or (and) shifting the performance curve?

  25. Purpose of Assessment(s)

  26. Misalignment of Competencies and Assessment? • United States Medical Licensing Examination (USMLE) • Step 1 (basic science) • Step 2CK (clinical knowledge) • Step 2CS (clinical skills) • Step 3 (application of medical knowledge) • MCC • QE I and QE II • Do these assessments reflect what is needed to practice?

  27. Measurement Issues (precision?)

  28. Scoring of Assessments • Do the scores reflect the domain of interest?

  29. What we do not Measure? • History taking • Physical examination • Communication (with the patient) • Clinical Decision Making • What’s missing • Teamwork • Resource management • Consultation • Trauma • Longitudinal care

  30. New Developments (Technology)

  31. Technology-Enhanced Scoring • Explicit outcome measures (policy capturing) • Code actions and timing based on machine readable output • Natural Language Processing (NLP) • Post simulation encounter exercises, essays • Analysis of Talk Time • Speech recognition • Automated Computer Vision Analysis • Empathy, rapport

  32. Discussion • The physician’s role will be even more critical in regard to the introduction, evaluation and best use of these technologies • role as “health advisor and knowledge navigator” • Computer technology is not the problem but the solution • The use of diagnostic, management, data mining and summarization algorithms is/will drastically alter(ing) medical education and assessment • AI in medicine (and assessment) will be a “team sport” predicated on a set of new competencies (statistics, computer sciences, etc.)

  33. Summary • Evolution of learning models & technologies not completely mirrored by similar changes in educational assessment • Assessment is still mostly “physician-centric, individually tailored” with traditional modalities (MCQs, OSCEs, static/linear representations of materials on screen) • Educational assessment must evolve alongside learning models/technologies

  34. Questions?

More Related