Competency Common Language: or, getting you, your residents, and your faculty on the same page

1. Competency Common Language: or, getting you, your residents, and your faculty on the same page Tom Cooney, MD, MACP Vice Chair for Education Oregon Health & Science University

2. Objectives • Define the educational concepts and terms you will (may) use in creating your new assessment system: • CBME • Competence • Milestones • EPA • Provide a common language for communicating with your faculty and residents • Identify challenges to accomplishing the goals of competency-based medical education (CBME)

3. Competencies and Milestones and EPA, Oh My!

4. We’re off to see the wizard….

5. Gaps in Individual Physician Readiness for “Unsupervised Faculty” • Office-based practice competencies • Inter-professional team skills • Clinical information technology skills • Population management skills • Reflective practice and CQI skills • Care coordination • Continuity of care • Leadership and management skills • Systems-thinking • Procedural skills Crosson et al. Health Affairs, 2011

6. Imperative for Education Redesign • Medical education has not sufficiently responded to: • Shifting patient expectations and demographics • Changing health care delivery systems • Quality improvement • Use of new technologies • Summary: Medical education is not meeting nation’s health care needs • Weinberger et al. Ann Int Med, 2010 • IOM Crossing Quality Chasm • IOM Health Professions Education

7. Flexner2.0? “Medical education is at a crossroads: those who teach medical students and residents must choose whether to continue in the direction established over a hundred years ago or to take a fundamentally different course, guided by contemporary innovation and new understandings about how people learn.” Cooke M, Irby DM, O’Brien BC Carnegie Foundation, c2010

8. Competency-Based Medical Education(CBME)

9. Competency-Based Medical Education CBME is an outcomes-based approach to the design, implementation, assessment and evaluation of a medical education program using an organizing framework of competencies. The International CBME Collaborators Frank et al. Med Teach, 2010

10. Traditional vs. CBME Adapted from: Carracchio et al. Acad Med, 2002

11. CBME approach • Learner-centered • Individualized, formative, mentor/coach • Competency-based • Pre-defined knowledge, skills and attitudes • Focused on learner abilities and not capabilities • Observable behaviors • Outcomes-driven • Measurable and improvable

12. Competency An observable ability of a health professional, integrating multiple components such as knowledge, skills, values and attitudes. Example: ACGME General Competencies The International CBME Collaborators, 2009

13. Competent Possessing the required abilities in all domains in a certain context at a defined stage of medical education or practice. Example: satisfactorily places CVC in simulation lab The International CBME Collaborators Frank et al. Med Teach, 2010

14. Competence Competence entails more than the possession of knowledge, skills and attitudes; it requires you … to apply these [abilities] in the clinical environment to achieve optimal results. ten Cate, Med Teach, 2010

15. Competence Put simply, competence is: • Synthetic. It requires the bringing together, simultaneously, of multiple abilities; • Contextual It varies with and is influenced by circumstances, settings, and culture, and requires one to know what to bring to a given situation

16. CBME is a new paradigm We must know the trainee has demonstrated competence and is ready to progress to the next stage of their career: • Requires clear definition of expected competencies (i.e. thing they need to do) • Requires assessment to determine whether these things are done consistently and within the contextual needs of the clinical environment

17. CBME v1: ACGME Competencies • Patient Care and Procedural Skills • Medical Knowledge • Practice-based Learning and Improvement • Interpersonal and Communication Skills • Professionalism • Systems-based Practice

18. ACGME: Systems-Based Practice subcompetencies • Residents must demonstrate an awareness of and responsiveness to the larger context and system of health care, as well as the ability to call effectively on other resources in the system to provide optimal health care • work effectively in various health care delivery settings and systems relevant to their clinical specialty • coordinate patient care within the health care system relevant to their clinical specialty • incorporate considerations of cost awareness and risk-benefit analysis in patient and/or population-based care as appropriate • etc

19. Struggles to transition to CBME • Programs had trouble moving from traditional framework (structure/process) to competency framework (outcomes) • Limitations with the core competencies framework • concepts were abstract and difficult to to teach and evaluate • competency framework implied an independence among the competencies that does not exist • framework did not reflect the synthetic nature of competence and the activities of physicians caring for patients

20. Struggles to transition to CBME “Those who educate medical students and physicians work in a world suffused with the concept of competency. This article examines the intellectual origins and hidden assumptions of this concept and argues that it is an inadequate, and even harmful, concept to use as a guiding motif for professional education. The competency model-- which tends to be top-down and prescriptive-does not provide the framework for for objective educational assessment that it claims to provide. The alternative apprenticeship model is more appropriate for professional education and is more consistent with what psychological research has shown about the acquisition of expertise.” Brooks M Persp Bio Med 2009

21. Challenges with the ‘Competencies’ “ The peer-reviewed literature provides no evidence that current measurement tools can assess the competencies independently of one another. Because further efforts are unlikely to be successful, the authors recommend using the competencies to guide and coordinate specific evaluation efforts, rather than attempting to develop instruments to measure the competencies directly “ Acad Med. 2009; 84:301–309.

22. Milestones

23. Milestone 1. A stone marker set up on a roadside to indicate the distance in miles from a given point. 2. An important event, as in a person's career, the history of a nation, or the advancement of knowledge in a field; a turning point.

24. The Milestones movement and ACGME-NAS • In 2007, the ACGME and American Board of Internal Medicine (ABIM) began to develop a set of internal medicine milestones • Goal: “explicate the 6 ACGME general competencies by describing a developmental progression of observable behaviors. • Organized within the ACGME competencies framework to give more practical and clinical relevance to the core competencies. • Provide more specific feedback and evaluation to residents and ensure that they acquire the necessary knowledge, skills, and attitudes for advancement • Result: 142 discrete IM Milestones

25. N Engl J Med 2012; 366: 1051-6

26. A key element of the NAS is the measurement and reporting of outcomes through the educational milestones… N Engl J Med 2012; 366: 1051-6

27. Accreditation: ACGME/RRC Institution and Program Learners Program Aggregation • Assessments within Program: • Direct observations • Audit and performance data • Multi-source FB • Simulation • ITExam CCC: Synthesis and Judgment Reporting Milestones No Aggregation Certification Board?? Faculty, PDs and others

28. What are these “Milestones”? N Engl J Med 2012; 366: 1051-6

29. NAS ‘Reporting’ Milestones:aka, The Educational Milestones ACGME NAS-FAQ http://www.acgme-nas.org

30. Educational (‘reporting’) Milestones • The milestones define the abilities expected of residents as they progress through training • Integrate knowledge, skills, values and attitudes • Developmental in nature • Inherently linked within/across core competencies • Framed in behavioral terms • They are observable • Sets the stage for assessment of competence

31. Internal Medicine Milestones

32. Anesthesiology Milestones

33. Pediatric Milestones

34. General Surgery Milestones

35. Orthopedic Surgery Milestones

36. Specialty Milestones Snapshot Sullivan G, Simpson D, Cooney T, Beresin E. A Milestone in the Milestones Movement: the JGME Milestones Supplement. JGME: 2013, 5(1):1-4

37. Criticisms of Milestones • Milestones are reductionist • Potential to deconstruct the learners’ performance into discreet tasks or checklists • Checking off milestones does not equate to competence • There are 100s of milestones embedded in the structure of the Reporting Milestones ! • Programs cannot assess all of them (even over 3-6 years)

38. Accreditation: ACGME/RRC Institution and Program Learners Program Aggregation • Assessments within Program: • Direct observations • Audit and performance data • Multi-source FB • Simulation • ITExam CCC: Synthesis and Judgment Reporting Milestones No Aggregation Certification Board?? Faculty, PDs and others

40. What are the demonstrated outcomes? • Superior? Above average? • Performs better than peers? • Appropriate for level of training? • Ready to enter unsupervised practice? • Do they match your expected outcome?

41. Accreditation: ACGME/RRC Institution and Program Learners Program Aggregation • Assessments within Program: • Direct observations • Audit and performance data • Multi-source FB • Simulation • ITExam CCC: Synthesis and Judgment Reporting Milestones No Aggregation Certification Board?? Faculty, PDs and others

43. Exercise • At each table, please discuss with your group what see as your current gaps in preparing for reporting NAS Milestones on your residents and fellows: • Faculty understanding and readiness • Resident/fellow understanding • Current assessment system and data generated • Residency Management System (data management)

44. Entrustable Professional Activities “… identify the critical activities that constitute a specialty – all the elements that society and experts consider to belong to that profession, the activities of which we would all agree should be only carried out be a trained specialist.” ten Cate et al AcadMed 2007

45. Entrustable Professional Activities “Patients’ and instructors’ … entrustment of responsibility to a trainee is an essential concept in this approach because they reflect the most important outcome of postgraduate training: a trainee’s readiness to bear professional responsibility” ten Cate et al AcadMed 2007

46. An Entrustable Professional Activity • Part of essential work for a qualified professional • Requires specific knowledge, skill, attitude • Acquired through training • Leads to recognized output • Observable and measureable, leading to a conclusion • Reflects the competencies expected • EPA’s together constitute the core of the profession ten Cate et al AcadMed 2007

48. Core Entrustable Professional Activities for Entering Residency (CEPAER) AAMC 2013

49. Relationship: EPAs and General Competencies Competency-Based Postgraduate Training: Can We Bridge the Gap between Theory and Clinical Practice? ten Cate Acad Med 2007

50. Internal Medicine:End-of-Training EPAs Manage care of patients with acute common diseases across multiple care settings Manage care of patients with acute complex diseases across multiple care settings Manage care of patients with chronic diseases across multiple care settings Provide age-appropriate screening and preventive care Resuscitate, stabilize, and care for unstable or critically ill patients Provide perioperative assessment and care Provide general internal medicine consultation to nonmedical specialties Manage transitions of care Facilitate family meetings Lead and work within inter-professional health care teams Facilitate the learning of patients, families, and members of the interdisciplinary team Enhance patient safety Improve the quality of health care at both the individual and systems level Advocate for individual patients Demonstrate personal habits of lifelong learning Demonstrate professional behavior