Milestones, EPAs, NAS…and Other ACGME Jargon

1. Milestones, EPAs, NAS…and Other ACGME Jargon Committee on Graduate Medical Education September 24, 2012 Sara LP Ross, MD

2. Objectives • To discuss the Next Accreditation System and what is known about how that will look in 2019 • To discuss the system of trainee evaluation – Milestones and EPAs

3. NAS: Next Accreditation System • Goals • Complete realization of the Outcomes promise • Free up “good programs” to innovate • Assist poor programs in improving • Reduce burden of accreditation • Establish and implement milestones to better track program and institutional performance • Provide accountability to the public • July 2012: Seven initial core specialties/RRCs begin NAS training • July 2013: NAS officially begins; seven specialties “go live”; remaining specialties begin training • July 2014: All specialties/RRCs using NAS

4. NAS: Next Accreditation System • 10-year self-study visit model: next visit 2019 for all Peds department programs • Structure, Resources, Core processes, Detailed processes, Outcomes • “What is your plan for the next 10 years to improve” • Residents will submit a confidential consensus list of five strengths and “opportunities for improvement” (OFIs) the residents wish to discuss • Site visitor will share strengths, but will only share OFIs if residents give permission (makes residents feel more connected to the site visit) • Annual program surveillance • Performance indicators for each specialty developed by a “community of educators” within the specialty • Annual Resident Survey • Core elements of the competencies • Levels of performance • Core methods of assessment • Annual Faculty Survey • Case Log Data

5. NAS: Next Accreditation System • Institutional 18 month review • Ongoing creation of Milestones • Programs will get a letter annually stating that they meet all “performance indicator thresholds” • May be placed on “accreditation with warning” at any point during the 10-year cycle • May warrant an “immediate visit” • If problems not fixed during a given window of time, program may be placed on probation

6. Site Visits • Structure of the visit: 2 site visitors • Brief meeting with PD • Resident and faculty interviews • Meeting with DIO • Meeting with PD • PIF Elimination – YES (most likely) • More focus on strengths of programs

7. Competencies • Competence: the ability to do something successfully (Oxford Dictionary of English) • Competencies: broad, general attributes of a good doctor • With attempt at evaluation they get widdled down to detailed skills/activities • In the end don’t really reflect the original meaning of the general competency • Competence = Attribute • Activity= Element of professional work

8. Entrustable Professional Activities(EPAs) • Units of work that may be awarded a more or less formal qualification at the moment when supervisors confirm the trainee is ready to assume responsibility for such activities

9. Entrustable Professional Activities (EPAs) • Which critical professional activities cover the relevant competencies of the profession? • How can supervisors learn when to entrust such activities to the trainee? • “Trust reflects a dimension of competence that reaches further than observed ability. It includes the real outcome of training – the quality of care” • Part of essential professional work • Require specific knowledge, skill and attitude • Lead to recognized output of professional labor • Confined to qualified personnel • Be independently executable within a timeframe • Be observable and measurable in its process and outcome (well done or not well done) • Reflect one or more competencies to be acquired

10. Domains of Competency Competencies Milestones EPAs

11. EPA/Competency Matrix Viewpoint: Competency-Based Postgraduate Training: Can We Bridge the Gap between Theory and Clinical Practice? ten Cate, Olle; Scheele, Fedde Academic Medicine. 82(6):542-547, June 2007. DOI: 10.1097/ACM.0b013e31805559c7

12. EPAs • May be acknowledged formally as a “statement of awarded responsibility” (STAR) • Five levels of proficiency • Has knowledge • May act under full supervision • May act under moderate supervision • May act independently • May act as a supervisor and instructor

13. Expected Levels of Confidence Time to achieve STAR in a specific EPA dependent on: The EPA The working environment The trainee The clinical teacher

14. EPA Mapped to Competencies/ Subcompetencies

16. Milestones • Developmental roadmap for the competencies and subcompetencies • Observable developmental steps moving residents from novices to experts/masters • Means of restructuring competencies into a measurable rubric of six domains of clinical competency

17. Domains of Competency Competencies Milestones EPAs

18. What milestone levels equate to different levels of proficiency? Who determines you can practice the EPA independently going forward? Minimum standards for advancement/graduation?

19. Where Are We in Pediatrics? • January 2012 The Pediatrics Milestone Project (51) • September 2012 Selection of 21 Pediatric Milestones to be reported on semi-annually - Requests for additional sites to study/ develop each of these milestones • Fall/Winter 2012 Program development of evaluation tool • July 2013 Implementation of Milestone reporting

21. Where Are We in Pediatrics? • January 2012 The Pediatrics Milestone Project (51) • September 2012 Selection of 21 Pediatric Milestones to be reported on semi-annually - Requests for additional sites to study/ develop each of these milestones • Fall/Winter 2012 Program development of evaluation tool • July 2013 Implementation of Milestone reporting

22. References • Carraccio C, Burke A. Beyond competencies and milestones: Adding meaning through context. J of Grad Med Ed. 2010;2(3):419-422. • ten Cate O. Trust, competence, and the supervisor’s role in postgraduate training. British Medical Journal. 2006;333:748-751. • ten Cate O, Scheele F. Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Acad Med. 2001;82:542-547.