DEPARTMENT OF PSYCHIATRY

1. The Milestone Project: An Update DEPARTMENT OF PSYCHIATRY Carol A. Bernstein, MD Associate Professor of Psychiatry Vice Chair for Education and Director of Residency Training Department of Psychiatry NYU School of Medicine Member, Board of Directors, ACGME Beth Israel Medical Center January 14, 2013

2. With Much Appreciation To:Thomas Nasca, MD, MACPTimothy Brigham, PhDACGME Staff

3. ACGME Mission Statement “We improve health care by assessing and advancing the quality of resident physicians' education through exemplary accreditation”

4. The Social Contract Compels Educators to assure that graduates: • Maintain the values and virtues of professionalism, including excellence in clinical practice, and meet society’s needs • Deliver safe, affordable, quality care in a fashion that models these values and virtues, and that meet society’s evolving needs Thomas J. Nasca, MD, MACP

5. The Actions of the ACGME must fulfill the Social Contract And ensure Sponsors that maintain: • the safety and quality of care of the patients under the care of residents today • the safety and quality of care of the patients under the care of our graduates in their future practice • the provision of a humanistic educational environment

6. Educating Physicians who commit to: • Being scientifically and clinically competent • Minimizing self interest in favor of the protection and promotion of the patient’s health-related interests (Altruism) • Maintaining and passing on medicine to future patients, physicians, and society as a public trust, not a merchant guild McCullough, L. The Ethical Concept of Medicine as a Profession. Advances in Bioethics, Volume 10, 17-27. 2006. Thomas J. Nasca, MD, MACP, modified by Timothy P. Brigham

7. Why change now?

8. Brief History • Late 1990’s: ACGME/ABMS Competencies developed • ACGME requires: • Competency framework for curriculum • Phased incorporation of competency evaluation • Culminates in “competency based accreditation” • The “field” left to develop “tools” for the “toolbox” • ABMS organizes individual certification in Competency framework • Initial Certification • Maintenance of Certification • Tools developed that may be relevant in GME phase T. Nasca, MD, MACP, 2011

9. The 2005 ACGME Strategic Plan 1: At its November 2005 retreat, the ACGME Executive Committee endorsed four strategic priorities designed to enable emergence of the new accreditation model: • Foster innovation and improvement in the learning environment • Increase the accreditation emphasis on educational outcomes • Increase efficiency and reduce burden in accreditation • Improve communication and collaboration with key internal and external stakeholders 1 ACGME 2005 Strategic Plan

10. External Pressures

11. Never be afraid to try something new. Remember that amateurs built the ark, and professionals built the Titanic. Anonymous

12. What Currently Drives the Structure and Content of our Residency Programs?

13. What Will Drive the Structure and Content of our Residency Programs in the Near Future?

14. “One definition of insanity is doing the same thing over and over again, but expecting different results.” Rita Mae Brown Sudden Death, 1983. p. 68

15. The Conceptual Change From… The Current Accreditation System Rules Corresponding Questions “Correct or Incorrect Answer” Citations and Accreditation Decision

16. The Conceptual Change To… The “Next Accreditation System” “Continuous Observations” Assure that the Program Fixes the Problem Number of Potential Problems Promote Innovation Diagnose the Problem (if there is one)

17. Goals of the “Next Accreditation System”(NAS)

18. Goals of NAS: • To foster the development of realistic outcomes • To free good programs to innovate • To help weak programs to improve • To reduce the burden of accreditation • To provide accountability for outcomes to the public

19. NAS in a Nutshell • Continuous Accreditation Model – annually updated • Based on annual data submitted, other data requested, and program trends • Scheduled Site Visits replaced by 10 year Self Study Visit • Standards revised every 10 years Organized by: • Structure • Resources • Core Processes • Detailed Processes • Outcomes

21. Phased Implementation • Phase 1 Specialties (and Subspecialties) • Internal Medicine • Pediatrics • Emergency Medicine • Diagnostic Radiology • Urology • Orthopedic Surgery • Neurological Surgery • Phase 2 Specialties (and Subspecialties) • All Other Specialties • Institutional Review • Transitional Year

22. Implementation Timeline PHASE 1 Ceased routine Site Visits PHASE 2 Will cease routine Site Visits First round of CLER visits (underway) PHASE 1 Will begin Continuous Accreditation PHASE 2 Will enter “preparation year” PHASE 1 Entered “preparation year” PHASE 2 Will begin Continuous Accreditation New Policies and Procedures take effect (7/1/2013) CLER = Clinical Learning Environment Review

23. Essential Elements • Continuous (as opposed to Periodic) Accreditation • Increased Focus on Outcomes • Demonstration of Desired Outcomes (Milestones) • Enhance flexibility of programs to innovate • Standardization Balanced with Specialty Specificity • Increased Emphasis on Patient Safety and Quality Improvement • Enhanced Continuity Across the Spectrum of Training • Transparency • Enhanced Institutional Accountability Thomas J. Nasca, MD, MACP

24. “Every system is perfectly designed to yield the result it produces.”

25. The Building Blocks ofThe Next Accreditation System

26. Trended Performance Indicators • Annual ADS Update • Program Attrition – Changes in PD/Core Faculty/Residents • Program Characteristics – Structure and Resources • Scholarly Activity • Board Pass Rate – Rolling Rates • Resident Survey – Common and Specialty Elements • Clinical Experience – Case Logs or other • Faculty Survey – Core Faculty • Semi-Annual Resident Evaluation and Feedback • Milestones • Annual Sponsor Site Visit (CLER) – Every 18 Months

27. ACGME Goals for Milestones • Accountability for effectiveness of educational programs in producing outcomes • Collaboration with AAMC, LCME to improve graduation level preparation • Collaboration with ABMS, AHA, ACCME, others to identify areas for milestone improvement at the time of graduation from residency Milestones Continuing Education/ (MOL – MOC) Medical Education MD Subspecialty Education (Fellowship) Specialty Education (Residency) Premedical Education BA/BS

28. ACGME Goals for Milestones • Outcome focused • Use of existing tools and observations of the faculty • Clinical Competency Committee triangulates the progress of each resident • ABMS Board, Academy, Program Directors and RRC jointly define the expectations (Milestones) • ABMS Board tracks the identified individual • ACGME Review Committee tracks unidentified individual trajectories • ACGME and ABMS are able to provide accountability for effectiveness of the educational program in producing outcomes for individual trainees T. Nasca, MD, MACP, 2012

29. ACGME Goals for Milestones • Specialty specific normative data and common expectations for progress of individual residents • Less prescriptive ACGME program requirements, lengthened program site visit cycles, less frequent standards revision • Promotion of curricular innovation • Enhancement of curricular and rotation design • Opportunity for communication and improvement across the continuum of medical education • Development of specialty specific evaluation tools and techniques T. Nasca, MD, MACP, 2012

30. ACGME Goals for Milestones • Able to provide accountability for effectiveness of educational program in producing outcomes • ACGME can work with: • AAMC, LCME to focus graduation level preparation • ABMS, AHA, ACCME, others to identify areas for milestone improvement at graduation from residency/ fellowship

31. ACGME Specialty Specific Competency Evaluation Program Requirements for reporting (Outcomes) Milestones & Core Evaluation Requirements • Specialty Specific • Teams • Board • PD Association • College • RRC • Residents Increase the Accreditation Emphasis on Educational Outcomes Thomas J. Nasca, MD, MACP Modified by T. Brigham

32. Expectations for Milestones • When and where possible, identification of appropriate assessment tools/methods/systems • Creation of a summary reporting document to be used by a Clinical Competence Committee in each program to report to the ACGME and Specialty Board

33. Not to you Not for you With you!

34. Clinical Competency

35. Clinical Competency Milestones: Observable developmental steps moving from Novice to Expert/Master • Organized under the rubric of the 6 domains of clinical competency • Describe a trajectory of progress from neophyte towards independent practice • Articulate a shared understanding of expectations • Set aspirational goals of excellence • Provide a framework and language for discussions across the continuum

36. General Competences • Patient Care and Technical Skill • Compassionate, appropriate, effective • Medical Knowledge • Know and can apply • Can do and apply • Practice-Based Learning and Improvement • Assessment of own patient care, evidence-based approaches, improvement • Interpersonal and Communication Skills • Effective exchange of information and collaboration with patients, their families, and health professionals • Professionalism • Committed to professional responsibilities, ethical principles and sensitivity to diversity • Systems-Based Practice • Awareness and utilization of the larger context and system of healthcare in providing optimal patient care

37. The Six Competencies, and the Continuum of ClinicalMedical Education – Dreyfus (modified) Model1 • Medical Knowledge • Patient Care and Procedural Skills2 • Interpersonal and Communication Skills • Professionalism • Practice Based Learning and Improvement • Systems Based Practice • Novice • Advanced Beginner • Competent • Proficient • Expert • Master • Undergraduate • Graduate • Continuing 1 as presented by Leach, D., modified by Nasca, T.J. American Board of Internal Medicine Summer Retreat, August, 1999. 2 Patient Care Competency modified 9/2010 by ACGME and ABMS

38. Miller’s Pyramid of Clinical Competence11Miller, GE. Assessment of Clinical Skills/Competence/Performance.Academic Medicine (Supplement) 1990. 65. (S63-S67 )

39. Key Elements of Miller’s Pyramid • Trained Observers • Common understanding of the expectations • Sensitive “eye” to key elements • Consistent evaluation of a given level of performance • Minimum Numbers of Quality Observations • Interpreter/Synthesizer Experts • Clinical Competency Committee (Resident Evaluation Committee)

40. Trajectory of Milestones

41. Clinical Professional Development

42. The Goal of the Continuum of Clinical Professional Development

43. Reporting Template

44. Milestones: Guiding Principles

45. Next Steps in the Outcomes Project • Milestone definitions: Descriptions (in specific behavioral terms) of the performance level expected of a resident by a particular time during their residency • Aggregation of resident performance on milestones as an indicator of overall programmatic educational effectiveness • Board use as part of eligibility for certification

46. “Somebody has to do something, and it’s just incredibly pathetic fantastic that it hasgets to be us.” Jerry Garcia The Grateful Dead edits, TJ Nasca Gratefully Not Dead

47. ACGME Milestone Project Team

48. Clinical Competence Committee

49. Six Month Evaluations • Determined by the CC Committee • Grouped by Core Competency • Established by Professional Consensus • Part of Programmatic Self-Evaluation and Accreditation • Measureable and Meaningful

50. Expert Panels Assembled by the ACGME for: • Systems-based Practice • Practice-based Learning and Improvement • Communication Skills • Professionalism • Assessment