Conflict of Interest

1. Conflict of Interest • Kim Walker – No conflicts of interest to disclose • Ann Dohn – No conflicts of interest to disclose • Nancy Piro – No conflicts of interest to disclose

2. PC002c Coordinators and Clinical Competency Committees: How to Streamline and Support the Work of your Program’s CCC Kim Walker, PhD Program Manager/Education Specialist Ann Dohn, MA DIO & GME Director Nancy Piro, PhD Program Manager/Education Specialist

3. Session Outcomes Participants will be able to: • Identify new aspects of the coordinators’ evolving role in program administration. • Understand and use program requirements as a guide for planning, organizing and implementing educational and assessment tools. • Develop and utilize a newly developed comprehensive resident performance profile tool to streamline the work of the CCCs.

4. New Role

5. Evolving Role for Coordinators in Evaluations Education & Evaluation Coordinator/Manager Extraordinaire Administrator Scheduler Supreme • Constructing new milestone evaluations to pilot/deliver • Reviewing evaluation completion data for accuracy • Aggregating data for the CCC from multiple sources and forms • Milestone data to ACGME • Deliver evaluations • Develop evaluation forms for PDs to approve • Schedule semi annual evaluations • Ensure summative evaluations completed and filed

6. Now I’m really confused! Outcomes Evaluations CCCs EPAs Milestones Goals and Objectives

7. Where do I begin?

8. Know the NAS Building Blocks: Concepts defined • NAS • Core Competencies • Milestones • EPAs • Curriculum and Evaluations • Clinical Competency Committee (CCC)

9. NAS – Next Accreditation System What is NAS – in a nutshell: “an outcomes-based accreditation process through which the doctors of tomorrow will be measured for their competency in performing the essential tasks necessary for clinical practice in the 21st century.” http://www.acgme.org/acgmeweb/tabid/435/ProgramandInstitutionalAccreditation/NextAccreditationSystem.aspx

10. Major Changes:Accreditation based on… • Pre-NAS • Competencies • Site Visits – Up to 5+ year cycles • Internal Reviews • ADS Updates • PIFs • Resident Surveys • Current (New) NAS • Competencies with Milestones • Self-Studies at ~ 8-10 year intervals • Detailed ADS Updates • CLER Visits ~ 18 – 24 months (Institution) • Resident &Faculty Surveys

11. The New Accreditation System (NAS)… Outcomes • Increased Annual reporting by Programs (online) • Reduced volume of accreditation demands … but increased attention to accuracy and completeness of information submitted online • PIF-less Surveyor visits (unless new application) • Two Field Surveyors per visit • No Faculty CVs (only PD)….but Faculty & Resident Scholarly Activity required.

12. The Six AGME Core Competencies Patient Care Interpersonal & Communication Skills Medical Knowledge Six Core Competencies For Quality Patient Care Practice-based Learning & Improvement Professionalism Systems-based Practice

13. What Are Milestones? • High Level - Milestones are simply defined as areas of competency/expectations for our trainees • Linked to six core competencies • Defined as a continuum of progressive growth/learning

14. Dreyfus Model (1980):Stages of developing expertise Source: Eraut, M. Developing Professional Knowledge and Competencies. (1994)

15. Milestone Level Definitions • Level 1:The resident is a graduating medical student/experiencing first day of residency. • Level 2: The resident is advancing and demonstrating additional milestones. • Level 3: The resident continues to advance and demonstrate additional milestones; the resident consistently demonstrates the majority of milestones targeted for residency.

16. Milestone Level Definitions (continued) • Level 4: The resident has advanced so that he or she now substantially demonstrates the milestones targeted for residency. This level is designed as the graduation target – not requirement. • Level 5: The resident has advanced beyond performance targets set for residency and is demonstrating “aspirational” goals which might describe the performance of someone who has been in practice for several years. It is expected that only a few exceptional residents will reach this level.

17. Reporting the Milestones V.A.1.b).(1).(b) prepare and assure the reporting of Milestones evaluations of each resident semi-annually to ACGME (Core) • Milestones are reported directly through ADS • Reporting windows are: • November 1- December 31 • May 1- June 15

18. Entrustable Professional Activities (EPA) • Professional life activities that define a medical specialty: • Ground the competencies in a physician’s everyday work • Activities lead to some outcome that can be observed • Complexity of the activities requires an integration of knowledge, skills and attitudes across competency domains

19. Examples of EPAs • Facilitate handovers to another healthcare provider either within or across settings • Contribute to the scholarly work of the subspecialty • Co-manage patients with generalists and other subspecialists Source: https://www.abp.org/abpwebsite/taskforce/reslib/24.ppt

20. Curriculum: Rotation-specific goals and objectives & links to milestones

21. Milestones Impact on Evaluations:Linking questions to milestones • Step Two: Ensure specific evaluation questions are linked to milestones Advises the referring health care provider(s) about the appropriateness of a procedure in routine clinical situations

22. Milestones Impact on Evaluation System • Allows for more objective methods of assessment and provide better feedback • Provides a process for early identification of residents that are having difficulties • All old and new evaluations and questions should be aligned with and tracked to milestones

23. Clinical Competency Committee (CCC) V.A.1. The program director must appoint the Clinical Competency Committee.(Core) V.A.1.a) At a minimum the Clinical Competency Committee must be composed of three members of the program faculty.(Core) V.A.1.a).(1) Others eligible for appointment to the committee include faculty from other programs and non- physician members of the health care team.(Detail) ACGME Common Program Requirements Approved: February 7, 2012; Effective: July 1, 2013 Approved focused revision: June 9, 2013; Effective: July 1, 2013

24. Clinical Competency Committee (CCC) V.A.1.b).(1) The Clinical Competency Committee should: V.A.1.b).(1).(a) review all resident evaluations semi- annually; (Core) V.A.1.b).(1).(b) prepare and assure the reporting of Milestones evaluations of each resident semi-annually to ACGME; and, (Core) V.A.1.b).(1).(c) advise the program director regarding resident progress, including promotion, remediation, and dismissal.(Detail)

25. Clinical Competency Committee (CCC) V.A.1.b) There must be a written description of the responsibilities of the Clinical Competency Committee.(Core) ACGME Common Program Requirements Approved: February 7, 2012; Effective: July 1, 2013 Approved focused revision: June 9, 2013; Effective: July 1, 2013

26. Clinical Competency Committee (CCC):Written description

27. Clinical Competency Committee (CCC) • How the CCC does its work can be decided by the Program Director • Subcommittees • Assigning residents to faculty members for pre-review • Pre-review work will vary • Scheduling and frequency of meetings

28. What Should a CCC Do First? • Understand their specialty Milestones (Posted on acgme.org) • Decide how to assess the Milestones – Program Evaluation Strategy • If necessary, identify new evaluation tools from program director associations, societies, colleges

29. NAS and Milestones and YOU • The program coordinator will play a crucial role in developing, implementing, collecting data on and reporting of milestone evaluation tools.

30. Managing it all…

31. U - R - IT! • Understanding RRC program requirements • Requirements applied to evaluation methods/process • Implementing new evaluation system • Tracking completion and accuracy (outliers) for data aggregation

32. 1. Understand Your Program’s New Requirements Core Outcomes Details

33. 1. Understand Your Program’s New Requirements • Each standard/requirement is categorized: • Outcome- All programs must adhere • Core- All programs must adhere • Detail– Considered mandatory for new programs and those that fail to meet core requirements. Allows high-performing programs to innovate. Source: Implementing The Next Accreditation System ACGME Webinar John R. Potts, III, M.D.: 4 November 2013

34. 1. Understand Your Program’s New Requirements Example Program Requirement: VI.B. Transitions of Care (Core) (Core) (Outcome)

35. 1. Understand Your Program’s New Requirements Example Program Requirement: VI.B. Transitions of Care When core and outcome not in compliance, then: Details

36. U - R - IT! • Understanding RRC program requirements • Requirements applied to evaluation methods/process • Linking milestones/EPAs and objectives to evaluation questions • Utilizing milestone scales • Implementing new evaluation system • Tracking completion and accuracy (outliers) for data aggregation

37. 2. Requirements Specific to CCC Review of Trainee • Aggregating/compiling multiple evaluations of individual trainees (V.A.1. Formative Evaluation) • Tracking trainee participation in conferences, journal clubs, didactics (IV.A.3. Didactic Sessions) • Monitoring duty hour compliance (VI.G. Duty Hours) • Reviewing involvement in quality improvement and patient safety activities (IV.A.5.c. PBLI) • Reviewing scholarly work (IV.B. Scholarly Work) • Monitoring and reporting procedure logs (IV.A.5.a)

38. U - R - IT! • Understanding RRC program requirements • Requirements applied to evaluation methods/process • Linking milestones/EPAs and objectives to evaluation questions • Utilizing milestone scales • Implementing new evaluation system • Tracking completion and accuracy (outliers) for data aggregation

39. 3. Implementing • Evaluation systems • Milestone-based/EPAs • Rotation-specific • Patient handovers • Define evaluator groups (faculty, staff, patients) • Set up and timing of delivery systems

40. 3. Implementing • Documentation and reporting systems for: • Conference attendance • Scholarly work (Learning Portfolios) • Quality Improvement and Patient Safety (Learning Portfolios / Safety reporting systems) • Duty Hours (recording, monitoring, reporting) • Case Logging (if applicable)

41. U - R - IT! • Understanding RRC program requirements • Requirements applied to evaluation methods/process • Linking milestones/EPAs and objectives to evaluation questions • Utilizing milestone scales • Implementing new evaluation system • Tracking completion and accuracy (outliers) for data aggregation

42. 4. Tracking and Reporting… • Start with the end in mind: • CCC biannual reporting windows to ACGME • November 1- December 31 / May 1 - June 15 • Back track and set calendar events for: • Periodic monitoring of evaluation completion • Running aggregate reports and reviewing milestone evaluation data • Reviewing case logs, learning portfolios, duty hours

43. Pulling the Data Together In-service training exams Quality Improvement Activities End-of-Rotation Evaluations Sim Lab Safety Incident Reports Clinical Skills Assessment Clinical Competency Committee Case Logs Nursing and Staff / Techs Evaluations Patient/ Family Evaluations Progress on Milestones

44. Managing it all:How will I pull this off?

45. The Toolbox

46. Creating a Resident Performance Profile Goals to support your CCC Resident performance data that is: • Comprehensive • Consolidated / Aggregated • Easy for CCC to identify strengths, areas for improvement, opportunities for advancement

47. Creating a Resident Performance Profile:Compiling and centralizing data

48. Creating a Resident Performance Profile:Apply visual formatting for trends

49. Creating a Resident Performance Profile Step 1 – Defining what to track

50. Creating a Resident Performance Profile Step 1 – Defining what to track