Internal Reviews

1. NATIONAL CAPITAL CONSORTIUM Uniformed Services University of the Health Sciences Program Coordinators Workshop – 27 January 2010 Internal Reviews What? When? How? DiannRichko Internal Reviews Manager National Capital Consortium

2. IR Goals and Objectives To evaluate: • Compliance with ACGME and RRC requirements • Curriculum & resources • Implementation of corrective plans for prior citations and findings To identify: • Strengths and weaknesses of the program • Best practices • Potential issues, if any, that need to be rectified before your site visit. This is a GOOD thing!

3. Step 1: IR TIMING

4. Timing…it starts upon accreditation • Identify the internal review date (midpoint) as stated on your most recent accreditation letter • Determine the time frame for your program’s internal review

5. Midpoint • Defined as the date that the internal review Executive Summary report should be presented to the GMEC • The midpoint must be met in order to comply with ACGME Institutional and Program requirements • If not met, citation for program and NCC

6. Determining your Internal Review date • Identify the midpoint • Work backwards from the midpoint and identify the GMEC meeting for the month of your midpoint date • The GMEC meets the 1st Wednesday of each month (confirm with NCC GME Office) • The Subcommittee on Internal Reviews receives the Executive Summary and makes recommendations. The SIR meets the 3rd week of each month • Allot time for the IR to take place and for the IR Committee to complete the Executive Summary Report

7. November Midpoint OctoberNovember 21 October = SIR Mtg. 3 November = GMEC Mtg. 16November = Midpoint

8. Timing: Special Considerations • Adjustments to the dates of either the GMEC or SIR meetings • Holidays • Major academy meetings for your program specialty • BRAC and program relocation

9. Step 2: NCC NOTIFICATION

10. The IR Manager: • Sends a notification email to the Program approximately 2-3 months in advance • Identifies the midpoint and latest timeframe for the IR • Identifies the Chair of the IR Committee • Provides documents used during the review process • Distributes Program Director Worksheet • Identifies the type of IR to be performed

11. Program Director Worksheet • Due to the Chair of the IR Committee at least 2 weeks prior to the IR date • Providing the information streamlines the IR process; may eliminate questions or identify new ones

12. IR Manager Provides • ACGME institutional & common program requirements checklist (standard for all NCC programs) • Program (specialty) and/or subspecialty requirements checklist(s) • Last IR Executive Summary and any follow-up • Last accreditation letter and any progress report(s) • Most recent ACGME resident survey report • ACGME and NCC guidelines on IRs

13. Step 3: PREPARATIONS

14. How to Prepare • Organize your documents • Review your ACGME requirements • The IR Committee Chair and the PD work on a date and time for the review • Reserve three separate rooms for the interview sessions for at least 2 – 2.5 hrs. • Rooms can include office space, pt. exam rooms, conference rooms, etc. • No refreshments are necessary

15. How to Document • No right or wrong way--it just must be documented! Spreadsheets, tables, charts, e-value, binders, etc. • Outcome measures (ex: show that negative evals on a rotation site resulted in a change or improvement) • Share ideas and network with other coordinators in your specialty! • Ask questions or ask for assistance; the NCC GME office is here to help!

16. Identify Program Participants Italics indicates “ACGME/NCC required participation” • Program Leadership: Program Director, Associate Program Director, Coordinator, Service Chief • Faculty: No set number, but enough present to give a good overview of the program; include local rotations • Trainees: • If 9 or less in your program – ALL must be present (unless at outside rotation) • If 10 or more: A minimum of one from each level year of training MUST be present. These reps must be peer-selected by the entire group of residents.

17. Step 4: The “Big Day”

18. Interview Sessions • Program Leadership meets with the IR Committee Chair. Documents are reviewed during this session • Faculty meet with IR Committee Faculty member • Trainees meet with the IR Committee Resident member • All occur simultaneously in the three locations

19. What is reviewed? • Discussion of strengths and weaknesses of program • Findings from: • Most recent ACGME accreditation letter • Prior internal review findings • Most recent ACGME resident survey findings • Corrective actions taken for all of the above

20. What is reviewed? (copies) • Fully executed (signed) MOUs and PLAs • Portfolios or training records (3 - 5) *** • Residency Program Handbook • Program Policies • Case or procedure logs • Duty Hour reports • Evaluations • Trainees; faculty; program; rotations • Summative & formative • Board passage rates

21. What is reviewed? (copies) • Annual program review meeting minutes • Conference/didactic schedules & attendance • Scholarly activity • Goals & objectives (ACGME Core Competency format) • Progressive levels of responsibility • Required training i.e. fatigue and sleep impairment; substance abuse etc. • Resources – space, personnel, library access, on-call rooms, computers, equipment • Other documentation as determined by your ACGME reqs

22. Your Role: • Understand the role your PD expects you to play • The IR Committee encourages your participation • Preparations for an IR are similar to a site visit – great learning experience • Have documentation organized for inspection • Assure the most participation by your program • Be prepared to shine!

23. After the interviews • IR Committee meets to compare findings • Sometimes post-discussions are held with the PD • The committee may seek clarification from PD post-IR • The Executive Summary report is drafted

24. Step 5: The Findings

25. Executive Summary Report • The IR Committee submits report to the SIR • PD is sent a copy; notified of GMEC meeting the report will be presented • The SIR reviews; makes recommendations on findings • GMEC approves/disapproves corrective actions requested by the SIR • Letter of Concern sent to the PD • PD submits a progress report to the SIR

26. IR Process of the IR Findings • EXEC SUMMARY • SIR • PD PROGRESS REPORT • APPROVED; • NO F/U REQUIRED • APPROVED; • F/U REQUIRED • GMEC

27. IR Outcome for Coordinator • May need to implement new documents, new tracking/trending methods • Keep this paper trail for future ACGME site visit • Retain copy of GMEC minutes in which your IR was presented

28. Special Internal Reviews Modified IR: • Program with no trainees enrolled • Occurs at midpoint • Only program leadership is interviewed • Program should demonstrate understanding and preparations to implement ACGME requirements

29. Special Internal Reviews The following IRs can occur at any time and are determined by the SIR, GMEC or DIO. These may or may not involve all interviews. • Targeted: performed after a standard IR where significant shortcomings are identified. Only key areas are targeted for re-review • Resignation of PD prior to site visit: to ensure smooth transition and documentation in place • Discretional IR: Can be initiated for any SIR concern

30. Educate Yourself • Be a valuable resource to your program • Explore the ACGME website; read RRC newsletters • Understand all program requirements • Update PD on changes; plan to implement new requirements • Review your program’s residency handbook • Attend GMEC, AHME and ACGME meetings • Familiarize yourself with the NCC Administrative Handbook

31. Diann Richko Internal Reviews Manager NCC GME Office Blg E, 2nd Floor, USUHS Ph: 301-319-0723 Fax: 301-319-0308 Email: drichko@usuhs.mil