Managing the ACGME Survey OPDO Meeting November 11, 2011

1. Managing the ACGME SurveyOPDO MeetingNovember 11, 2011 Marita S. Teng, MD Program Director Department of Otolaryngology Mount Sinai School of Medicine New York, NY

2. ACGME Survey – What is it? • Online survey, 34 questions • Administered annually for residents and fellows • Timing: mid-January thru early June • Requires 70% completion rate • Aggregate program-level data provided to PD and DIO if 70% response rate reached • Site visitors and RRC have data access regardless of response rate

3. ACGME Survey – What is it? • Topics covered • Duty Hours • Faculty • Evaluation Systems • Educational Content • Resources • Overall Experience

4. Data from Survey • From the ACGME website • “Data from the survey are intended as a diagnostic tool to focus the field staff’s questions during the on-site visit… [and] to verify and clarify the information from them. Field staff will probe and clarify any grey shaded areas, and pay particular attention to the duty hour items and to items with a substantial number of non-compliant responses.” • If residents indicate problem areas corrected, site visitor will document. • If questions/phrases misunderstood by residents, site visitor will document.

5. When to Worry? • “Substantial number of noncompliant responses” • 20% is a common institutional standard • Internal systematic review and action plan required • RRC may send warning letters to programs for poor results • Accreditation cycle may be shortened, effectively triggering an immediate site visit

6. Problems specific to us • Small programs • One or two disgruntled residents can significantly alter the data • Confidentiality is affected • Interns • Data often reflect their experiences on services other than Otolaryngology, and are hard to pinpoint because of varying rotation schedules and anonymity of survey But no “excuses” are accepted!

7. “Safe Surveying” • Survey instructions sent out only after a intro/Q&A session • Explain purpose of survey, clarify questions • Survey form available on ACGME website – go item by item • This is not the forum for complaints • Consider NOT having the PD conduct the session • Associate PD? • Chief resident(s)? • Program coordinator?

8. “Safe Surveying” • Remind the residents about certain items: • Duty hours • Confidentiality of your evaluation system • Availability/accessibility of Goals & Objectives • Resources & systems for communication/support • education committee meetings • faculty mentors • chief or senior residents • institutional ombudsman • Sleep deprivation education • SAFER curriculum (ppt)

9. Otolaryngology Problem Areas • We are better than most! • Focus on areas of around 5% or greater national rates of non-compliance

10. Duty HoursSeven Questions • 80-hour week (over 4 wks) • 1 day off in 7 (over 4 wks) • No more than Q3 call (over 4 wks) • 10 hours off between duty periods • 24 + 6 continuous hours in-house • Home call does not preclude “rest and reasonable personal time” • Called into hospital from home  counts towards 80hrs

11. Compiled Otolaryngology Data:Duty Hours Duty hour question guide (PDF) for residents on ACGME website

12. Compiled Otolaryngology Data:Faculty

13. Faculty Issues • Consider faculty workshops • Mentorship • Giving feedback • Teaching techniques • Resources • GME office • Dean’s office • Faculty Development Programs

14. Compiled Otolaryngology Data:Evaluation

15. Evaluation issues • Confidentiality is compromised by nature of being a small program • Consider pooling attending or rotation evaluations, then distribute at the end of each year • Evaluations need to be meaningful • Shortening the form • Requiring written comments • Issues identified by residents should be followed up; improvements should be documented

16. Compiled Otolaryngology Data:Educational Content

17. Educational Content Issues • Time for research • How protected is the time? • Balance between education and service • Physician extenders – “non-educational” work • Continually re-evaluate quality of didactics • Faculty mentorship & teaching

18. Compiled Otolaryngology Data:Resources

19. Resource Issues • Interdisciplinary teams • Tumor Board? • Case conferences? • Combined Grand Rounds? • QM or QI teams? • Dealing with problems confidentially • Institutional resources in the GME office • Dealing with problems without fear of intimidation or retaliation • Resident-only meetings

20. Overall Experience • Which of the following best summarizes your opinion of your residency program?

21. Take-Home Points • Explain and prepare residents • Purpose of the survey • Specific questions • Can re-administer survey internally • Increase communication • Regular meetings with residents • Regular meetings with faculty • Lots of channels • Increase overall morale • We are in good shape! • Take as opportunity to improve program

22. Thank You • Mount Sinai School of Medicine GME office • Scott Barnett, MD – Associate Dean for GME • Paul Johnson • Ben Malkin, MD • Associate Program Director, ENT, MSSM