Robert P. Sticca, MD, FACS Jay M. MacGregor, MD Randolph E. Szlabick, MD, FACS

1. Is The ACGME Resident/Fellow Survey a Valid Tool to Assess General Surgery Residency Programs Compliance with Work Hours Regulations? Robert P. Sticca, MD, FACS Jay M. MacGregor, MD Randolph E. Szlabick, MD, FACS Department of Surgery University of North Dakota SMHS

2. Background • 2003 – ACGME work hours regulations implemented • 2004 – ACGME Residents/Fellows Survey • 2008 – Special Message from CEO ACGME (Thomas Nasca, MD) • Standard Approach to Programs Across All Specialties with Potential Duty Hours Violations Identified in the Resident Survey

3. ACGME Special Message • The Monitoring Committee has accepted the Resident Survey as a reliable, sensitive and reasonably specific tool • The Monitoring Committee has set reasonable thresholds for categorization of a program as having Potential Duty Hours Violations (PDHV) based on the Resident Survey • The validity of the Resident Survey has been demonstrated though significant correlations with site visit findings to warrant added scrutiny for programs meeting the thresholds of potential noncompliance found in the group targeted for follow-up. • There is a significant correlation between resident reported violations of duty hours with deficiencies in other important areas of the learning environment(correlation 0.51, p<0.0001) Nasca, T. www.ACGME.org

4. ACGME Algorithm for Noncompliant Programs Nasca, T. www.ACGME.org

5. UND Experience • 2005 – RRC site visit – additional categorical resident requested • 5 year approval, no deficiencies • Additional resident denied, insufficient educational justification • 2007 – RRC site visit – additional categorical resident requested • 5 year approval, no deficiencies • Additional resident approved • 2009 – RRC site visit – mandatory due to residents responses on survey • 5 year approval - commendation

6. ACGME Survey - Problems • Answer selections vague • High threshold for compliance • Small program effect • Consequences of survey punitive • Consequences based on survey results and not actual work hours data

7. Current Study • Purpose – obtain general surgery residents views on work hours regulations and ACGME survey • Design – 20 question survey emailed to all Gen. Surg. Program Directors – 5/09 • 9 questions – work hours • 9 questions – ACGME survey • 1 both, 1 demographic, comments • Study period – 5/09 – 10/09 • Multiple reminders, pleas for participation • Internet Survey – surveymonkey.com

8. Questions

9. Results • Answers collected, analyzed - 11/09 • Aggregate • Level of training • Junior – PGY 1, 2 • Senior – PGY > 2 • Statistical analysis – openepi.com • P value < 0.05, statistically significant

10. Results - Responses 965 – 13.1 % of U.S. General Surgery Residents 961 completed entire survey (99.6%) Junior Residents – 441 Senior Residents – 520

11. Results Number Times Survey Taken

12. Results Instruction on Survey

13. Results – Survey Interpretation

14. Results Effect on Program

15. ResultsHonesty/Accuracy

16. Conclusions • Evaluation mechanism in which • 52% do not understand questions • 14% did not answer truthfully • 37% do not feel is accurate • Not best tool to evaluate work hours compliance in residency programs • Consequences significant • Open letter to GME community 10/28/09 • www.acgme.org Nasca, T. www.ACGME.org

17. Thank you !! Program Directors Residents