Measuring the Impact of Resident Work Hours Reform: Recent Findings and Next Steps

1. Measuring the Impact of Resident Work Hours Reform: Recent Findings and Next Steps Patrick S. Romano, MD MPH Professor of Medicine and Pediatrics Division of General Medicine and Center for Healthcare Policy and Research University of California, Davis May 21, 2009

2. Disclosure • Financial support from NHLBI RO1 HL82637 (Kevin Volpp, University of Pennsylvania, PI) • Have you (or your spouse/partner) had a personal financial relationship in the last 12 months with the manufacturer of the products or services that will be discussed in this CME activity? NOT APPLICABLE

3. Acknowledgments Investigative team • Kevin Volpp, MD PhD, University of Pennsylvania • Jeffrey H. Silber, MD PhD, Director, Center for Outcomes Research, Children’s Hospital of Philadelphia • Amy K. Rosen, PhD, Bedford VA Center of Excellence in HSR&D • Paul Rosenbaum, PhD (Statistician), Wharton School • Lisa Bellini, MD, Program Director, University of Pennsylvania Staff • Orit Even-Shoshan, MS, Anne Canamucio, MS, Tiffany Behringer, MS, Yanli Wang, Hong Zhou, Liyi Cen, Mike Halenar Other sources of slides • John Welch, MD (former pediatric resident) • Garth Utter, MD, Department of Surgery

4. Learning Objectives • To Explain Recent and Proposed Policy Changes to Limit Resident Work Hours • To Summarize Recent Evidence Regarding the Impact of ACGME Work Hour Rules Implemented in 2003 • To Discuss Ideas and Methods for Future Research in this Area

5. The Birth of Residency 1889 with the opening of The Johns Hopkins Hospital Osler, Halsted, and Kelly (bedside teaching) Based on German model Room, board, and laundry provided; salary optional until 1965

6. Core Concepts and Practices Graded responsibility, especially for inpatients Variable and lengthy training period Pyramidal system of promotion Restrictive lifestyle (100-120 hrs/week, continuous shifts up to 36 hours)

7. Libby Zion Case 18 year old college student, with known history of depression, taking Nardil® (MAOI), was brought into New York Hospital on October 4, 1984 Presented with fever, agitation and strange jerking motions of body, with occasional disorientation Admitted with diagnosis of “viral syndrome with hysterical symptoms”

8. Libby Zion Case Ordered Demerol® to control her shaking Later in evening Libby became more agitated The intern was contacted at least twice, ordered physical restraints and Haldol® Patient finally fell asleep At 6:30 a.m. her temperature was noted to be 107°F Emergency measures were attempted Patient suffered a cardiac arrest and died

9. Aftermath Sidney Zion’s efforts to change the system 1986 Grand Jury does not indict those involved but strongly criticizes system Bell Commission established to investigate and make recommendations July 1, 1989 Section 405.4 of Title 10 of the New York Codes, Rules and Regulations of the Department of Health go into effect Work hours: Maximum 80 hrs/week averaged over 4 weeks, 24 consecutive hours (12 in ED), at least 1 scheduled 24 hour break per week On-site supervision 24 hrs/day, 7 days/week

10. The 80 hour work week “The specific "80-hour week" was actually determined by a colleague on my porch and was based on the following informal reasoning: (1) there are 168 hours in a week; (2) it is reasonable for residents to work a 10-hour day for 5 days a week; (3) it is humane for people to work every fourth night; (4) subtracting the 50-hour week (10 hours per day x 5 days) from 168 hours leaves 118 hours; (5) divide 118 by 4 (every fourth night) and add to the 50 hours and, eureka, that equals an 80-hour week.” Bell JAMA 2007: 298(24):2865-2866

11. Timeline June 1998 - New York State Department of Health found work hours often exceeded regulatory limits April 2001 - OSHA is petitioned to regulate work hours nationwide November 2001 Representative John Conyers (D-Mich) introduced federal legislation to restrict resident work hours In response ACGME announced its guidelines effective July 1, 2003

12. ACGME Work Hour Restrictions Principles, Supervision, Fatigue, Duty Hours, On-Call activities, and Moonlighting Limited to 80 hours per week One day in seven free of all responsibilities 10 hour off-duty period between work periods In house call no more frequently than every 3rd night In house call must not exceed 24 consecutive hours (with up to 6 extra hours for transition of care)

13. Resident Education and QOL • Meta-analysis of studies that assessed a system change designed to counteract the effects of resident work hours, fatigue, or sleep deprivation; included an outcome directly related to residents; and were conducted in the United States. • 54 articles met inclusion criteria (12 IM, 6 ob/gyn, 7 pediatrics, 25 surgery, 4 other) • Interventions included night and day float teams, extra cross-coverage, and physician extenders. Fletcher, JAMA, 2005

14. Resident Education and QOL • Interventions to reduce resident work hours resulted in mixed effects on both operative experience and on perceived educational quality but generally improved residents’ quality of life (i.e., more sleep, better mood, better family relationships, better satisfaction). • Interpretation of the outcomes of these studies is hampered by suboptimal study design and the use of nonvalidated instruments. The long-term impact of reducing resident work hours on education remains unknown. Fletcher, JAMA, 2005

15. N Engl J Med 2004;351:1838-1848

16. Effect of Reducing Interns’ Work Hours Investigated effect of reducing interns’ work hours on serious medical errors in ICU Randomized, prospective crossover trial Conducted in MICU and CCU, 20 interns on 3 week rotations Q3 day “traditional” call versus 4-day schedule without extended shifts >16 hours (7am-3pm on day 1, 7am-10pm "day call" on day 2, 9pm-1pm “night call" on days 3-4) Compared rates of serious medical errors (by masked direct observation) made by interns on traditional vs. intervention schedule

18. Effect of Reducing Interns’ Work Hours Traditional work week: 74-92 hours (mean 85) Intervention work week: 57-76 hours (mean 65) No change in staffing or other personnel Randomly assigned order, and spread throughout year Interns worked 19.5 hrs/week less (P<0.001), slept 5.8 hrs/week more (P<0.001), and had fewer attentional failures (EOG slow eye movements) during on-call nights (0.33/hr= 2.6/overnight versus 0.69/hr=5.5/overnight; P=0.02) on the intervention schedule

19. Relationship between work hours and sleep duration on two schedules

20. Most but not all interns slept more on intervention schedule

21. Results of RCT 2203 patient days, 5888 hours of observation During traditional schedule, interns made: 35.9% more serious medical errors 27.8% more intercepted serious medical errors 56.6% more nonintercepted serious medical errors 20.8% more serious medication errors 5.6 times as many serious diagnostic errors No difference in procedural errors No significant change in other staff errors No significant change in # of medications or procedures, tests interpreted, LOS, mortality

22. Incidence of Serious Medical Errors Landrigan C et al. N Engl J Med 2004;351:1838-1848

23. Evaluations of 1988 Libby Zion Laws • No relative improvements in mortality for AMI, CHF, pneumonia in teaching vs. non-teaching hospitals1 • Significant increases in proportion of patients with at least one complication (35% vs. 22%, p=.002) and in delays in diagnostic tests (17% vs. 1.9%, P<.0001)2 • Compliance poor (60% of surgical residents exceeded 95 hours per week – 1997 audit) 1 Howard, Silber, Jobes JGIM 2005 2 Laine JAMA 1993

24. Effect of work hours reform in NY teaching hospitals on smoothed rates of Patient Safety Indicators Poulose BK, et al., Ann Surg 2005;241:847-860

25. Effect of work hours reform in NY teaching hospitals on smoothed rates of Patient Safety Indicators Poulose BK, et al., Ann Surg 2005;241:847-860

26. Benefits versus Harms • Benefits: • More sleep, better sleep, better quality of life • Better cognitive performance, fewer errors • Caveat #1: while duty hour rules reduce total number of hours work per week, 30 hour shifts allowed • Caveat #2: we don’t know how much more residents are actually sleeping • Harms: • Less opportunity to observe trajectory of illness • More frequent hand-offs • Studies have shown higher rates of significant adverse events when patients are “cross covered” (26% vs. 12%, OR=3.5)1 1Petersen et al, Annals of Int Med 1994 121: 866-872.

27. First study of impact on mortality:Inpatient only, different samples by year 0.25% absolute reduction; 3.75% relative reduction Shetty, Ann Intern Med, 2004

28. Our Study CohortsVolpp K, et al. JAMA 2007;298(9):975-1001 • All unique patients admitted between July 1, 2000 and June 30, 2005 (3 yrs pre-reform, 2 yrs post-reform) • Principal diagnoses: AMI, CHF, GI bleed, or stroke • DRG classification of general, orthopedic, or vascular surgery • VA • 320,685 patients, 131 hospitals • Data from VA Patient Treatment File (PTF) and Beneficiary Identification Record Locator System (BIRLS), VA Office of Academic Affiliations • Medicare • 8,529,595 patients, 3321 hospitals • Data from the Medicare Provider Analysis and Treatment File (MEDPAR), denominator files, Medicare Cost Reports

29. Effects measured by comparing pre- to post-reform changes in mortality in hospitals of differing teaching intensity EFFECT = Diff A – Diff B (D-in-D) Diff A1. DiffA2 DiffB ACGME Reform Logistic regression used to adjust for patient comorbidities, secular trends, hospital site where treated using “difference in differences”

30. Medicare - No significant relative change in mortality according to teaching intensity Volpp KG et al. JAMA; 2007: 298 (9): 975-983.

31. VA - Significant relative improvement in mortality among medical patients in post-reform year 2 Volpp KG et al. JAMA; 2007: 298 (9): 984-992.

32. How big were these effects? Medical patients: Improvement in mortality from pre-1 to post-2 of 0.70 percentage points (11.1%) for hospitals in 75th compared to 25th percentile Volpp KG et al. JAMA; 2007: 298 (9): 984-992.

33. VA Hospitals Medicare VA hospitals much more teaching intensive Volpp KG et al. JAMA; 2007: 298 (9): 975-983; Volpp KG et al. JAMA 2007; 298(9): 984-992.

34. Do effects of reform on mortality vary across hospitals (Medicare)?

35. Why no improvement in quality among Medicare patients? • Design flaws • 30 hour shifts allow acute sleep deprivation • Current design does not respect circadian rhythms • Sleep inertia at night when paged • Implementation • Compliance likely incomplete; may be worse than in VA hospitals, given higher work intensity • Offsetting factors • Worsened continuity • Higher work intensity • Sicker patients

36. Why improvement in some groups but not others? • VA vs. Medicare • VAs more teaching intensive (“dose response”) • Better information systems may have mitigated some of the continuity of care (hand-off) problems • Confounding due to other changes • Medical vs. surgical • Differences in balance between reduction in fatigue and continuity? • Differences in compliance? • Differences in effort to address discontinuity through structured sign-out, increased attending involvement?

37. Failure to Rescue:Death among surgical patients with potentially treatable complications

38. Highest risk patients fared no differently than lower risk patients - Medicare Volpp KG et al. JGIM 2009. In Press.

39. Highest risk patients fared no differently than lower risk patients - VA Volpp KG et al. JGIM 2009. In Press.

40. The concept of prolonged stays

41. Non-teaching (0) Very Minor/Minor (>0 & <.25) Major (>0.25& <0.6) Very Major (>0.6) The rate of prolonged stays varies little over time Silber et al. 2009. Medical Care

42. Odds of prolonged stay change at similar rates in more vs. less teaching intensive hospitals Silber et al. Medical Care 2009. ap<0.05 bp<0.01 cp<0.001

43. AHRQ Patient Safety Indicators • Technical composite • iatrogenic pneumothorax • foreign body left in during procedure • postoperative wound dehiscence • accidental puncture or laceration • postoperative hemorrhage or hematoma • Continuity of Care composite • postoperative physiologic or metabolic derangement • postoperative pulmonary embolism or deep vein thrombosis (PE/DVT) • postoperative sepsis • Collaborative Care composite • postoperative hip fracture • postoperative respiratory failure • selected infections due to medical care

44. PSI composite rates change at similar rates in hospitals of different teaching intensity Rosen et al. Medical Care. 2009. In Press.

45. Odds of experiencing a PSI generally changed at similar rates in more vs. less teaching intensive hospitals Rosen et al. Medical Care. 2009. In Press.

46. Results Summary • Good news? • No evidence of worsening of outcomes for a broad range of measures within either Medicare or VA • No evidence of harm (or benefit) for high-risk patients • Question about prolonged length of stay for vascular surgery patients • Bad news? • No evidence of significant relative improvements in outcomes except for medical patients in VA in post-reform year 2

47. Institute of Medicine 2008 report The Institute of Medicine formed a consensus committee to:  1) synthesize current evidence on medical resident schedules and healthcare safety. 2) develop strategies to enable optimization of work schedules to improve safety in the healthcare work environment. The strategies recommended will take into account the learning and experience that residents must achieve during their training. The recommendations will be structured to optimize both the quality of care and the educational objectives.

48. What do we not know? • Hours per week residents are actually working • How much more sleep residents are actually getting • Impact on broader range of clinical outcomes • Longer-term impact on clinical outcomes • Impact on educational outcomes • How residents are spending their time • What approaches have helped programs successfully adapt • Comparative effectiveness and cost effectiveness of different approaches • Role of hospital finances IOM Report on Resident Work Hours. 2008.

49. IOM Recommendations • “Safe transportation options” • Minimize work “that is of limited or no educational value, is extraneous to their program’s goals and objectives, and can be done well by others” • “Adequate time to conduct thorough evaluations of patients and for reflective learning…” • Specialty-specific, RRC-set limits on caseload • Supervisory physician (resident) in house at all times • Schedule overlap time and facilitate safe handoffs

50. IOM Recommendations • 80 hour week • 5 hour mandatory nap between 10pm-8am if overnight shifts used • Averaging of days off not allowed; 1 day per week and 5 days per month • All moonlighting counted against limits • 10 hours off after day shift; 12 hours off after night shift; 14 hours off after extended shift