Moving from FIRST to SECOND:

1. Moving from FIRST to SECOND: Karl Bilimoria MD MS John B. Murphy Professor of Surgery Vice President – Quality, Northwestern Medicine Director, Surgical Outcomes and Quality Improvement Center Associate Program Director Feinberg School of Medicine, Northwestern University Yue-Yung Hu MD MPH Assistant Professor of Surgery Associate Program Director for Wellness Lurie Children’s Hospital David Hoyt, MD Executive Director American College of Surgeons A Proposal for Another National Trial to Improve the Surgical Learning Environment

2. Surgical Community Came Together for FIRST Trial and Changed National Policy

3. FIRST Trial Examined Resident Well-Being Randomize General Surgery Programs Flexible Policy Standard Policy vs. Data Collection: July 1, 2014 to June 30, 2015 PATIENT OUTCOMES Primary Outcome: Death or Serious Morbidity Composite Secondary Outcomes: Death, Serious Morbidity, Any Morbidity, Failure to Rescue, Reoperation, Complications RESIDENT OUTCOMES Primary Outcomes: Overall Wellbeing and Education Secondary Outcomes: Resident perceptions of care, continuity, education, wellbeing, personal safety; Test performance

4. Prevalence of Burnout Symptoms among General Surgery Residents in the U.S. • At least weekly symptoms • Emotional exhaustion 53.2% • Depersonalization 25.6% • Psychiatric wellbeing • Median 3 (IQR 1-5) • 43% met criteria for poor psychiatric wellbeing 39% From national post-ABSITE survey with 99% response rate

5. In Examining the Long-term FIRST Trial Results, Resident Well-being Worsened Over Time in Both Arms

6. Focus Groups and Interviews with PDs • “Always hearing about resident burnout and wellness problems” • Hard to quantify • Programs have no way to measure or benchmark • Hard to solve • Not sure what to implement or even what the options are • Interventions are only available by word-of-mouth; no catalog • Interventions are context-specific and rarely surgical – not easy to export • No robust data on whether existing interventions work

7. FIRST Trial Participants Poised to Become an Innovative Surgical Education Trials Group Abington Memorial Hospital Alegent Creighton Health, Creighton University Medical Center Baptist Memorial Hospital Memphis Barnes-Jewish Hospital at Washington University Medical Center Barnes-Jewish West County Hospital Baylor University Medical Center Baystate Medical Center Beaumont Hospital Grosse Pointe Beth Israel Deaconess Medical Center Brigham & Women's Hospital Brigham and Women's Faulkner Hospital Bronson Methodist Hospital Carilion Clinic/ Carilion Medical Center Carle Foundation Hospital Carolinas Medical Center Christiana Care Health System Cleveland Clinic Foundation Cooper University Hospital Dallas County Hospital District d/b/a Parkland Health & Hospital System Danbury Hospital Dartmouth-Hitchcock Medical Center Duke University Hospital Eisenhower Army Medical Center Emory University Hospital Erlanger Health System at Chattanooga Exempla Saint Joseph Hospital Fletcher Allen Health Care - Hospital George Washington University Hospital Good Samaritan Hospital (TriHealth) Gundersen Lutheran Medical Center Hackensack University Medical Center Hahnemann University Hospital Hartford Hospital Hennepin County Medical Center Henry Ford Hospital Hospital of the University of Pennsylvania Houston Methodist Hospital Indiana University Health - Methodist Hospital Indiana University Hospital, IU Health Inova Fairfax Hospital Intermountain Medical Center Iowa Methodist Medical Center Johns Hopkins Hospital Kaiser Foundation Hospital Oakland Kaiser Foundation Hospital Sacramento Kaiser Permanente San Francisco Medical Center Kaiser Permanente Santa Clara Medical Center Kaiser Sunnyside Medical Center Kapiolani Medical Center for Women & Children Lahey Clinic Legacy Emanuel Hospital & Health Center Legacy Good Samaritan Medical Center Maine Medical Center Massachusetts General Hospital Mayo Clinic Arizona d/b/a Mayo Clinic Hospital Mayo Clinic Hospital Rochester - Methodist Campus Mayo Clinic Rochester – Saint Marys Campus Medical Center of Central Georgia Medical University Hospital Authority Memorial Health University Medical Center Memorial Hermann Hospital - TMC Memorial Hermann Southwest Memorial Medical Center Mercy Medical Center (Des Moines, IA) Meriter Hospital Methodist University Hospital MetroHealth Medical Center Morristown Medical Center Mountain States Health Alliance d/b/a Johnson City Medical Center Naval Medical Center Portsmouth, Virginia New Hanover Regional Medical Center Newark Beth Israel Medical Center Newton-Wellesley Hospital Northwestern Memorial Hospital Ochsner Clinic Foundation Oregon Health & Science University Orlando Regional Medical Center (FSCI) OSF Saint Francis Medical Center Penn State Milton S. Hershey Medical Center Pennsylvania Hospital, UPHS Providence Portland Medical Center Providence St. Vincent Medical Center Rhode Island Hospital Riverside County Regional Medical Center Robert Packer Hospital Ronald Reagan UCLA Medical Center Rush University Medical Center Saint Francis Hospital - Memphis, TN Saint Francis Hospital & Medical Center Saint Joseph Mercy Hospital Saint Louis University Hospital Saint Mary's Hospital (CT) Saint Thomas West Hospital Saint Vincent Hospital - Indianapolis Santa Barbara Cottage Hospital Scott & White Hospital Sinai Hospital of Baltimore Sparrow Hospital Spectrum Health Hospitals Stamford Hospital Stanford Hospital and Clinics Straub Clinic and Hospital Sutter West Bay Hospitals d/b/a California Pacific Medical Center Tampa General Hospital (FSCI) Tarrant County Hosp District / JPS Health Network Temple University Hospital The Christ Hospital The Hospital of Central Connecticut The Jewish Hospital The Miriam Hospital The Nebraska Medical Center The Ohio State University Wexner Medical Center The Queen's Medical Center The Regional Medical Center at Memphis Thomas Jefferson University Hospital Truman Medical Center Tufts Medical Center UF Health Jacksonville (FSCI) UMass Memorial Medical Center UNC Hospitals University Hospital (Newark, NJ) University Hospital (San Antonio, TX) University of Alabama at Birmingham Hospital University of California Davis Medical Center University of California Irvine Medical Center University of California San Francisco Medical Center University of Colorado Hospital University of Connecticut Health Center Finance Corp On behalf of University of Iowa Hospitals and Clinics University of Kansas Hospital - Kansas City, KS University of Kentucky Hospital University of Maryland Medical Center University of Minnesota Medical Center, Fairview University of Missouri Hospital University of Tennessee Medical Center-Knoxville University of Texas M.D. Anderson Cancer Center University of Texas Medical Branch University of Utah Hospitals and Clinics University of Virginia Health System at Charlottesville University of Washington Medical Center University of Wisconsin Hospital & Clinics UPMC Presbyterian UT Southwestern University Hospital Vanderbilt University Hospital Vidant Medical Center Wake Forest Baptist Health Wellmont Bristol Regional Medical Center West Virginia University Hospitals William Beaumont Hospital (Royal Oak) Winchester Medical Center Womack Army Medical Center York Hospital Beaumont Hospital Grosse Pointe Detroit Receiving Hospital Harper University Hospital Henry Ford Hospital Huron Valley Hospital Sparrow Hospital Spectrum Health Butterworth St. John Hospital and Medical Center St. Joseph Mercy Hospital University of Michigan Health System

8. The SECOND Trial is a Partnership

9. The SECOND Trial: A National Pragmatic, Cluster-Randomized Trial 320 Eligible Surgical Residency Programs Control vs. Intervention Timeframe: 9/1/2019 to 6/30/2022 Resident Wellbeing: Burnout*, job satisfaction, attrition, suicidal thoughts Educational Outcomes: Exam performance, CaseLog, Operative Performance Patient Outcomes: ACS NSQIP and Medicare ~7/1/2022: All Get Intervention

10. All Programs Receive Wellness Report & Suicidality Interventions • Each program receives a nationally benchmarked report of their residents’ well-being based on de-identified responses from ABSITE survey • All programs get interventions to address suicidal thoughts • Recipients: • PD • Chair • DIO

11. Intervention Programs Receive Additional Information, Access to Wellness Toolkit, & Implementation Support • Toolkit • Targeted Reports • Wellness Toolkit • Emotional Exhaustion • SMARTR training • Peer support training • Difficult case • conferences • Protected half-days • Scheduled group • counseling Targeted Improvements Toolkit • Implementation Support • Process improvement coaching • Collaboration networking • Topical expertise

12. Additional Themes about the Learning Environment Are Under Development • Service vs Education Balance • Autonomy • Camaraderie • Faculty Engagement • Dealing with Adverse Events • Bullying • Harassment • Discrimination • Duty Hours Violations • Residents Preparedness to Respond ABSITE resident survey PD survey ACGME resident survey ACGME faculty survey Interviews/focus groups

13. Wellness Toolkit is Currently Being Developed • Program Director Survey (Wellness Inventory) • Phone calls with program directors & coordinators to clarify/gather additional detail • Literature review & guidelines • Program Tours (20-30) • Identify by • Data • PD survey • Snowball sampling • Observations • Interviews/focus groups • Policy/procedure review • Residents • Attendings • Nurses • Chair and PD • GMEC / DIO

14. Sample Wellness Toolkit Interventions Chair: Mary Hawn, MD PD: David Spain, MD (dspain@stanford.edu) Chair: Rebecca Minter, MD PD: Jacob Greenberg, MD (jgreenberg@uw.edu) Chair: John Sweeney, MD PD: Keith Delman, MD (kdelman@emory.edu) High Performance Team Training Lactation Policy Half-Weekday Off per Month • How we did it • Identifying a psychologist who understands surgical training • Convincing surgeons to go • Arranging coverage • What did it cost? • Keys to success • How we did it • Finding appropriate space at each clinical site • Training faculty/residents • Keys to success • How we did it • Arranging coverage • Mitigating safety issues • Keys to success

15. Potential Ways to Ensure Education and Patient Care Are Unaffected by Well-Being • For resident education • ABSITE (scores) • ABS QE & CE (pass rates) • ACGME CaseLog (procedural volumes) • ABSITE surveys (perception) • SIMPL/ZwischMe (operative autonomy)* • For patient care • ACS NSQIP (death, morbidity)* • Medicare (death, morbidity, cost) • Intervention vs. Control • Intention to treat • As treated *subset of programs

16. Frequently Asked Questions • How do you protect confidentiality? • For residents • ABSITE survey data de-identified • Program-specific reporting in quartiles (not number or % of residents) • Suppress items with low denominators (e.g., gender, minority) • Program Tour interviews confidential; no feedback to leadership

17. Frequently Asked Questions • How do you protect confidentiality? • For residents • For programs • ACGME has no access to data (contractual) • Programs are not allowed to advertise their own data (contractual) • Individual program chairs/PDs decide whether to release their own data to their own residents/faculty/administration

18. Frequently Asked Questions • How do you protect confidentiality? • Does being in the control arm mean I can’t work to improve my residents’ wellness? Does being in the intervention arm mean I have to do everything in the Wellness Toolkit? • Pragmatic Trial: no mandates or prohibitions, all programs get to pick entirely what they work on and implement • Control Arm: Review data; Intervene on their own (without Wellness Toolkit access) and/or do nothing • Intervention Arm: Review data; Select interventions from Toolkit and/or elsewhere and/or do nothing • All report what they worked on in annual Program Director Survey • We want it to be easy for programs to participate

19. Proposed Timeline PD Survey PD Survey PD Survey PD Survey Program Tours Survey Survey Survey Survey Survey Toolkit Development Enrollment Toolkit Dissemination Dissemination of Intervention to Control Arm Randomization 2nd Report 1st Report 3rd Report 4th Report

20. How Do We Sign Up? • Complete Program Director Survey (personalized link was emailed to you) • SECOND Trial enrollment form • Signatories: PD, Chair, DIO • IRB process similar to FIRST Trial • Northwestern IRB: non-human subjects research • Up to each program whether they submit to their IRB and how • We can provide documentation if you choose to submit • We may call you after enrollment to clarify your Program Director Survey results

21. Moving from FIRST to SECOND: SECOND@northwestern.edu Karl Bilimoria MD MS Yue-Yung Hu MD MPH A Proposal for Another National Trial to Improve the Surgical Learning Environment