“Overnight Activities of Medical Students On Call: Is it really educational?”

1. “Overnight Activities of Medical Students On Call: Is it really educational?” Jaron McMullin, B.S. Rebecca Greenband, B.S. Raymond Price, M.D. Leigh Neumayer, M.D., MS Division of General Surgery, University of Utah

2. Background • Healthcare is shifting from inpatient to outpatient • Traditional inpatient clerkships have been slow to change • Surgery, medicine, pediatrics, psychiatry, OB/GYN • Students spend up to every third night in the hospital

3. Documenting Student Activities: Self Report vs. Observer Report • Cook, 1992 • Correlation of 0.86 between observer records and student journals • 25 students observed for 59.25 total hours No other efforts to validate self-report

4. Past Efforts to Document Student Activities • Self-report via diaries and/or questionnaires • Libbin, 2003; Bahn, 2003; Millikan, 2001; Murray, 2001; Lawrence, 1994; Haken, 1988 and others • Both self report and observer report • Cook, 1992; La Palio, 1983 None offer a breakdown of overnight call activities and whether they are educational

5. Study Objectives • Compare self-report to an objective third-party observer • Determine how students spend their time on overnight call • Educational • Non-educational • Personal

6. Methods: Study Approval • Approval granted by • University of Utah Health Sciences Center (UUHSC) IRB • Intermountain Health Care Human Subjects Committee

7. Methods: Participant Eligibility • Students on the required 6 week third year surgery clerkship were recruited • Students rotating where overnight call is assigned, either UUHSC Hospital or Latter-day Saints Hospital, were eligible. • Students at either the Veterans’ Affairs or Primary Children’s Hospitals do not take overnight call and were not enrolled

8. Methods: Recruitment • Volunteering students • Gave informed consent • Were randomized to one of two groups: self-report or shadow • Were to return journal forms upon completion

9. Methods: Journals • Participants kept 1, 2, or 3 journals of ~24 hours each • Wednesdays were not included because they are a scheduled didactic day

10. Methods: Journals • Weekend “days” • 7am Saturday until 7am Sunday • 7am Sunday until 7am Monday • Weekday journals • Started at the time the student arrived at hospital • Completed at 7am the following morning • Shadowed students were overtly followed by one of two trained observers • 24 hours on weekends • 6pm to 7am on weekdays

11. Methods: Activity Definition • Educational:patient care activities related to educational objectives (working up a patient, actively participating in procedures, reading surgery-related literature and/or texts, etc.); • Non-educational:patient care activities unrelated to educational objectives (such as gathering information for residents, finding radiographs, etc.); • Personal:personal care (sleeping, showering, eating etc.).

12. Methods: Analysis • Calculated proportion of time that was Educational, Non-educational, or Personal • Students’ weekday journals (before 6pm) served as a baseline of educational activity

13. Method: Analysis • T-tests compared student weekday journals to • Weeknight • Weekend days • Weekend nights • Paired t-tests compared student journals to concomitant observer journals • Bonferroni correction was used for multiple testing; a p-value of <0.005 was considered significant

14. Results: Recruitment • 34 students of 74 eligible students rotating between May, 2004 and February, 2005 • gave informed consent • were randomized to one of two groups: self-report or shadow • 5 students (15%), all self-report, withdrew or didn’t turn in journals despite multiple reminders

15. Results: Journals • 29 students completed logs • 16 students shadowed at night; 13 were not followed • 68 days and 70 nights of student journals • >1600 hours of journal compiled and analyzed • 27 weeknights, 11 weekend (24 hour) days of observer journals • 615 hours of journal compiled and analyzed

16. Results: Educational Student vs. Shadow

17. Results: Non-educational Student vs. Shadow

18. Results: Personal Student vs. Shadow

19. Results: Comparisons between Student Groups and with Observer Journals • Shift comparisons between Shadowed and Self-report student groups showed NO significant differences • Observed Students versus Observer showed NO significant differences on any shift

20. Results: Comparison of activities between shadow and self-report groups.

21. Results: Comparing Weekdays to Weeknights • More time in educational activities during the day (p<0.005) • More personal time (usually sleep) at night • No difference in percent of time in non-educational activities

22. Results: Comparing Weekdays to Weekends • More time in educational activities on Weekend days • No differences in time spent in non-educational activities • More time in personal activities on weeknights and weekend nights than on weekend days (p<0.005)

23. Discussion: Validating Self-Report • Students’ and shadows’ records varied by less than 10% • In a study of smaller magnitude, Cook et al (1992) that found a 0.86 correlation between observer and student records • On average, records of students who were shadowed showed no difference from non-observed students This supports future use of self-report as an accurate method to collect student activity data

24. Discussion: Student Activities • Weekdays • Nearly 80% of time engaged in educational activities • This despite not counting Wednesdays, our scheduled didactic day (Wednesdays) • Weekend days • Over 60% of students’ time spent in educational activities • Weeknights, weekend nights • Is the 45% of educational time worth the sleep deprivation? • Should overnight call perhaps be assigned just during 4th year electives rather than 3rd year clerkships?

25. Discussion: Limitations • Our study averages data from students in two hospitals • University of Utah Hospital (tertiary-care, state funded, teaching hospital) and Latter-day Saints Hospital (tertiary-care, private hospital) • Each hospital has unique teaching settings and in-house call schedules • Different patient populations and payor mixes might find differences from what we found • Student and trained shadow judgment was used to categorize activities; attending surgeons may categorize the activities differently

26. Conclusions • This data will be useful in planning future activities for medical students while on the surgery clerkship • Time in the hospital on weekdays and weekend days tends to be largely educational • The trade-off of sleep deprivation for little educational gain should be considered when requiring 3rd year surgery clerks to take overnight call