Implementation and evaluation of a standardized briefing checklist at shift handoff in a pediatric emergency department

1. Implementation and evaluation of a standardized briefing checklist at shift handoff in a pediatric emergency department Paul C. Mullan1, MD MPH Sartaj Alam2, PhD Charles G. Macias2, MD MPH Deborah Hsu2, MD Med Binita Patel2, MD 1Children’s National Medical Center – George Washington School of Medicine 2Texas Children’s Hospital – Baylor College of Medicine

2. Disclosure • No financial disclosures or conflicts of interest from any study authors

3. Background • High error rates with serious consequences occur in Emergency Departments1 • Ineffective communication was a root cause in 82% of sentinel events2 • National Patient Safety Goal 2011-2012: • Improve the effectiveness of communication among caregivers3 1To Err is Human, IOM, 2000 2Joint Commission (2010 data) 3Joint Commission, 2011.

4. Background • ED handoff risk factors for error: • Interruptions, fatigue, lack of pre-determined teams • ED Handoff adverse outcomes: • Delays in care and disposition1 • Increased medico-legal liabilities2 • Survey of EM and PEM fellowship directors3 • 72% agreed that a standardized sign-out system would improve communication and reduce errors 1Smith D, 2011. 2Kachalia A, 2007. 3Sinha M, 2007.

5. ED Handoff Components 1. Patient information 2. Situational awareness information • Resources, challenges, safety issues • Goal: anticipate and react more effectively • Potentially easier to standardize

6. Accomplishing Situational Awareness • Briefing1 • Planning event prior to another event • Explain goals • Get team input • Cover contingencies • Ensure role awareness • Operating Room: • Surgical Safety Checklist: mortality ↓1.5 to 0.8% (p=0.003)2 • Communication failures ↓3.9 to 1.3 per surgery (p<0.01)3 • ED: large variation in briefing content and practice 1AHRQ, 2009. 2Haynes, 2009. 3Lingard, 2008.

7. Briefing Checklist Creation • Best practice guidelines • Checklist, human factors, aviation literature • Stakeholder focus groups • Staff training and feedback • Periodic modifications per user feedback • 10 versions in study period

8. Briefing Checklist

9. Objective • To describe the implementation and user perceptions of a standardized briefing checklist for improving situational awareness during physician handoff in a pediatric ED • Pediatric Active Shift Signout in Emergency Department: • PASSED Checklist

10. Briefing Checklist Setting • 3° pediatric level-one trauma center • Academic ED • ~80,000 patients annually • Checklist used in main ED area only • 24 beds, 2 resuscitation beds • Central desk area with computer availability

11. Briefing Checklist Users • Staffing present at 8 AM & 8 PM checklist usage: • PEM providers: attendings(1-3), fellows (1-3) • Pediatric & EM Residents (4-8) • Medical students (0-4) • Charge nurses (1-2) • Respiratory therapists (0-2)

12. Methods • Part 1 • Quantitative observational study of checklist process measures • Part 2 • Qualitative perception survey by checklist users

13. Part 1: Quantitative Observational Study of Checklist Process Measures • Usage rate (if any items checked) • Completion rate (≥80% of items checked) • Timing • Identification of potential safety events

15. Checklist & Handoff Performance5/6/2011 – 5/5/2012 (n=732) Mean checkout duration of 18 minutes (Checklist + patient information) Mean of 14 patients per handoff Mean of 82 seconds/patient

16. Potential Safety Events 84% of checklists: ≥1 Event 24% of checklists: ≥3 Events No significant change in mean time per patient handoff if 0, 1, 2, 3, or 4 potential safety events

17. Part 2: Qualitative Perception survey by checklist users • 1 year post-implementation • Population: user experience in the ED pre- and post-checklist • PEM attendings, PEM fellows, peds residents, charge nurses • Online survey development • Face validity focus groups, pilot tested, clinical sensibility tool testing, intra-rater reliability, Perceived contributions to 3 domains: • Situational awareness • Institute of Medicine quality domains • Usability • 3 reminder emails over a two-week period

18. Perception Survey Test-retest intra-rater reliability: Kappa of 0.62 Cronbach’s alpha of 0.70 Excluded: 2 of 94 (2%) were partial respondents

19. Survey Results:Situational Awareness Domain *p<0.05 1: 0-20% 2: 21-40% 3: 41-60% 4: 61-80% 5: 81-100%

20. Survey Results:Institute of Medicine Quality Domains *p<0.05 1: Strongly disagree 2: Disagree 3: Neutral 4: Agree 5: Strongly agree

21. Survey Results:Usability Domain *p<0.05 1: Strongly disagree 2: Disagree 3: Neutral 4: Agree 5: Strongly agree

22. Limitations • Recording bias: • Commission & omission errors possible • No verification process • Contamination bias: • Simultaneous debriefing program • Unable to causally link briefing checklist to safety outcomes

23. Conclusions • The PASSED briefing checklist has a high usage and completion rate • Potential safety events frequently identified • Users perceived that the briefing checklist improved team checkout • Situational awareness • Communication • Quality of care • Future studies • Standardized patient specific handoff information • Patient outcomes

24. Appreciation • Fellows, attendings, residents, & charge nurses of Texas Children’s Hospital • TCH section leadership • Joan Shook, Paul Sirbaugh • Survey development helpers • Presentation review: • Binita Patel, Jim Chamberlain • Study implementers • Binita Patel • SartajAlam • Charles Macias • Deborah Hsu

25. References • Agency for Healthcare Research and Quality. TeamSTEPPS: Instructor guide. Accessed 2011;2012 • Agency for Healthcare Research and Quality. Briefs and Huddles Toolkit Overview. Aug 2009. • Cheung DS, Kelly JJ, Beach C, et al. Improving handoffs in the emergency department. Ann Emerg Med. 2010;55:171-80. • Haynes AB, Weiser TG, Berry WR. A surgical safety checklist to reduce morbidity and mortality in a global population. New Engl J Med. 2009;360;5: 491-499. • Kachalia A, Gandhi TK, Puopolo AL, et al. Missed and delayed diagnoses in the emergency department: A study of closed malpractice claims from 4 liability insurers. Ann Emerg Med. 2007;49:196-205. • Kohn, L., Corrigan, J., Donaldson, M., Committee on Quality of Health Care in America, Institute of Medicine, eds. To err is human - building a safer health system. 2000 • LingardL, Regehr G, Orser B, et al. Evaluation of a preoperative checklist and team briefing among surgeons, nurses, and anaesthesiologists to reduce failures in communication. Arch Surg 2008;143:12–17. • Sinha M, Shriki J, Salness R, Blackburn PA. Need for standardized sign-out in the emergency department: A survey of emergency medicine residency and pediatric emergency medicine fellowship program directors. AcadEmerg Med. 2007;14:192-6. • Smith D, Burris JW, Mahmoud G, Guldner G. Residents' self-perceived errors in transitions of care in the emergency department. J Grad Med Educ. 2011;3:37-40. • The Joint Commission. Sentinel event data - root causes by event type 2004 - 1Q 2012. April 2012;2012 • The Joint Commission: National Patient Safety Goals. Accessed 4/25/2013: http://www.jointcommission.org/assets/1/18/2011-2012_npsg_presentation_final_8-4-11.pdf

26. Conclusions • The PASSED briefing checklist has a high usage and completion rate • Potential safety events frequently identified • Users perceived that the briefing checklist improved team checkout • Situational awareness • Communication • Quality of care • Future studies on patient outcomes needed Questions?

27. End of presentation

28. To improve situational awareness and quality of care delivered to ED patients, we used QI methodology to develop and implement a briefing checklist for use at ED physician handoff rounds. We hoped that our checklist used at handoff would provide an additional protective component of safety that might prevent adverse events (35). The PASSED briefing checklist was adopted by users in the ED as evidenced by its high rate of use. In the vast majority of handoffs, the checklist identified safety components that might not have been discussed otherwise by the team. As a balance measure, the identification of safety components during the briefing checklist did not add to the average time required to handoff patients. Our average patient handoff duration of 82 seconds per patient is comparable to adult ED setting studies that reported an average of 73-92 seconds (10,22). We were satisfied with the 89% completion rate especially given the multiple changes in checklist content and architecture that occurred during the study period; greater than 80% completion of a checklist has been associated with improved patient outcomes in the surgical literature (27). Our perception survey did demonstrate significant improvements with use of the checklist in both combined and individual user groups in all areas of ED situational awareness and the IOM quality domains of safety, efficiency, and effectiveness. For our primary survey outcome, the combined and individual user groups all agreed that the checklist had helped to improve communication. The only question that had disagreement among user groups related to the appropriate duration of handoff with the new checklist between two groups: pediatric residents and the charge nurses. This might have been due to a general sentiment among residents that any process that adds time to the end of their 12-hour shift is inappropriate. For the charge nurse shifts (and for the majority of PEM attending and fellow shifts), the added handoff process did not add to the length of their shift; these data argue in favor of handoff being integrated into, not added onto, an ED shift. We identified many key components to the successful creation and implementation of a checklist. One early concern from stakeholders was the need to keep the checklist short and relevant. We utilized focus groups, pilot testing before implementation, elicitation of end-user feedback post implementation, and feedback-based modifications to the checklist to keep and gain high acceptance of the checklist among users. From a human factors standpoint, we realized early on that the layout and ordering of checklist items significantly affected completion rates. By placing more checklist items at the start of the checklist, prior to starting the discussion of individual patients, we improved overall compliance. We attribute this finding to team members being conditioned to walking away from rounds at the end of discussing the last ED patient on the tracking board in the pre-checklist implementation system. Finally, in the early implementation phase, creating standard practices helped improve process adoption. The number of shifts that particular PEM attendings work in our main ED varies which can contribute to unfamiliarity with the process. However, our PEM fellows are consistently present in the main ED. By requiring fellows to lead checkout when they are present at handoff, we encouraged development of leadership skills and also helped model and provide consistency to the process. Initially compliance decreased on non-fellow days, but it improved with time as more PEM attendings participated in the new process and accepted it as standard routine. We also realized that national holidays were a high-risk time for low usage of the checklist; of the 16 checklists not used (missing or blank), 8 (50%) were from dates within 3 days of national holidays. Future rapid cycle interventions of this QI project will aim to standardize individual patient handoffs, incorporate formal teaching elements into the handoff, and implement the checklist in non-learner based areas of our ED. • LIMITATIONS • There were some limitations in the design and results of our study. Because frequency of checklist completion and potential safety events reported were determined by the handoff leaders’ recorded values on the checklists, it is possible that there could have been under- or over-reporting of some checklist components. One factor limiting this phenomenon was that the checklist was completed in visible proximity to multiple other providers who were often looking at the checklist simultaneously. A video study would be needed to assess for accurate rates of checklist omission and commission error. Also, this study was not able to assess objective impacts on patient outcomes or ED time processes given the retrospective nature of the study design, the lack of pre-intervention data, and multiple other confounders that would interfere in establishing any direct causality by the checklist itself. To partially account for this factor, all survey questions were given to subjects with ED experience both before and after the checklist implementation, and questions were designed to assess for the before-and-after effects attributable to the intervention itself. Whereas the results of our survey were generally very positive, these perception changes could have been influenced by other simultaneous changes that occurred during the time of this study. There were no other programs specifically targeted at the handoff process; however, a communication-oriented program on debriefing team members in situ following pediatric resuscitations was implemented during the study period (36). From a timing standpoint, we were limited in that we captured only the time from handoff start to finish; a more precise method, which was deemed not to be feasible from an end-user acceptability standpoint, would have been to record the duration of checklist completion specifically. Anecdotally, the duration of completing the checklist typically was 1-3 minutes, similar to what has been reported anecdotally elsewhere (9). Lastly, our checklist did not address the standardization of communicating actual patient information during the handoff process. Efforts are currently underway at our institution and within the American Academy of Pediatrics’ Section of Emergency Medicine to address this area. • CONCLUSION • The handoff of patient care in the ED has the potential for identifying serious safety events. Cognitive aids such as checklists have been studied as a means to improve provider cognition that can be impaired by internal stress, environmental factors, and human factors (37,38). We have developed and implemented a briefing checklist to be used at ED handoff to improve ED situational awareness and to identify potential safety events. We demonstrated a high rate of identifying these events, and users perceived that the checklist contributed to improved communication along with enhanced efficiency, effectiveness, and safety of care delivery. Future studies are needed to examine the effect of such a briefing checklist on ED throughput processes and patient outcomes.

29. 1. Why is this topic important? Handoffs of care at physician shift change in the emergency department (ED) are at high risk for medical errors and potential safety events. A standardized method for improving situational awareness of the ED might improve patient safety. • 2. What does this study attempt to show? This study describes the creation and implementation of a standardized briefing checklist used at ED shift handoff. • 3. What are the key findings? A mean of 1.7 potential safety events were identified at each handoff. Users were surveyed and noted that the checklist promoted better communication, safety, and situational awareness. • 4. How is patient care impacted?By identifying potential safety events at the start of a shift, checklist users perceived to have better situational awareness of their environment. Future studies are needed to examine the effect on ED throughput processes and patient outcomes.

30. Framework Analysis of Free-Text Comments A:Attending. C:Charge Nurse. F: Fellow. IOM: Institute of Medicine.