THE SURGICAL SAFETY CHECKLIST Dr Jacqueline Hannam Associate Professor Simon Mitchell

1. THE SURGICAL SAFETY CHECKLIST Dr Jacqueline Hannam Associate Professor Simon Mitchell Department of Anaesthesiology University of Auckland

2. Complications after inpatient operations occur in up to 25% of patients Reported crude mortality rate after major surgery is 0.5–5% In industrialized countries, nearly half of all adverse events in hospitalized patients are related to surgical care, and at least half of these considered preventable WHO SSC implemented at ACH in 2008 as part of the global pilot study (NEJM 2009;360:491-9) Overall study outcome; Mortality pre vs post = 1.5 vs 0.8% Complications pre vs post = 11% vs 7%

3. The checklist at Auckland Benjamin A BMJ 2008;336:1241-1245

4. What to audit? 1. Is it used? Van Klei et al

5. What to audit? 2. Is it used correctly? Increased odds of complication or death if information sharing was omitted or poor either intra-operatively or at patient handover.

6. Definitions Compliance = “verbal communication of that item by the checklist administrator or other OR team member during SSC administration” Engagement = “Engagement was rated according to the number of OR teams engaged. At least one team member had to be engaged in SSC administration for the team to be considered engaged, and engagement was defined as listening or contributing to SSC administration with cessation of other activities and conversations.”

7. Method of measurement? Retrospective note review: Possibly prone to inaccuracies and may not represent actual checklist compliance Cannot measure team engagement with process Direct observation: Time consuming and potentially costly, but preferred option

8. Accuracy of retrospective note review Checking the checkers: an audit of self-reporting checklist use • 41 SSC domains in two OR suites • Observer rated compliance with domains and domain items using previous definitions • Observer-rated compliance compared with self-reported compliance (boxes checked on SSC form in patient notes) • Accuracy in documentation investigated using logistic regression analysis with adjustment for confounding factors (operating suite and SSC domain)

9. Checking the checkers - results • Domain compliance recording accurate in 75% of domains • Item compliance recording was accurate for 51% of items • Likelihood of accurate self-reporting greater for items that were observed as being completed than for those that were not (OR 22.85, 95% CI 10.78-48.42, p<0.0001) • Example - item compliance reported as 86% at Suite 1 (68% true positive + 18% false positive) versus 82% of items at Suite 2 (15% + 67%). At face value these seem comparable BUT item compliance as rated by observers was 68% at Suite 1 and just 15% at Suite 2!

10. Checking the checkers - results Tendency to report compliance favourably when items are NOT completed Reliance on self-reported rates of compliance with SSC is likely to grossly over-estimate true compliance Big variation in observed compliance levels between OR suites within the same hospital, not detectable by retrospective note review Inaccurate representation of checklist practices within centres could hinder attempts to identify areas where improvements could be made.

11. Audit 1: Direct observation 100 cases at ACH Primary outcomes Secondary outcomes Compliance with administration of checklist domains and domain items Team Engagement 2 months, 2010 – 2011 (2 years post checklist roll out)

12. Domain compliance Sign In: 99% Time out: 94 % Sign Out: 2 % Item compliance Mean (range) Sign In: 56% (27 – 100) Time out: 69% (33 – 100) Sign Out: NA

13. Team engagement

14. Key findings of audit Sign In and Time Out are completed most of the time; including items that intuitively seem most important Sign out not being done!! Staff engagement sub-optimal A drift in administration standards appeared to have occurred since the study

15. Interventions? Present findings to OR staff Feedback and discussions with group “Sign Out is not linked to a specific event in patient management” Link Sign Out to the end of the swab & instrument count …re-audit to assess the intervention

16. Simultaneous audit of 100 cases at a study and a non-study hospital, 2011 Primary outcomes Secondary outcomes Team Engagement Compliance with administration of checklist domains and domain items

17. Compliance with domain administration

18. Compliance with checklist item administration

19. Team engagement Led by anaesthetist at hospital 2 Led by surgeon at hospital 2

20. Key Points A hospital which rolled out the checklist independent of a study protocol exhibited poor compliance This hospital may be more typical of mainstream New Zealand Implementation (or re-implementation) strategies potentially make a significant difference A further drift in engagement at Hospital 1, but Sign Out had improved Senior team members leading domains gets better engagement

21. Next step – sustain improvements • Attitude • Analysis and addressing of inappropriate beliefs • Motivation • Education • Addressing the more difficult quality issues • Team behaviours • Leadership • Involve all 3 professional groups • Change in OR checklist leadership

22. Current status – trialling OR team leadership

23. Domains Domain items