Systematic learning from mistakes: achievements and challenges

1. Systematic learning from mistakes: achievements and challenges Andy Sutherland, NHS Information Centre for health and social care

2. “All men make mistakes, but only wise men learn from their mistakes” (Winston Churchill) “Learn from the mistakes of others – you can never live long enough to make them all yourself” (John Luther)

3. Summary • Background and approach • Incidents – handling and analysis • Publication process • Improvement cycle • Present position • Reflections and next steps

4. Background • No consistent approach to handling • Internal panic • External and internal blaming • Perception of repeating errors • Low customer confidence

5. Approach • A system for handling things that go wrong • A system for learning from them

6. Adverse incidents process • Notification • Confirmation • Evaluation • Handling plan • Handling

7. or

8. AAHAA… • Alert • Internally to Director and Head of Profession • Assess • Impact, options, considerations • Handling plan • What to do, who to tell, when; • Authorisation • Action The first ‘A’ does not stand for ‘Action’!

9. And then • Review • Learning • Implementing changes

10. …by means of • Review meetings • Analysing root causes and drawing out lessons • Openly available documentation • Library of incidents • Library of root causes and lessons • Regular learning fora (therapy and action) • alerts

11. Analysis • 352 incidents March 2008 to May 2011 • Categorised by potential damage to NHS IC • 33 high • 199 medium • 120 near miss – eg trapped internally

12. Root cause analysis scoring • None • Evidence of thought, but not cause • A cause, but not a root cause • A reasonable root cause analysis

13. Good and bad root causes • “The cause of the problem was most likely due to the template being copied from another table.” • “High level of risk identified but not effectively managed” • “Not having a system… that was proven to meet clear and specific requirements.”

14. Lessons learned scoring • No evidence • Evidence of thought • Lessons described • Evidence of lessons implemented

15. Lessons learned – good and bad • “Additional checks to be implemented on the final report” • “Processes will become ever more robust now that the work has been brought within the IC” • “All web entries should have clear review dates attached …process for reviews…”

16. Incidents by month, March 08 – May 11

17. Incidents by month - Root cause found?

18. Incidents by month – lessons learned?

19. Number of incidents by department - lessons learned?

20. Source of incidents

21. Reflection • Incidents being reported • Handling improved (better feedback) • Root causes and lessons learned patchy • Little evidence of learning across organisation • Scope for action on publications

22. Publication process • Systematic approach • Guidance on each stage • Clear responsibilities • Clear records A process to improve!

23. Output Documents, Approvals and Records Planning Process Initiation Production Publication Input Guidance and Templates Completed Publication Reviewing Publication process

24. Approval: Roles and Responsibilities defined. Confirm agreement to policies and procedures Design Approval including customers and stakeholders Mandate Approval Publication Brief Approval Plan Approval Publication Mandate Brief Plan Design and Development Production Create the Team Process Initiation Publication Mandate Publication Brief Publication Plan Template Guidance on creating the publication team Design and Development guidance Planning

25. Approved set of data Data analysis approval Final draft approval Data Preparation approval Analysis Prepare draft publication Planning Data Preparation Publication Data Analysis process Preparing the draft publication guidance Data Preparation guidance Protocols for checking the analysis Style Guide Production

26. Approval and record of confirmation of proof reading Pre-Publication approval Production Pre-publication Printing and Distribution Reviewing Pre-Publication guidance Guide to press release production Printing and distribution guidance Publication

27. Publication review approval Publication Publication review with users Completed publication Opportunities for improvement guidance Review

28. Responsibilities Press Release Chief Executive EDG (Directors) Head of Profession Programme Head Programme Manager Section Head Quality Programme Manager Brief Design and Dev Briefing and Press Rel. Review Mandate Approval Brief Design and Dev Plan Create the Team Design and Dev Process/Pub Rev Design and Dev Data Preparation Analysis Final Draft approval Briefing Press release Prepublication Printing Process/Pub Review

29. Records…Template

30. Links to incidents • The Planning stage includes review of lessons learned across the NHS IC • The Production stage incorporates lessons (eg extra checks) from incidents • The Review stage includes drawing out lessons learned from incidents during production… …and feeds back into planning

31. Improvement cycle • Incidents lead to lessons • Lessons lead to Alerts Improved processes • Publication process holds improved processes and ensures they are implemented • Improved processes lead to • Fewer incidents

32. Example • Breach of the Code of Practice – pre-release access list issued late • Root cause: excessive willingness to accommodate late changes • Lesson: set cut off time and freeze • Implemented and promulgated through process • No further incidents

33. Present position • Better handling • Reduced panic • Involvement of Directors • Engagement of external stakeholders • Better feedback • High level of reporting? • Few unreported incidents coming to light • Salutary examples of complications from not reporting • Evidence of lessons learned • But…

34. … • Improvement still needed on root causes • Some good but some bad practice • Learning needs to be promulgated across the organisation

35. Reflections • Organisational change is hard • It takes time • It is necessary to • Make it easy for people to do the right thing • Avoid blame • but • Keep up the pressure • Be open: a mistake made feels bad; a mistake learned from feels good

36. Next steps • Stronger management emphasis on drawing out root causes and lessons - KPIs • Developing experts to help with this • Continuing support – learning fora • More regular ‘alerts’ • Benchmarking

37. “A man’s errors are his portals of discovery” (James Joyce) “This is also true for organisations” (Andy Sutherland)