‘Cybernetics’ and Patient Safety Research: A Retort from NES

1. ‘Cybernetics’ and Patient Safety Research: A Retort from NES Paul Bowie Associate Adviser in Postgraduate GP Education Diane Kelly Assistant Director of Postgraduate GP Education West Region Glasgow, UK paul.bowie@nes.scot.nhs.uk diane.kelly@nes.scot.nhs.uk

2. Workshop Purpose • To describe two case studies of patient safety research in primary care and promote debate around differing perspectives on research priorities, approaches and usefulness • Overview of NES Research (see handouts) • Two Case Studies • Open Discussion • Prof. Huw Davies

3. “The study of control and communication in the animal and the machine” (Norbert Weiner, 1948) “The science of effective organisation” (Stafford Beer, 1974) “The interdisciplinary study of the structure of regulatory systems… It includes the study of feedback, black boxes and derived concepts such as communication and control in living organisms. Machines and organisations including self-organisations” (Wikipedia) Some Random Definitions of ‘Cybernetics’?

4. What Drives NES Patient Safety Research Priorities? • Pre-defined policy • Solutions-focused for the frontline • “Usefulness” • “Pragmatism” • “Critic or contributor” (Vincent, 2009) • “…setting priorities to focus on the most critical aspects of patient safety is essential to yield the maximum possible benefit especially when research funds are limited.” (WHO, 2009)

5. CASE STUDY ONE Screening Electronic Patient Records to Identify Avoidable Harm: A Trigger Tool Process for Primary Care

6. What is a Trigger Tool? • A trigger tool is a checklist of clinical ‘triggers’ which a reviewer looks to identify when screening patient records. • ‘‘Triggers’’ are easily identifiable ‘flags, occurrences or prompts’ in records that alert reviewers to potential adverse events - previously undetected. • - E.g. an international normalised ratio (INR) of 5.0 would be a ‘‘trigger’’ for the reviewer to examine the record in greater detail for evidence of the patient suffering some type of related haemorrhage • Most efficient method of detecting and ‘measuring’ error and harm? • - incident reporting, significant event analysis, complaints & litigation • Evaluation of UK Safer Patients Initiative (Benning et al., 2011) – Doubt and debate • ‘Pseudo-innovation’ (Walshe, 2009)

7. Background • Policy shift - SPSP migrating to primary care • Primary care: • - limited knowledge of harm/experience of safety initiatives • How can we ‘measure’ and learn more about harm? • How can we engage primary care workforce in more explicit and meaningful efforts to improve safety? • Interest in IHI ‘trigger tool’ and transferability to primary care • NES support as a research priority (Patient Safety Group and R&D)

8. Transferability Research • Literature review • Identification of candidate triggers • Consensus and validation methods: agreement on 10 core Triggers • e.g. group interviews, modified Delphi, content validity index exercise • Triggers e.g. 10 consultations in past 12 months; any home visit; abnormal blood results; repeat medication added or cancelled • Pilot test in five GP practices (500 random EPRs 5x100) • - positive predictive value, sensitivity, specificity, inter-rater reliability • 9.5% harm rate detected, 57% judged preventable, ‘severe’ cases originate in hospital care (de Wet & Bowie, 2009) • Conclusion: It ‘works’, but: • - Concerns about reliability as a ‘measurement tool’ • - Feasibility in routine clinical practice • - Alternative application as a research method?

9. What Happened NextSafety Improvement in Primary Care (SIPC) • 80 general practices participating in collaborative working – complex social intervention • Trigger Tool is a core intervention – ‘to measure harm events’ in sub-populations under study • Realistic Evaluation (Pawson & Tilley, 1997): what works, why and in what contexts? • Feedback loops (Lyn Halley & Carl de Wet): observational work, focus groups, documentation reviews, and semi-structured interviews

10. SIPC – Interim Findings • Further refinement of process (de Wet & Bowie, 2011) • Most positively received element of intervention bundle • Foreseen and unforeseen consequences: • - ‘measurement difficult’ (feasibility & reliability issues) - ‘real-time’ improvements - uncovering previously unknown harm risks - identification of patient safety-related learning needs - positively received, no resistance as yet - widespread implementation potential • Key purpose evolving: a mechanism to Screen rather than Measure?

11. Potential Policy & Cultural Impact? • GP Appraisal • GP Specialist Training • Out-of-hours service • SPSP plan for primary care • NES educational support • ‘Game Changer’: • - Awareness and acknowledgement of the scale and impact of the problem • - Proactive engagement in learning about harm avoidance • - “…enable the primary care team to refocus and prioritise learning and improvement efforts on identifying harm and developing preventative measures to mitigate future risks to patients’

12. Challenges and Further Research • Proxy measure of harm: desirable? useful? reliability? feasible? etc. • How best to train the clinical workforce? • How best to implement further? • How do clinicians provide evidence of engagement? • Can we peer assess this activity? • Validate a core list of ‘never events’? • Even more ‘Cybernetics’? • Clashing or complimentary research priorities and approaches: NES and University sector?

13. Collective Learning, Change and Improvement in Healthcare Teams St Andrews June 2011 Diane Kelly

14. The Story so far....

15. Introduction An idea learning organisation concept Theory into practice? Collaboration St Andrews +NES Learning practice inventory All members of GP team

16. Rushmer R, Kelly D. R.,Lough M, Wilkinson J.E, Davies H.T.O. Introducing the Learning Practice – I. The Characteristics of Learning Organisations in Primary Care. Journal of Evaluation in Clinical Practice (2005) 10:3 375-386 Rushmer R, Kelly D. R.,Lough M, Wilkinson J.E, Davies H.T.O. Introducing the Learning Practice – II. Becoming a Learning Practice Journal of Evaluation in Clinical Practice (2005) 10:3 387-398 Rushmer R, Kelly D. R.,Lough M, Wilkinson J.E, Davies H.T.O. Introducing the Learning Practice – III. Leadership, Empowerment, Protected Time and Reflective Practice as Core Contextual Conditions. Journal of Evaluation in Clinical Practice (2005) 10:3 399-405 Rushmer R K, Kelly D, Lough M, Wilkinson J, Greig G & Davies H T O. The Learning Practice Inventory: diagnosing and developing Learning Practices in the UK Journal of Evaluation in Clinical Practice (2007) Vol 13 No 2: 206-211 Kelly D, Lough M, Rushmer R, Wilkinson J, Greig G & Davies H T O. Delivering Feedback on Learning Organisation Characteristics – Using a Learning Practice Inventory Journal of Evaluation in Clinical Practice (2007) 13(5):734-40 Kelly D.R., Lough J.M., Rushmer R., Greig G., Crossley J., Davies H.T.O. Diagnosing a learning practice: the validity and reliability of a Learning Practice Inventory (LPI) Quality and Safety in Health Care (2011);20:209-215

17. Questions for NES Can/How to support teams Many assumptions

18. Chapter 1 a - Primary research Into collective learning, change and improvement in primary care teams (GP, pharmacy and dental practice teams) Bunnis S., Kelly D.R. The unknown becomes the known:collective learning and change in primary care teams. Medical Education (2008) 42 (12) 1185-1194 Bunnis S., Kelly D.R. Research paradigms in medical education research Medical Education (2010) 44(4):358-66

19. Key findings Informal collective learning is a responsive coping mechanism generated by patient need How Experiential Evolving Implicit Relational Natural tendency towards QI

20. Chapter 1 b – Designing a facilitated intervention To promote collective learning and improvement In GP teams – whole team approach Use of LPI was starting point 6 Facilitated sessions over 1 year Action research Evaluation

21. Bunnis S., Gray F., Kelly D. Collective learning, change and improvement in health care: trialling a facilitated learning initiative with general practice teams Journal Evaluation in Clinical Practice (2011) Bunnis S, Gray F. Kelly D. Collective learning, change and improvement in healthcare: piloting a facilitated learning initiative with general practice teams. In PREPARATION

22. Findings Intervention introduces tools, processes and shows how to use them to enhance shared learning and create more effective collective change Teams designed and introduced ways to enhance their own effectiveness Whole team engagement maximised effectiveness Engagement enhanced by practice generation of data via the Learning practice inventory.

23. LPP used with Scottish CHP (Community health partnership) Dental team in England Hospital Nurses in USA Plans for use with practices across health authority area in North of England Chapter 1 c- Learning practice programme (LPI + facilitation)

24. Chapter 2 a - Primary research into collective learning, change and improvement in secondary care teams

25. Findings Fluidity of membership ‘Team’ a contested notion Potential for QI in secondary care inhibited: Professional boundaries Assumptions re contribution to team effectiveness Untapped expertise and awareness.

26. Chose NOT to assume possible to repeat LPP in secondary care Chose to build on Paul’s research and focus on medication handling Aim- to enhance patient safety through collective learning Chose to undertake Participatory research with a care of the elderly ward Phase 1- observational study Phase 2 - interviews Chapter 2 b – Secondary Care

27. 2 c- Recommendations Create opportunities for ward staff AND management staff to engage in reflective dialogue Through use of ‘burning questions’ to reach deeper sense of their identity as a team Give managers and staff a way to begin to identify, prioritise and respond to patient safety issues

28. Tools have an important role....but - they are not enough - More is needed and the time is NOW - Discuss Epilogue