Where next for improving and evaluating patient safety?

1. Where next for improving and evaluating patient safety? Dr. Dale Webb Director of Evaluation & Strategy The Health Foundation

2. Future challenges for patient safety

3. Loss of prominence in the NHS? • A ‘loss of momentum’ and ‘vacuum’ following the demise of NPSA, Patient Safety First and changes at the NHS Institute • “There is a lot of fog out there … I don’t hear clear messages … there’s no sensible narrative for people in the service” (Medical champion for patient safety at national level) • “I’m concerned that patient safety will be lost off the agenda in the NHS reorganisation. If it doesn’t drop off consciously it will be compromised in the way that people cut costs” (Medical champion for safety) • “We are lucky in Wales: our annual operating framework keeps patient safety on the agenda” (Medical Director, Acute Trust) • The Scottish Patient Safety Programme had “helped on a number of levels – both at clinical and board level” (Medical Director, Acute Trust)

4. Other challenges for patient safety Strengthen the evidence base and build the link between safety and productivity: “What is the common currency for everyone at the moment? It’s finance. That safe care means cheap care. We need evidence to support the ‘invest to save’ argument” (Medical Director, Acute Trust) “Let’s not over-exaggerate the case for quality improvement” (Medical champion for safety) Strengthen organisational capacity for continuous quality improvement Spreading and embedding safety improvements Exploring the role of patients and the public in creating safer healthcare

5. Implications for the Health Foundation’s safety work

6. Sustaining a safety focus in hard times Safer Patients Network, launched in June 2009, aimed to be a self-sustaining, member-driven group of organisations. No longer realistic to continue with this ambition in the current financial climate We will commit to support a broader community of safety-minded organisations to maintain and develop their focus on safety during the challenging times ahead • Establish a managed virtual network • Open to membership across the UK • A go to place for resources, access to expertise and connections to peers • Provide the interface between regional & national initiatives, promoting shared learning • It will clearly signpost specific areas of interest (communities of practice) • Develop a wide range of active support to leaders, managers and frontline staff • Focus on spreading and developing innovative approaches

7. Sustaining a safety focus in hard times Rolling programme to introduce new ideas and maintain momentum • Interest-group based webinars and podcasts • ‘Expert in residence’ programme • ‘Local team in spotlight’ programme • Annual learning event • Community of practice events • Skills marketplace/’time bank’ • Explore involvement of patients and the public

8. Safer Clinical Systems A clinical system that delivers value to the patient, is demonstrably free from unacceptable levels of risk and has the resilience to withstand normal and unexpected variations and fluctuations

9. Our research shows poor reliability • Failures in reliability pose real risk to patient safety 15% of outpatient appointments affected by missing clinical information • Important clinical systems and processes are unreliable Four clinical systems measured had failure rate of 13%-19% • Wide variations in reliability between organisations • Unreliability is the result of common factors Lack of feedback mechanisms and poor communication. • It is possible to create highly reliable systems The Health Foundation May 2010

10. Building Safer Clinical Systems • Define and describe the system & its context • Assessing and diagnosing hazard & risk • Testing & implementing solutions

11. Key Features Award holders are taken through a systematic approach which will involve: • A tailored learning and development programme • Expert help • On-site support facilitated by a named person • Peer-review • Opportunity to review progress at key intervals • Central learning events

12. Phase 2 Systems approach allows teams to identify and address those parts of a patient pathway that exposes patient to potential harm. It explores factors outside the clinical pathway that affect the care provided, including organisational context • In Phase 2 we will be working within patient pathways, focusing on these supporting processes and systems: • Safe, reliable prescribing in patient pathways (e.g. prescribing by staff throughout the pathway, together with upstream processes such as information transfer and downstream administration) • Safe, reliable clinical handovers in patient pathways (e.g. transfer of clinical information, tasks, responsibility and authority)

13. Other areas of work The role of corporate services in closing the gap and improving quality and safety Scoping work in primary care Role of patients

14. 1) Do these challenges resonate for you?2) What do you think about our network plans?3) What’s missing?

15. Challenges in evaluating efforts to improve safety and quality

16. Tensions between evaluation & quality improvement • The ‘treatment’ is social change • The theoretical and empirical foundations of QI • Data for improvement v. judgement • Flexible v. null hypothesis • Real-time v. static data

17. The RCT faultline • A faultline that appears elsewhere: • Critique of positivism in the 1970s and 1980s • Health promotion’s response to evidence-based medicine in the early 1990s

18. New evaluation methods or new evaluative mindsets?

19. Evaluation is a fractured discipline • Schools include: • Experimental • Hermeneutic • Fourth generation • Realistic • Complexity • Theory-based • Utilisation driven • Stakeholder-based • Values based • … and a dozen others!

20. Pretenders to the throne • Realistic Evaluation • Theory of change • Complexity Theory • Process Normalisation Theory

21. Incommensurable paradigms? • Dominant tendency in the literature for plurality/non-integration • Small number argue for a more cumulative approach to scholarship • Boundary conditions of individual theories and methods • A metatheory of evaluation?

22. New methods are not what we need mostWe do need more synoptic, integrated, approachesTo do this, we need a different mindset – a different perspective

24. Data for judgement and improvement • Measurement for improvement: • provides visually compelling real-time data and, critically, variation around the mean time • Used well (and carefully annotated to note when changes to care take place), this can help to understand and explain the transformative characteristics of adaptive systems • Measurement for judgement: • The counterfactual question ‘what would have happened anyway if the programme hadn’t existed?’ is important if we are determine the ‘additive effect’ of a programme above and beyond over things going on in the system of care, but only when conditions permit

25. Build intervention theory • Funders, QI technical providers and evaluators working together • Dosage, time frame, tempo, locus of activity • Stretch goals v. evaluative goals • Benefits: • Makes clear whether we are truly in innovation, demonstration or scale up mode • Ensure we have a realistic expectation at the outset of likely impact relative to the dose, and that measurements are being taken at the specific places where we expect to see an effect • Consensus on the levels of evidence and likely impact of the changes that are being proposed

27. Recover ontological depth • We need to understand how institutional and other contexts frame decisions and actions • Real time knowledge capture about the implementation process • Evaluation ‘getting under the skin’ of the intervention • Causal description: a description of the consequence attributable deliberately to varying a treatment & • Causal explanation: the mechanisms through which and the conditions under which that causal relationship holds (Shadish et al., 2002: 9)

28. Opportunities for collaboration? • ‘Think pieces’ to identify points of connection • Develop new thinking about the design and evaluation of QI interventions • Journal supplement on the science of evaluating improvement

29. Let’s not throw the baby out …

30. 1) Do you agree with me?!2) How mature are the approaches used to evaluating safety interventions?3) How should the evaluation community be positioning itself in relation to safety & quality?