Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk

1. Using Registry data to drive improvement - what makes clinicians take note of statistics UnderstandingVariation Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk

2. Acknowledgements Mohammed A. Mohammed, Department of Public Health & Epidemiology, University of Birmingham

3. Quality & Variation • Conventional tools • Standard setting • Clinical Audit • Ranking or League tables • Hypothesis testing • Effect • Pass or fail • Action on those thatfail • Another way ..

4. PROCESS OF WRITING Shewhart’s Concepts • Letter a SPECIAL CAUSE ACTION: FIND & ELIMINATE COMMON CAUSE ACTION: PROCESS

6. Basis of Control Limits • Tchebycheff’s theorem X mean +/- t SD P > 1 - 1/t2 • t=3 • Economic • common cause vs special cause

8. Special cause variation - action • Data • Accuracy • Definition of errors • Raw materials • Difficulty of tasks • Equipment, facilities, staffing • Typewriters, workload, lighting • Processes, procedures • How is the work organised? • People • Skill levels and techniques

10. Application to Health Care • Case studies

11. Surgeon Variability • Surgeon Survived Died % • A 82 16 16 • B 58 8 12 • C 49 9 16 • D 45 7 13 • E 37 15 29 • F 41 5 11 • G 35 3 8 • H 26 11 30 • I 31 5 14 • J 27 7 21 • K 28 4 13 • L 19 2 10 • M 18 3 14 McArdle & Hole BMJ 1991;302:1501-5

12. Surgeon Variability McArdle & Hole BMJ 1991;302:1501-5

13. Fractured Hips • 90 day mortality (N=580; 104 deaths 18%) • Hospital Mortality • 1 19/79 24% • 2 5/24 21% • 3 16/79 20% • 4 19/80 24% • 5 12/80 15% • 6 4/81 5% • 7 14/79 18% • 8 15/63 19% Todd et al BMJ 1995;301:904-8

14. Fractured Hips Todd et al BMJ 1995;301:904-8

15. Western Electric Company Rules • Additional rules for detecting special causes • 1 data point >3 sigma from mean • 2 out of 3 data points >2 sigma from mean • 4 out of 5 data points >1 sigma from mean • 9 successive data points on one side of mean • Trend of 6 successive data points

16. Special cause variation: nine successive data points below the mean

17. Renal Registry Data • Average haemoglobin per quarter

18. Trend of 6 data points Run Chart – sequential data points + mean

19. Interpretation • Rising trend in mean Hb nationally • Difficult to interpret changing Hb in a single centre except in relation to rising trend nationally

21. Consistent with national average 9 data points above mean i.e. own long term average Below national average 10 data points below mean

23. 9 data points above mean Average determined from first 8 data points Run Chart – difference between this centre + UK Average

24. Control Chart – Run Chart + 3 sigma limits

25. Consistent with two stable processes: before Sept 04& after Sept 04

26. Special cause variation - action 1. Data (including definitions) 2. Raw materials (case-mix) 3. Equipment, facilities, staffing 4. Processes, procedures 5. People

27. Monitoring Many Centres

28. Walter A Shewhart 1931

29. “The central problem in management and leadership …is failure to understand the information in variation” William E Deming 1986 Out of the Crisis MIT pg 309

30. Summary • Shewhart’s concepts • Understand variation • Simple & powerful • Guide action • Wide application • Continual improvement • Clinical governance • Other implications ..

31. Implications • Prediction • Limits of common cause variation • Statistical control • Action • Common cause variation • League tables, ranking, hypothesis testing all misleading • Improve process/system as a whole • Special cause variation • Investigate and eliminate (or learn lessons) • Data order important

33. How It Works In Industry • Balanced set of measures

34. Balanced Set Of Measures Customer Aim Internal External Financial Four or five measures for each box

35. Balanced Set Of Measures Patient Experience Strategic Effectiveness Clinical effectiveness Aim Financial / Resources

36. Special Causes • Identify special causes in each domain • Collate & prioritise for action • Low hanging fruit first