Using routine data to measure and promote safety and quality in hospitals

1. Using routine data to measure and promote safety and quality in hospitals Stephen Duckett Adjunct Professor, ACERH/UQ (Joint work with Dr Terri Jackson Associate Professor, ACERH/UQ)

2. Overview • What do we know about patient safety and patient harms in Australia? • What do we count as patient harm? • How can we measure and monitor it? • How much does it cost? • How can use of routine data help us understand the economics of improving patient safety? • Payment policy and patient safety

4. What do we know about Australia? • Landmark 1995 QAHC study: • Careful (expensive) methods • Incidence: 16.6 % of multi-day inpatient stays • Revised down to 10% (comparative US study) • Costs to system: $900 mil pa • The newspapers: • Bundaberg • The NSW ‘Cam affair’ (Cambelltown & Camden) • WA’s Royal Commission • Australian Commission on Safety & Quality in Health Care (Building on Aust Council for S&QHC (1999-2005)

5. What and how you ‘count’ depends on why you want to count it WHY count?

6. We need to monitor rates routinely: using hospital discharge data • ICD-10-AM has specific codes: • T 80.0-88.9 ‘Complications of surgical & medical care’ • ‘End of chapter’ codes • Y 40-84.9 ‘External cause--complication of surgical or medical care’ • Australia has world-class quality hospital data • Victorian (and now Oz) ‘condition-onset flag’ (C-prefix) denoting: • Condition required treatment or extended LOS • Condition was not present on admission

7. Improving patient safety requires hospital-acquired incidentcases • Problem of distinguishing ‘comorbidities’ from ‘adverse events’ • Incident vs prevalent cases • Not GP, nursing home, admission from another hospital • Overall rate in Victoria (2000/01): 8.25% (15.9% for multi-day stays) • Only two-thirds (5.61%) were ‘Incident’ cases • The ‘C-prefix’ adds valuable information • 41% of all hospital-acquired diagnoses were missed by ICD alone, eg, UTI, atrial fibrillation, pneumonia • Result: new national ‘Condition Onset’ flag on all Dx Jackson TJ, Duckett SJ, Shepheard J, & Baxter KG. ‘Measurement of adverse events using ‘incidence flagged’ diagnosis codes’ Journal of Health Services Research and Policy, 11 (1):21-25; 2006.

8. Strengths of these data • Strengths • Timely & cheap data collection • Standardised definitions and coding rules • Includes both dramatic and mundane • Independent reporting (not bedside) • Current payment incentives for thorough coding

9. Weaknesses of these data • May miss same-day patient harms • Less coding investment • Symptoms appearing post-discharge • Prefix not currently audited • Coders may miss clues clinicians could spot • No judgement about ‘preventability’ • Notorious inter-rater reliability problems in this judgement • May be a ‘strength’ as today’s routine complication becomes tomorrow’s ‘preventable’ adverse event…

10. Risk factors and outcomes for hospital-acquired diagnoses (HA Dx) • Age is an important predictor of HA Dx • 29% of patients over 85 vs 9% in the 5-9 age group • Risk of HA Dx varies considerably by medical specialty • Gastroenterology 86.0% vs ENT surgery 10.5% • ALOS strongly associated with HA Dx • 11.2 days with HA Dx vs 6.9 days without • ALOS both a risk factor and an outcome • In-hospital mortality associated (not caused) • 4.09% with HA Dx vs 1.75% without Duckett SJ, Jackson TJ, Hong Son Nghiem (2008).Risk factors and outcomes for incident complications in multi-dayadmissions to Victorian hospitals (manuscript).

11. Voluntary Sentinel Event reporting compared with routinely-coded hospital-acquired diagnoses* • Study Objective: To compare two sources of data on hospital sentinel events to evaluate strengths and weaknesses of both • Problems with voluntary reporting: • Stigma and blame attached to involvement in adverse outcomes • Resulting reluctance to report • Safety-aware hospitals appear to have ‘worse’ outcomes • Focus on single events rather than rates reinforces individual rather than system causation Jackson T, Moje C, Shepheard J, and McMillan A. Poster presented to the 2007 Australian Conference on Safety and Quality in Healthcare, August, 2007.

12. Using routine data to identify sentinel events

13. Public Reporting • Risk adjustment important for public comparisons • Reputation risk • Risk of patients being denied treatment • Fairness to providers • Public officials expected to intervene when quality declines • What do you report? • What providers have ‘done’ or • What they are now doing to improve?

14. Any measurement approach must balance the risks

15. Problems with public reporting using routine clinical data • Timeliness – data provided on an annual basis, about 15 month delay in publication • Mistrust of ‘administrative’ data • Not ‘risk adjusted’ • Insufficient clinical detail • Aggregated Data – unable to detect runs • Nothing about cause of differences in rates

16. Queensland Health’s Variable Life Adjusted Display (VLAD) Plot of the cumulative difference between expected and actual outcomes over a period of time Coory M, Duckett SJ, Sketcher-Baker K. Using control charts to monitor quality of hospital care with administrative data. International Journal for Quality in Health Care 20(1): 31-39. .

18. Simplified Variable Life Adjusted Display (VLAD) Patient survives - VLAD increases by the probability of the patient dying

19. Simplified Variable Life Adjusted Display (VLAD) / 2 Patient dies: VLAD decreases by the probability of the patient surviving

20. Simplified Variable Life Adjusted Display (VLAD) / 3 2nd patient survives: VLAD increases by the probability of the 2nd patient dying

21. Simplified Variable Life Adjusted Display (VLAD) / 4 2nd patient dies: VLAD decreases by the probability of the 2nd patient surviving

22. Variable Life Adjusted Display (VLAD) Plot of the cumulative difference between expected and actual outcomes over a period of time

24. Lessons: Sometimes statisticians’ methods are somewhat dubious Be wary of replicating them, even if you think you understand them

25. Specific Clinical Indicators • Developed in consultation with clinical expert groups • Indicators reviewed annually and refined on basis of feedback • Data from the Queensland Hospitals Admitted Patients Data Collection and the Perinatal Data Collection • Sex, age and comorbidities used to risk-adjust for illness severity and co-existing conditions

26. Clinical Indicatorsbeing developed Mortality • Acute Myocardial Infarction • Heart Failure • Stroke • Pneumonia • Fractured Neck of Femur Readmission • Acute Myocardial Infarction • Heart Failure • Knee Replacement • Hip Replacement • Depression • Schizophrenia • Paediatric Tonsillectomy and Adenoidectomy Complications of Surgery • Laparoscopic Cholecystectomy • Vaginal Hysterectomy • Abdominal Hysterectomy • Fractured Neck of Femur • Colorectal Carcinoma • Knee Replacement • Hip Replacement • Prostatectomy O&G • Caesarean Section Rate • Std Primiparae Induction of Labour Rate • Std Primiparae Perineal Tears (3rd or 4th Degree) • Hysterectomy – on women < 35 yo Future • Falls • Pressure ulcers • Overall Surgical Complications • AHRQ Patient Safety Indicators

27. Web-based documentation of VLADs

28. How bad is a ‘bad run’?

29. Variable Life Adjusted Displaywith control limits

30. Flagging criteria • 30% higher than expected mortality– automated message emailed to the district manager and clinical lead, encouraging internal investigation and report to Area Clinical Governance Unit (50% for non-mortality flags) • 50% higher than expected – flagged to Area Clinical Governance Units to ensure they are involved in further investigation (75% non-mortality) • 75% higher than expected - identified to State Patient Safety and Quality board and in public reporting as being statistically significantly different from the average (100% non-mortality) Duckett SJ, Coory M, Sketcher-Baker K. Identifying variations in quality of care in Queensland hospitals. Medical Journal of Australia 187 (10): 571-575.

31. VLAD charts allow tracking to specific case files

32. Detailed data underpinning the VLAD system

33. QH Pyramid Model of Investigation

34. Key strength of routine data: links to patient costs • Victorian and Queensland CWS data • n≈ 1.6 mil records pa • n≈ 100 large public hospitals • Validated for use in hospital funding • What can cost data tell us? • Economic burden of adverse events • Economic priorities for prevention efforts • Business case for: • prevention efforts • medical research • Cost-effectiveness analyses for: • prevention programs • new patient safety devices and procedures

35. Incident cases represent a large economic burden to the health care system • Patients with at least one C-prefixed adverse event: • Stay nearly 10 days longer than other patients • Cost $ 6826 more per episode (controlling for DRG, age and co-morbidity) • Extrapolated to entire hospital system: • At least $511.5 mil additional cost in Victoria (2003/04) • Adds 18.6% to hospital expenditures • Around $2 bil pa nationally • Even if only 40% preventable: $200 mil pa could be saved in Vic; $800 mil nationally Ehsani JE, Jackson TJ and Duckett SJ. ‘The incidence and cost of adverse events in Victorian hospitals, 2003-04’ Medical Journal of Australia, 184;11; 5 June 2006

36. Re-admissions* add to this cost burden • Current work on Victorian separations with a PDx in the T80-88.9 range of ICD-10-AM: • 16,734 admissions with a PDx of a ‘complication of surgical or medical care’ • $70.6 mil pa additional public expenditure on these cases in Victoria** *Includes admissions for HA Dx from primary care and nursing homes **McNair P, Jackson TJ, Borovnicar D. ‘Costs of Victorian admissions for treatment of adverse-event principal diagnoses, 2005/06’ submitted for publication, August 2008.

37. Queensland Health P4P approach • Incentives at margin ($8M out of $7B budget, 0.1% of total, larger % of specialty) • Range of specialties, data collection approaches, foci • Specialties: • Mental health • Stroke • Emergency department care • COPD • Medication Duckett, S.J et al (2008) ‘P4P in Australia: Queensland’s New Clinical Practice Improvement Payment’ Journal of Health Services Research and Policy 13:174-177

38. Mental healthindicators

39. Acute stroke indicators

40. Emergency indicator

41. Medication indicator

42. COPD indicator

43. Queensland Health approach • Moral suasion will be used to encourage most of (80%) CPIP payment to flow to clinical unit where indicator activity is being undertaken. • CPIP only to be used for non recurrent expenditure items • CPIP to be used to enhance not expand service (eg increase number of operations performed)

44. CPIP - summary • Started in January 2008 • Experiments with very different approaches initially • Mostly Pay for process adherence/reporting rather than outcomes • Can be rolled out to other indicators, add/drop indicators • Stephen_duckett@health.qld.gov.au