1 / 30

Hospital quality indicators

Hospital quality indicators. Johan de Koning, MPH, PhD Senior researcher Department of Social Medicine Academic Medical Centre University of Amsterdam. Outline presentation. The Slovak Hospital Indicator Set Quality check….! Risks and Challenges in indicator use

conway
Download Presentation

Hospital quality indicators

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Hospital quality indicators Johan de Koning, MPH, PhD Senior researcher Department of Social Medicine Academic Medical Centre University of Amsterdam

  2. Outline presentation • The Slovak Hospital Indicator Set • Quality check….! • Risks and Challenges in indicator use • Presenting and Interpreting performance data • Q & A

  3. Purpose and Users • Internal quality improvement - for management information to monitor, evaluate or improve hospital functioning (long or short term) • External accountability and transparency - for accountability requested by stakeholders: - National or regional government - Health Insurances companies - Health care surveillance authority - Public?

  4. Criteria for selection of indicators • The overall importance of the aspects of quality being measured - Expected burden of disease - Effectiveness of the intervention • The scientific soundness of the measures • The feasibility of collecting data on the indicators • Interest for hospitals and external stakeholders (government, HICs, HCSA).

  5. Partly based on PATH indicators (pre-tested) Workshops, hospital pilot's, other meetings Visits of project staff to the Netherlands Other indicator sets (e.g MOH, international sets) 29 indicators- Clinical effectiveness and safety - Efficiency - Staff orientation and staff safety - Patient centeredness The Slovak Hospital Indicator Set

  6. Indicators (hospitals / day surgery centres) AMI mortality rate Primary Caesarean section rate Cataract day surgery rate Inguinal hernia day surgery rate Readmission for AMI Readmission for pneumonia Cancelled surgery Complaint resolution Infection due to medical care Wound infections Transfusion reactions Inpatient hip fracture Unanticipated admission for cataract day surgery Unanticipated admission for inguinal hernia day surgery Acute Myocardial Infarction Indicators: - Aspirin on Arrival - Aspirin prescribed on discharge - ACEI or ARB for LVSD - Adult smoking cessation advice - Beta Blockers prescribed on discharge - Beta Blockers on arrival - Mean time to thrombolysis Heart Failure Indicators: - Discharge instructions - LVF assessment - ACEI or ARB for LVSD - Adult smoking cessation advice Preventing pneumonia in ventilated patients Hip fracture mortality rate Acute stroke mortality rate Total hip replacement mortality rate

  7. Indicator specifications AMI mortality rate Indicator: ____________ Indicator ID number: ____________ Indicator Name: ____________ Description: ____________ Rationale: ____________ Type of measure: ____________ Improvement: ____________ Numerator / Denominator: ____________ Data elements: ____________ Risk adjustment: ____________ Source of data: ____________ Data reported as: ____________ Upper and lower triggers: ____________

  8. Appraisal of indicators through Research and Evaluation (AIRE-instrument)De Koning JS, Klazinga NS, 2005

  9. Appraisal of indicators through Research and Evaluation (AIRE-instrument)De Koning JS, Klazinga NS, 2005 Indicators evaluated against - 20 quality items: Statements - 4 quality domains: 1. Purpose, relevance, focus (5) 2. Involvement of stakeholders (3) 3. Evidence-based (3) 4. Rigour of development, clarity and applicability (9) - 4-point Likert Scale: (strongly agree – strongly disagree)

  10.  Are the quality indicator described in unambiguous terms and comprehensible language?   Are all the terms used in the description clearly described?  Are the reasons for applying the indicators explicitly stated?  Are the target groups (e.g. patient groups) of the quality indicators identified?  Is (are) the patient(s) receiving the care to be measured clearly defined (e.g. as to sex, age, stage of the disease)?  Are the factors influencing the indicator value clearly described?  Are corrective adjustment procedures described that may help to improve inter-performer comparability?+/-  Are the performance thresholds for the indicators explicitly stated? +/-  Are the sources used to collect the data for the indicators and influencing factors clearly described? AIRE-instrument: quality check…!

  11. Field-testing • Give answers to questions about “feasibility”, “usefulness” and “interpretability” of individual indicators. • Reliability / Quality of reporting: consistent identification and reporting of data..? • Whether changes of indicators are needed.

  12. Way forward: risks and challenges…!

  13. Important lesson learned:  The more direct and serious the consequences of high or low scores, the more manipulation and perverse reactions…! i.e. systems ‘less’ punitive, ‘less’ manipulation and perverse reactions… Induced by high pressure to perform (financial incentives / punishment). Unintended effects of indictors (1)

  14. Unintended effects of indictors (2) - Tunnel vision: strong focus on a few indicators, while other aspects get less attention (risk: measured performance may improve, overall quality may fall). - Focus on easily measurable indicators (e.g. waiting lists for non-acute surgery) while more important indicators of unmet need being ignored. - Creating a “short term culture” (focus on this year good performance replaces more strategic thinking on fundamental long term improvements).

  15. Unintended effects of indictors (2) Examples: - Waiting times: GPs reduce referral rate / specialists increase their thresholds for treatment. - Hospital reluctance to deal with high-risk patients or procedures: damage of performance profiles. - Purposefully underachieve in one year to show steady improvement over time (organisations at a high level to start with may look worse).

  16. Case report “I was just at a theatre meeting today [where] we were discussing that cancellations on day of operation is a bad mark, a cross in the box there. So what do we do? We cancel them the night before so they are not recorded as cancellations on the same day.”

  17. Methodological issues (variation & noise)

  18. Analyzing en interpreting performance data…!

  19. League tables Technique for displaying comparative rankings of performance indicators scores for similar providers (e.g. hospitals) Purpose: • Identify (few) providers with scores greater or lower than expected 2. Show range of variation between providers

  20. Mean score League tables Ranking Ranking Adab, P. et al. BMJ 2002;324:95-98

  21. Advantages and drawbacks Drawbacks: • Depend on quality of data • Stimulate “creative reporting” • Enhance unintended adverse consequences: “gaming” • Stimulates a culture of naming and shaming Advantage: • Stimulate competition among hospitals • Encourage managers and health care insurers to prioritize quality instead of costs only • Identifies poor performing clinicians or hospitals, start inquiry…!

  22. Control charts Requirement indicator system: Sort out “signals” from background “noise”…! Special causes that Variation explained by need further investigation common causes

  23. Common “noise” / special cause “signal”    Surgeon specific mortality rates after colorectal cancer surgery HR= adjusted hazard ratios

  24. Control charts Adab, P. et al. BMJ 2002;324:95-98

  25. Mean score Control charts Ranking

  26. 30 day mortality following coronary artery bypass grafts in 25 English hospitals Alarming! Warning! (A) A ‘‘forest’’ plot showing 95% confidence intervals compared with the ‘‘target’’ overall average rate. (B) A ‘‘funnel’’ plot of observed rate against number of operations (3SD from the target).

  27. Emergency (within 30 days) readmission rates following discharge (A) A ‘‘forest’’ plot showing 95% confidence intervals compared with the ‘‘target’’ overall average rate. (B) A ‘‘funnel’’ plot of observed rate against number of operations (3SD from the target).

  28. Emergency (within 30 days) readmission rates following discharge Action: Do not use the indicator…? Improve risk stratification…? Analysis by clustering…? (B) A ‘‘funnel’’ plot of observed rate against number of operations (3SD from the target).

  29. Managed to develop a good quality “first version” hospital indictor set. Further field testing required. Next to external accountability / transparency indicators should stimulate ‘internal’ quality improvement. Development of plan for data-analysis and presentation: no simple calculation of numbers or percentages. Last but not least: treat indicators as indicators…! Additional quality information essential to understand performance (e.g. quality reports). Conclusion

More Related