The National Program for Quality Indicators In Community Healthcare: Methodological Issues

1. The National Program for Quality Indicators In Community Healthcare: Methodological Issues Orly Manor 7th meeting of the Eastern Mediterranean Region of the International Biometric Society (EMR-IBS) Tel-Aviv April 22-25, 2013

2. “Efforts to improve quality require efforts to measure it”(Casalino, 2000:NEJM)

3. Healthcare in Israel National Health Insurance law (1995) • Universal healthcare • Standard basket of medical services • Four health plans (kupot cholim) • “Justice, equity and solidarity…medical services will be offered based on medical considerations, with reasonable quality.”

4. Healthcare in Israel • Health Tax - progressive , paid to the National Insurance Institute (funds are distributed to the health plans according to a capitation formulae) + modest copayment directly to health plan for specific services • Open enrollment, no option to reject applicant, annual option to switch plans • Managed competition between plans (uniform benefits) • Competition is based on quality and nature of services

5. Israel National Program for Quality Indicatorsin Community Healthcare (QICH) • Supervisory bodies were established “to follow and assess the influence of the NHI law on health services in Israel, their quality, effectiveness and cost" • QICH started as a research project (Porath & Rabinovitz, 2002) and later adopted as a national program. • Full cooperation and support by all four health plans

6. Mission To provide the public and policy-makers information on the quality of community healthcare provided in Israel. This information covers various health categories and is intended to promote and improve the standard of healthcare in Israel.

7. Main Product Annual report presenting national results of quality indicators in community healthcare Enables • the evaluation of developments and changes in healthcare over time • the early identification of risk factors in the Israeli population and in sub-populations • the comparison of healthcare quality in Israel with other countries

8. 8 "Not everything that counts can be counted, and not everything that can be counted counts." Einstein

9. Quality Indicators- Methods 9 • Measures of clinical performance (structure, process, outcome) • Based on electronic health records from the four health plans • All indicators are rates • Some indicators are conditional on others • Covariates: age, sex, SEP (proxy)

10. Quality indicators (2013) 10

11. Example: Diabetes 11 • Blood glucose levels of individuals with diabetes are directly related to the development of complications: cardiovascular disease, blindness, kidney failure • Monitoring blood glucose by periodic hemoglobin A1c testing and achieving adequate glycemic control

12. Example: Diabetes 12 Prevalence measure: • Rate of individuals with diabetes mellitus from the entire population (overall and by age and gender) Process measure: • Rate of individuals with diabetes with documented levels of hemoglobin A1c (HbA1c) Outcome (intermediate) measure: • Rate of individuals with controlled levels of HbA1c from patients with diabetes with documented levels of hemoglobin A1c (HbA1c)

13. Methodological Issues • Criteria • Population coverage • Data quality-measurement error • Data sources • Consistency of measures • Reporting

14. Criteria Implementation

15. Population Coverage • Population-based, near-complete coverage (not sample) • Transfers between health plans • Births/deaths • Other populations: e.g., soldiers

16. Data Quality- misclassification error 16 • Estimating the prevalence of a medical condition in the absence of neither a gold standard nor an additional classification. • We wish to estimate - prevalence, sensitivity and specificity , yet df=1. • We can use a Bayesian approach- simultaneous inferences of the prevalence, sensitivity and specificity and positive and negative predictive value (Joseph et al 1995) • Selecting priors- experts’ opinion, understanding sources of data and errors (Greenland 2009)

17. Data Quality- Consistency of Measures 17 Uniform definitions across health plans • Membership • Data collection period • Numerator and denominator

18. Data Sources • Medical records (e.g., documentation of BMI) • Nurse’s records (e.g., documentation of vaccination) • Pharmacy claim records (e.g., medication purchase) • Laboratory results (e.g., HbA1c levels) • Hospital procedure codes (e.g., CABG) • Other (e.g., mammography)

19. Data Quality – Sources of Data and Sources of Error • Medical records (e.g., documentation of BMI) • Nurse’s records (e.g., documentation of vaccination) • Pharmacy claim records (e.g., medication purchase) • Laboratory results (e.g., HbA1c levels) • Hospital procedure codes (e.g., CABG) • Other (e.g., mammography) Automated vs manual data input (variation between and within health plans) Pop-up options vs typing in vaccine name, historical data (variation between health plans) ATC vs YARPA/LARGO (variation between health plans) Standardized values for calibration (variation between laboratories) MOH codes used for billing (too broad) Self reported, billing-based

20. Data Quality – Checks and Audit • Internal (health plans) • Data checks (BI) • Feedback loops/criterion validity • Quasi-external (directorate) • Between and within health plan data checks (outliers) • Comparison with existing national data • External (independent auditor) • Process audit of infrastructure changes • Process audit of indicator implementation

21. Reporting • Transparency + court ruling of the public reporting of indicators by health plan • Case mix: substantial differences between health plans by SEP • Limited available data on SEP Friedberg et al. (2011). Rand Corporation: Methodological Issues in Public Reporting

22. Reporting - Adjusting for Case Mix 22 • Israel is divided into statistical areas. • The Israel Central Bureau of Statistics calculates SEP scores for each statistical area using recent census information • Currently-using GIS each person’s address (unidentified data) is linked to his statistical area and the respective SEP score

23. Reporting -Benchmarking 23 • Setting benchmarks: setting, testing, comparing??

24. Diabetes care

25. Diabetes care Diabetes care includes routine monitoring and proper control of: • Blood glucose levels (93%) • Cholesterol levels (90%) • Blood pressure measurement (92%) • BMI assessment (86%) • Eye examination (65%) • Influenza (55%) and pneumococcal vaccination (77%)

26. Diabetes care 26

27. International comparisons – Diabetes care (2010) *US data from the National Committee for Quality Assurance, HEDIS data set for 2010

28. Diabetes care – Israel and England (QOF) (2009)

29. Thank you Directorate: Orly Manor, Arie Ben-Yehuda, Amir Shmueli, Ora Paltiel, Ronit Calderon and Dena Jaffe Directorate staff: Wiessam Abu Ahmad, Galit Shefer. Clalit Chaim Bitterman Orit Ya’akobson Arnon Cohen Margalit Goldsprecht Tamara Koren Maccabi YairBirenboim EinatElran Nesya Gordon Rachel Marom Guy Levy Meuhedet Liora Valinsky Yossi Zini Alon Yaffe Leumit Daniel Vardi Eran Matz Doron Dushnitzky Nirit Peretz External auditor: AlizaLukach Israel National Institute for Health Policy Research Advisory boards

30. Importance and Relevance

31. Evidence – Moving Target HbA1c control “Intensive therapy was stopped after a mean of 3.5 years due to increased mortality”The Action to Control Cardiovascular Risk in Diabetes Study Group N Engl J Med 2008; 358:2545-2559 .

32. Ability to Quantify – Defining Diabetes 32

33. Reporting 33

34. Quasi-external Audit