Measuring and Improving Quality in Managed Care: Some Statistical and Computing Issues

1. Measuring and Improving Quality in Managed Care: Some Statistical and Computing Issues Randall K. Spoeri, Ph.D. Vice President Medical and Quality Informatics HIP Health Plans New York, NY HIP Health Plans/Medical & Quality Informatics

2. Coverage • Measurement and Quality in Managed Care • Some Statistical and Computing Issues • Measure Definitions • Data Availability and Quality • Risk Adjustment • Analytical/Interventional Use of Results • Predictive Modeling • Thoughts About the Future HIP Health Plans/Medical & Quality Informatics

3. About HIP Health Plans • A managed care company with membership of over 1 million in metropolitan NYC and FL • Has a long history of over 50 years of service to NYC residents • 2000 revenues budgeted at $2 Billion • Has a strong commitment to quality measurement and improvement HIP Health Plans/Medical & Quality Informatics

4. Measurement and Quality in Managed Care • Managed care has experienced rapid expansion in recent years • With that expansion, concerns have been expressed about the quality of care delivered • In response, quality management efforts have relied heavily on performance measurement and accreditation review HIP Health Plans/Medical & Quality Informatics

5. Measurement and Quality in Managed Care • The National Committee for Quality Assurance (NCQA) has been the major provider of accreditation reviews and performance measurement standards • The Health Plan Employer Data and Information Set (HEDIS) is the NCQA-sponsored collection of standardized performance measures widely used by Managed Care Organization (MCOs) HIP Health Plans/Medical & Quality Informatics

6. Measurement and Quality in Managed Care • With the growth of managed care and the increasing focus on quality and performance measurement, MCOs have begun to expand their staff to include medical and quality informatics personnel HIP Health Plans/Medical & Quality Informatics

7. Measurement and Quality in Managed Care • Skills of Medical and Quality Informatics (MaQI) staff include: statistics and statistical computing, epidemiology and public health, psychology, and economics • Demand is high for this type of individual, not only in MCOs, but also in allied areas such as hospital organizations and pharmaceutical companies HIP Health Plans/Medical & Quality Informatics

8. Measurement and Quality in Managed Care • Statisticians and computing professionals can make major contributions to the health care industry, especially managed care • The opportunities are many, as are the challenges HIP Health Plans/Medical & Quality Informatics

9. Some Statistical and Computing Issues • Measure Definitions • Data Availability and Quality • Risk Adjustment • Analytical/Interventional Use of Results • Predictive Modeling HIP Health Plans/Medical & Quality Informatics

10. Measure Definitions • In order that comparisons can be made among health plans, it is essential that measure definitions be uniform, unambiguous, and precise • In the managed care world, HEDIS is the standard set of performance measures used by MCOs HIP Health Plans/Medical & Quality Informatics

11. Measure Definitions • The recently announced HEDIS 2001 will have 52 measures across 7 domains of care: Effectiveness of Care, Access/Availability of Care, Satisfaction with the Experience of Care, Health Plan Stability, Use of Services, Informed Health Care Choices, and Health Plan Descriptive Information HIP Health Plans/Medical & Quality Informatics

12. Measure Definitions • Example measures: Childhood Immunizations, Breast Cancer Screening, Beta Blocker Treatment After a Heart Attack, Practitioner Turnover, Cesarean Section Rate • Developing specifications (i.e., detailed definitions) for these measures is an arduous task, often involving statistical issues HIP Health Plans/Medical & Quality Informatics

13. Measure Definitions • Some statistical issues include: • Measure reliability/validity • Data source (administrative or medical record) • Complete enumeration or sample (or hybrid) • Sampling methodology • Sample size • Technical Panels were convened by NCQA to advise on these and other issues HIP Health Plans/Medical & Quality Informatics

14. Data Availability and Quality • Encounter data are administrative data supplied by a provider (i.e., a physician) to a health plan on outpatient visits by the plan’s members • The incentives to supply these data range from none to weak, to strong: monetarily or contractually or both HIP Health Plans/Medical & Quality Informatics

15. Data Availability and Quality • Consequently, encounter data repositories are often incomplete and frequently of poor quality • Many physicians don’t see or understand the importance of this encounter data, and view it as yet another disruption imposed by MCOs HIP Health Plans/Medical & Quality Informatics

16. Data Availability and Quality • The movement from paper-based systems for submission of encounter data, to electronic ones has promise • The ever-expanding use of encounter data for such things as physician performance measurement (so-called “provider profiles”) has begun to garner physician attention HIP Health Plans/Medical & Quality Informatics

17. Data Availability and Quality • Federal and state mandates may also help to improve the completeness of encounter data • But even when data repositories are complete, there remain major questions about the quality of the data contained therein • Encounter, claim, pharmacy, lab, and other health care data are still of uneven quality HIP Health Plans/Medical & Quality Informatics

18. Data Availability and Quality • One approach, taken by NCQA, in connection with the quality of HEDIS data is to require that the complete system for this data’s production be audited (i.e., externally reviewed and verified): structure, process and outcome HIP Health Plans/Medical & Quality Informatics

19. Data Availability and Quality • Other techniques used in data quality assurance include the examination of source code used for administrative data pulls, medical record reviews to measure the accuracy of administrative data counterparts, and medical record re-review to assess the accuracy of primary chart abstraction HIP Health Plans/Medical & Quality Informatics

20. Data Availability and Quality • With the increasing reliance of MCOs on data and measurement, through complex data structures like data warehouses, the completeness and accuracy of the information becomes paramount • Statisticians and IS/IT professionals need to work with clinicians and administrators to continuously improve these data systems HIP Health Plans/Medical & Quality Informatics

21. Risk Adjustment • When comparing entities such as hospitals and physicians, it is important that an adjustment be made for differences in the mix of patients being treated • For example, it would be inappropriate to compare one hospital providing mostly obstetrical care in the suburbs to an inner city trauma center hospital using mortality HIP Health Plans/Medical & Quality Informatics

22. Risk Adjustment • Similarly, comparing a physician seeing predominantly young patients to another physician seeing a large proportion of Medicare patients is not appropriate when looking at pharmacy usage • Many of the measures used to assess hospital and physician performance need risk adjustment, a leveling of the field based upon differing patient characteristics HIP Health Plans/Medical & Quality Informatics

23. Risk Adjustment • Characteristics used generally include age, gender, diagnosis and procedure, as well as comorbidities, for example • One commonly used approach to risk adjust performance statistics is to use the concept of “observed and expected” HIP Health Plans/Medical & Quality Informatics

24. Risk Adjustment • The observed result could be a hospital’s average length of stay for a certain surgical procedure • The expected result would be that predicted using epidemiological or statistical methods for the patient population undergoing the surgical procedure at that hospital HIP Health Plans/Medical & Quality Informatics

25. Risk Adjustment • Statistical issues include methodology selection, model calibration, and explication • Popular methods include multiple and logistic regression and survival analytic (Bailey-Makeham) • Calibration issues often surround data: timeframe, geography, availability and quality, breadth HIP Health Plans/Medical & Quality Informatics

26. Risk Adjustment • Explication: Users of the data often become suspicious of methods they don’t understand (e.g., complex statistical methods), even when the methods are sound • Computing issues arise in building and handling the enormous databases common in managed care (e.g., encounters) HIP Health Plans/Medical & Quality Informatics

27. Risk Adjustment • Computing challenges also arise out of building and using complex risk adjustment models, or even in using models developed by others (i.e., commercial vendors) • Nevertheless, these issues and challenges provide opportunities for statisticians, computing professionals, and clinically prepared individuals, working together HIP Health Plans/Medical & Quality Informatics

28. Analytical/Interventional Use of Results • The best measurements and measurement systems are of no value unless they are followed by action and behavior change • Some analyses prompt action and intervention by their very nature…an example follows using real HIP data HIP Health Plans/Medical & Quality Informatics

29. HIP Health Plans/Medical & Quality Informatics

30. Analytical/Interventional Use of Results • Most analyses, however, require an intervention strategy • For example, learning that an MCO has a low rate of childhood immunizations could prompt a variety of actions HIP Health Plans/Medical & Quality Informatics

31. Analytical/Interventional Use of Results • Example interventions: • Reminder to Primary Care Physician • Reminder to Pediatrician • Reminder to Parents • Reminder stickers for medical chart • Promotional brochures • Educational forums HIP Health Plans/Medical & Quality Informatics

32. Predictive Modeling • Most MCOs have undertaken disease management initiatives for high prevalence/high cost conditions, such as asthma, diabetes, congestive heart failure and hypertension • As part of these efforts, predictive modeling is sometimes undertaken to identify high risk individuals with a given disease HIP Health Plans/Medical & Quality Informatics

33. Predictive Modeling • The goals are to: • Avoid costly ER visits or hospitalizations • Treat individuals at higher risk earlier in the disease process, and monitor them more closely • Reduce the risk of adverse outcomes • Improve satisfaction and quality of life HIP Health Plans/Medical & Quality Informatics

34. Predictive Modeling • Some of the methodologies used include • Multiple Regression (Linear and Nonlinear) • Logistic Regression • Neural Networks • Time Series • Many of the disease management vendors that have come on the scene tout predictive modeling as a value-added benefit HIP Health Plans/Medical & Quality Informatics

35. Predictive Modeling • Many MCOs do some of this work themselves or in partnership with disease management vendors • There is a need, however, to assure that the statistical methodologies are being properly utilized, and that the data used to both build and apply the models be accurate • Results to date have been encouraging HIP Health Plans/Medical & Quality Informatics

36. Thoughts About the Future • MCOs will continue to expand their performance measurement and improvement activities • Accreditors (e.g., NCQA), Federal and State Agencies (e.g., HCFA and DOH/DOI), and purchasers/employers are driving much of this expansion HIP Health Plans/Medical & Quality Informatics

37. Thoughts About the Future • The use of statistical methods in MCO quality management will increase • Administrative data systems must improve to greatly reduce the need for medical chart abstraction • Opportunities for statisticians with a bent toward computing will abound in MCOs HIP Health Plans/Medical & Quality Informatics

38. Thoughts About the Future • “The future ain’t what it used to be” - Yogi Berra HIP Health Plans/Medical & Quality Informatics