1 / 49

Approaching Predictive Modeling

Approaching Predictive Modeling. From a Different Perspective. Steven S. Eisenberg, MD Chief Science Officer United HealthCare. 4 th Annual DM Colloquium June 23, 2005. Overview. Types of predictive modeling What predictive modeling can do The usual The less usual (focus)

marie
Download Presentation

Approaching Predictive Modeling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Approaching Predictive Modeling From a Different Perspective Steven S. Eisenberg, MD Chief Science Officer United HealthCare 4th Annual DM Colloquium June 23, 2005

  2. Overview • Types of predictive modeling • What predictive modeling can do • The usual • The less usual (focus) • Summary • Q&A

  3. The future ain't what it used to be My Favorite Philosopher on Predictive Models

  4. What Are We Hoping to Learn? • From an actuarialperspective • more accurately predict utilization and cost of populations • adjunct to better and more accurate pricing decisions • Perhaps flattening the actuarial cycle • From a medical management perspective • identify individuals at very high risk for high utilization • open the door to managing those individuals • case management/disease management • prior to the high utilization • mitigating some of the impact • healthier population • lower costs Leading to Improvements in: Disease Management Patient Care Quality Cost Management

  5. The Tools • Model is a mathematical representation of reality • Relevant, consistent input data are needed • The outcome must be measurable • A way to relate the two mathematically must exist • Currently well over 100 different models • For our (healthcare) purposes these really break down into three groups: • Artificial Intelligence • Statistical Models • Rules Based Algorithms

  6. Artificial Intelligence Statistical Models CART Rules Based (Groupers)

  7. One Size Fits All ?

  8. What Predictive Modeling Can Do • Stratify members • Primary or secondary • Enhance impact of interventions • Identification of high utilizers • Assign risk scores • Describe comparative severity of illness • Identify members not receiving proper care/requiring special care • Case Management/Disease Management • Highlight inconsistency/inefficiency of care • Prospectively identify adverse events • Allow focused interventions • Maximize benefits of disease management • Allow intervention earlier in disease cycle • Financial forecasting – Actuarial risk

  9. Application of Predictive Models • Identifying/managing complexly ill members (hospitalization avoidance) • Refining disease management strategies • Managing pharmacy services (integrated with medical management) • Underwriting more precisely • Reimbursement based on illness burden • Assessing physician management strategies

  10. Additional Uses of Modeling • Influence adoption of best practices • Track effectiveness of interventions • Establish pay for performance • Set more accurate premiums • Develop contracts with providers • Actuarial • Help plan network composition • Based on member needs • Develop specific, targeted interventions • Probabilities for certain outcomes • Practice guidelines • Practice standardization • Decrease variation

  11. There is no one model that does everything the best What are you trying to do? Is there a model that fits the problem (or the data) better? What is available to you? Can you use whatever model/data you have available? What can you afford? What are you willing to compromise on? Choosing the Right Model

  12. The “Usual Suspects” • Most DM programs / Healthplans use grouper rules based algorithms prospectively • ERG’s/ETG’s • DCG’s • ACG’s • Most Fraud & Abuse programs use • Decision trees • Rules based algorithms • Neural nets • Pattern analysis

  13. Some Less Usual Suspects • There are some “inexpensive” and less cumbersome ways to do some predictive modeling • Trend lines • Time series • Markov Models • Pharmacy only models Statistical Models

  14. Trend Lines = Regression • Definition: the technique of fitting a simple equation to real data points • Linear regression is the most common type • e.g. y=a+bx+e • Other Types • Multilinear regression • Logistic Regression • It is a mathematical way of assessing the impact and contribution of diverse/disparate variables on a process or outcome • Linear regression is used for continuous variables • Logistic regression is used for binomial variables

  15. Change in $PMPM Cost Over Time Trend Lines • “Poor man’s” Predictive model • Built into Excel

  16. Change in PMPM Cost over Time Example

  17. Change in PMPM Cost over Time Doing a Prediction Double click on the trendline

  18. Change in PMPM Cost over Time Project the Trendline 6 Months Forward The Prediction By doing so you are making the assumption that all the variables are and will remain constant

  19. Time Series • Time series analysis accounts for the fact that data points taken over time may have an internal structure reflecting a pattern or more than one pattern • Trend • Seasonal variation (seasonality) • General aspects • Trend • systematic linear or (most often) nonlinear component that changes over time and does not repeat or at least does not repeat within the time range captured by our data • Seasonality • May have a relationship similar to trend but tends to repeat itself in sytematic intervals over time

  20. Common Uses of Time Series • Economic Forecasting • Sales Forecasting • Budgetary Analysis • Stock Market Analysis • Yield Projections • Process and Quality Control • Inventory Studies • Workload Projections • Utility Studies • Census Analysis Example: Pharmacy Utilization

  21. Example • Pharmacy Utilization over time – Excel w Trend Line Trend Line

  22. Trend Component 95% CI Seasonality Component Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Example When Analyzed via Time Series

  23. Example

  24. The Prediction Using Time Series

  25. Markov Models • A probabilistic process over a finite set of possibilities, {S1, ..., Sk}, usually called its states • The model is capable of showing the probability of any given state coming up next, pr(xt=Si), and this may depend on the prior history (to t-1). • originally introduced in the late 1960’s and early 1970’s • used for a variety of applications in science and technology • Markov disease state simulations portray the progression of disease over time • It does this by dividing the disease into discrete “states,” • specifying the risks of progression per unit time between those states, • assigning utilities and costs to each state, • and conducting a simulation with a defined end-point.

  26. State Transition Figure Transition Probability Matrix Markov Models Are Often Represented Graphically

  27. The Midwest Healthplan Project A Real World Example of Using Markov Models

  28. Project Overview • A large Midwest Healthplan wants to understand the movement of members by segments over time and be able to identify future high cost utilizers • Leverage their investment in medical management programs/techniques • Help control runaway medical inflation • Make points with their large employers by showing proactive management • Wanted to do it without having to buy and support another technology

  29. Project Overview, cont. • Focusing on certain higher risk parts of their book of business we developed the Markov Model for them to better understand their historical movement of members from one disease state level (severity) to another over time • at a population level • The model is then being run to predict what is predicted to occur over the ensuing 6-12 months • The Healthplan can then de-encrypt and identify these members and reach out to them with case/disease management • at an individual level • Outcomes and costs can be monitored over time • Pre- Post Analysis • Matched Cohort Analysis

  30. Diabetes Example – Predicting Member Counts (By Age Band and Gender) (all diabetes)

  31. The Distribution of Disease States Baseline Population Level

  32. The Markov Model Prediction

  33. 4 00.03% 3 6.30% 0% 2 1 20.50% 73.17% Individual Member Transition Prediction Member # A012556

  34. Final Analysis* *with thanks to Ken Kubisty, Bearing Point Solutions

  35. Pharmacy Risk Groups • Rules based, member centric • Uses only pharmacy, demographic, and eligibility data as the inputs • Developed by Symmetry Health Data Systems • Assigns weighted risk score individuals based on • distribution of drugs a member is taking, age, and sex • weights differ by: • Threshold assumption -- $250K, $100K, $50K, $25K • Stop-loss amount is typically used as the cut-off point • Combines PRG profile and weights • represents relative health risk for a member for future period

  36. Advantages • Data • Availability • Cleanliness and accuracy • Timeliness • Cost effective – IT and administration • Supports more frequent risk assessment • Predictive accuracy • R squared and other predictive measures close to those of claims based systems

  37. Disadvantages • Pharmacy plus medical claims can improve accuracy – e.g. • Members w/ medical use, w/o pharmacy use • Conditions where drugs not integral component of treatment • Further stratification within a disease • Incentives • linking risk to specific drug treatments may not provide best incentives for efficient and quality care • Linking risk to disease prevalence • harder to do without disease categorization

  38. Example

  39. The DHS Pilot Project A Real World Example of Using PRG’s

  40. State of MN, Department of Human Services • Desire to extend disease management to FFS Medicaid population • ~100,000 • High risk population • High morbidity/Chronic Illness • Very low income • Distrust of managed care • Need to demonstrate to legislature that concepts work for this population • Establish the opportunity for a formalized DM approach to this population • Collect a series of success stories • Provide the data and the stories to the legislature

  41. The Approach • Use the tool as the first pass to provide the basic output file • Rank order of patients by prospective risk • Analyze the medical history of the highest risk members • Create a clinical vignette of their medical history • ? Focus on those conditions and diseases that have a track record of success in disease or case management • Focus only on the top few percent of highest risk members • About 250 for the pilot project

  42. The Pilot • Members included • Medicaid FFS only • Continuously enrolled for at least 18 months • Members excluded • Primary serious mental health diagnoses • Members in skilled or unskilled nursing homes • Primary concern is cognition • Need for a short time frame • Program began mid January ‘04 • Program ended mid June ’04 • Final Dataset – 14,443 members • members with highest prospective risk score had a complete claims dump for the prior 18 months • Highest 2% underwent detailed claims analysis

  43. 11 females, 13 males Range 20-63 y.o. average 45.4 y.o. Costs for 18 months $5,432 - $491,331 Average $117,945 Total # of Claims/Pt 195 – 2,531 Average 1,129 Diagnoses Diabetes - 11 Chronic Renal Failure/ESRD - 9 Post kidney transplant - 5 HIV positive – 5 AIDS - 2 Cystic Fibrosis - 3 Active malignancy - 2 Smokers - 5 Results – Top 24 Patients

  44. Clinical Vignettes • Member #1 is a 30 year old woman with long standing Cystic Fibrosis. She has problems with malaise, fatigue, skin disease, and hair loss as well as multiple dislocated vertebrae in her neck. She had a very rocky 18 month course with multiple recurring episodes of pneumonia requiring hospitalization as well as multiple episodes of dehydration and bouts of painful Herpes Simplex. • Her prospective risk score was over 27 and she had the highest total expenditure in the dataset of $491,000 for the 18 month time period.

  45. Summary and Conclusions • Predictive Modeling is a tool • It is a method, not an answer in itself • Modeling is only an arrow to add to the quiver—it is not the whole quiver • Consider the use of multiple models • just as multiple forms of assessment are done for diagnosis • May increase reliability and accuracy • Predictive modeling is also a way to better understand your data accuracy • and conversely where you have problems with your data

  46. Challenges of Predictive Modeling • All of the models are more accurate at the aggregate (population) level than at the individual level • Most results published are at the population level • Population level may work well for actuarial • Medical Mgmt is typically focused on the individual • You can adjust (improve) the results by changing the threshold, the specificity, sensitivity, etc. • Models demonstrate better R squared values when outliers are excluded • e.g. Stop-loss amounts • But the outliers may be exactly the members that you are trying to find to have the impact you are looking for

  47. Summary & Conclusions • There is no one clearly superior predictive model • Certain approaches may be more valuable for underwriting • Other approaches may be more valuable for managing care • The actionability quotient must also be considered • If you cannot act on the results, the study is merely interesting • Linking models with interventions can help you improve quality and efficiency of care

  48. Summary and Conclusions • All predictive models tend to overpredict low utilizers and under predict very high utilizers • Some of this may be mitigated by using a threshold and excluding costs beyond a certain point (typically at a stop-loss amount) • But this can exclude exactly those folks you may want to identify • None of the models can predict “random” events • Trauma • Pregnancy • Catastrophic Claims • Measurement of “success” is very difficult • How do you “unmanage” a case to determine savings? • But the tools are very valuable, getting better, and can be made to work • You will see increasing success over the next several years

  49. You can either take action or you can hang back and hope for a miracle. Miracles are great, but they are so unpredictable. Peter Drucker Are There Any Questions?

More Related