Approaching Predictive Modeling

1. Approaching Predictive Modeling From a Different Perspective Steven S. Eisenberg, MD Chief Science Officer United HealthCare 4th Annual DM Colloquium June 23, 2005

2. Overview • Types of predictive modeling • What predictive modeling can do • The usual • The less usual (focus) • Summary • Q&A

3. The future ain't what it used to be My Favorite Philosopher on Predictive Models

4. What Are We Hoping to Learn? • From an actuarialperspective • more accurately predict utilization and cost of populations • adjunct to better and more accurate pricing decisions • Perhaps flattening the actuarial cycle • From a medical management perspective • identify individuals at very high risk for high utilization • open the door to managing those individuals • case management/disease management • prior to the high utilization • mitigating some of the impact • healthier population • lower costs Leading to Improvements in: Disease Management Patient Care Quality Cost Management

5. The Tools • Model is a mathematical representation of reality • Relevant, consistent input data are needed • The outcome must be measurable • A way to relate the two mathematically must exist • Currently well over 100 different models • For our (healthcare) purposes these really break down into three groups: • Artificial Intelligence • Statistical Models • Rules Based Algorithms

6. Artificial Intelligence Statistical Models CART Rules Based (Groupers)

7. One Size Fits All ?

8. What Predictive Modeling Can Do • Stratify members • Primary or secondary • Enhance impact of interventions • Identification of high utilizers • Assign risk scores • Describe comparative severity of illness • Identify members not receiving proper care/requiring special care • Case Management/Disease Management • Highlight inconsistency/inefficiency of care • Prospectively identify adverse events • Allow focused interventions • Maximize benefits of disease management • Allow intervention earlier in disease cycle • Financial forecasting – Actuarial risk

9. Application of Predictive Models • Identifying/managing complexly ill members (hospitalization avoidance) • Refining disease management strategies • Managing pharmacy services (integrated with medical management) • Underwriting more precisely • Reimbursement based on illness burden • Assessing physician management strategies

10. Additional Uses of Modeling • Influence adoption of best practices • Track effectiveness of interventions • Establish pay for performance • Set more accurate premiums • Develop contracts with providers • Actuarial • Help plan network composition • Based on member needs • Develop specific, targeted interventions • Probabilities for certain outcomes • Practice guidelines • Practice standardization • Decrease variation

11. There is no one model that does everything the best What are you trying to do? Is there a model that fits the problem (or the data) better? What is available to you? Can you use whatever model/data you have available? What can you afford? What are you willing to compromise on? Choosing the Right Model

12. The “Usual Suspects” • Most DM programs / Healthplans use grouper rules based algorithms prospectively • ERG’s/ETG’s • DCG’s • ACG’s • Most Fraud & Abuse programs use • Decision trees • Rules based algorithms • Neural nets • Pattern analysis

13. Some Less Usual Suspects • There are some “inexpensive” and less cumbersome ways to do some predictive modeling • Trend lines • Time series • Markov Models • Pharmacy only models Statistical Models

14. Trend Lines = Regression • Definition: the technique of fitting a simple equation to real data points • Linear regression is the most common type • e.g. y=a+bx+e • Other Types • Multilinear regression • Logistic Regression • It is a mathematical way of assessing the impact and contribution of diverse/disparate variables on a process or outcome • Linear regression is used for continuous variables • Logistic regression is used for binomial variables

15. Change in $PMPM Cost Over Time Trend Lines • “Poor man’s” Predictive model • Built into Excel

16. Change in PMPM Cost over Time Example

17. Change in PMPM Cost over Time Doing a Prediction Double click on the trendline

18. Change in PMPM Cost over Time Project the Trendline 6 Months Forward The Prediction By doing so you are making the assumption that all the variables are and will remain constant

19. Time Series • Time series analysis accounts for the fact that data points taken over time may have an internal structure reflecting a pattern or more than one pattern • Trend • Seasonal variation (seasonality) • General aspects • Trend • systematic linear or (most often) nonlinear component that changes over time and does not repeat or at least does not repeat within the time range captured by our data • Seasonality • May have a relationship similar to trend but tends to repeat itself in sytematic intervals over time

20. Common Uses of Time Series • Economic Forecasting • Sales Forecasting • Budgetary Analysis • Stock Market Analysis • Yield Projections • Process and Quality Control • Inventory Studies • Workload Projections • Utility Studies • Census Analysis Example: Pharmacy Utilization

21. Example • Pharmacy Utilization over time – Excel w Trend Line Trend Line

22. Trend Component 95% CI Seasonality Component Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Example When Analyzed via Time Series

23. Example

24. The Prediction Using Time Series

25. Markov Models • A probabilistic process over a finite set of possibilities, {S1, ..., Sk}, usually called its states • The model is capable of showing the probability of any given state coming up next, pr(xt=Si), and this may depend on the prior history (to t-1). • originally introduced in the late 1960’s and early 1970’s • used for a variety of applications in science and technology • Markov disease state simulations portray the progression of disease over time • It does this by dividing the disease into discrete “states,” • specifying the risks of progression per unit time between those states, • assigning utilities and costs to each state, • and conducting a simulation with a defined end-point.

26. State Transition Figure Transition Probability Matrix Markov Models Are Often Represented Graphically

27. The Midwest Healthplan Project A Real World Example of Using Markov Models

28. Project Overview • A large Midwest Healthplan wants to understand the movement of members by segments over time and be able to identify future high cost utilizers • Leverage their investment in medical management programs/techniques • Help control runaway medical inflation • Make points with their large employers by showing proactive management • Wanted to do it without having to buy and support another technology

29. Project Overview, cont. • Focusing on certain higher risk parts of their book of business we developed the Markov Model for them to better understand their historical movement of members from one disease state level (severity) to another over time • at a population level • The model is then being run to predict what is predicted to occur over the ensuing 6-12 months • The Healthplan can then de-encrypt and identify these members and reach out to them with case/disease management • at an individual level • Outcomes and costs can be monitored over time • Pre- Post Analysis • Matched Cohort Analysis

30. Diabetes Example – Predicting Member Counts (By Age Band and Gender) (all diabetes)

31. The Distribution of Disease States Baseline Population Level

32. The Markov Model Prediction

33. 4 00.03% 3 6.30% 0% 2 1 20.50% 73.17% Individual Member Transition Prediction Member # A012556

34. Final Analysis* *with thanks to Ken Kubisty, Bearing Point Solutions

35. Pharmacy Risk Groups • Rules based, member centric • Uses only pharmacy, demographic, and eligibility data as the inputs • Developed by Symmetry Health Data Systems • Assigns weighted risk score individuals based on • distribution of drugs a member is taking, age, and sex • weights differ by: • Threshold assumption -- $250K, $100K, $50K, $25K • Stop-loss amount is typically used as the cut-off point • Combines PRG profile and weights • represents relative health risk for a member for future period

36. Advantages • Data • Availability • Cleanliness and accuracy • Timeliness • Cost effective – IT and administration • Supports more frequent risk assessment • Predictive accuracy • R squared and other predictive measures close to those of claims based systems

37. Disadvantages • Pharmacy plus medical claims can improve accuracy – e.g. • Members w/ medical use, w/o pharmacy use • Conditions where drugs not integral component of treatment • Further stratification within a disease • Incentives • linking risk to specific drug treatments may not provide best incentives for efficient and quality care • Linking risk to disease prevalence • harder to do without disease categorization

38. Example

39. The DHS Pilot Project A Real World Example of Using PRG’s

40. State of MN, Department of Human Services • Desire to extend disease management to FFS Medicaid population • ~100,000 • High risk population • High morbidity/Chronic Illness • Very low income • Distrust of managed care • Need to demonstrate to legislature that concepts work for this population • Establish the opportunity for a formalized DM approach to this population • Collect a series of success stories • Provide the data and the stories to the legislature

41. The Approach • Use the tool as the first pass to provide the basic output file • Rank order of patients by prospective risk • Analyze the medical history of the highest risk members • Create a clinical vignette of their medical history • ? Focus on those conditions and diseases that have a track record of success in disease or case management • Focus only on the top few percent of highest risk members • About 250 for the pilot project

42. The Pilot • Members included • Medicaid FFS only • Continuously enrolled for at least 18 months • Members excluded • Primary serious mental health diagnoses • Members in skilled or unskilled nursing homes • Primary concern is cognition • Need for a short time frame • Program began mid January ‘04 • Program ended mid June ’04 • Final Dataset – 14,443 members • members with highest prospective risk score had a complete claims dump for the prior 18 months • Highest 2% underwent detailed claims analysis

43. 11 females, 13 males Range 20-63 y.o. average 45.4 y.o. Costs for 18 months $5,432 - $491,331 Average $117,945 Total # of Claims/Pt 195 – 2,531 Average 1,129 Diagnoses Diabetes - 11 Chronic Renal Failure/ESRD - 9 Post kidney transplant - 5 HIV positive – 5 AIDS - 2 Cystic Fibrosis - 3 Active malignancy - 2 Smokers - 5 Results – Top 24 Patients

44. Clinical Vignettes • Member #1 is a 30 year old woman with long standing Cystic Fibrosis. She has problems with malaise, fatigue, skin disease, and hair loss as well as multiple dislocated vertebrae in her neck. She had a very rocky 18 month course with multiple recurring episodes of pneumonia requiring hospitalization as well as multiple episodes of dehydration and bouts of painful Herpes Simplex. • Her prospective risk score was over 27 and she had the highest total expenditure in the dataset of $491,000 for the 18 month time period.

45. Summary and Conclusions • Predictive Modeling is a tool • It is a method, not an answer in itself • Modeling is only an arrow to add to the quiver—it is not the whole quiver • Consider the use of multiple models • just as multiple forms of assessment are done for diagnosis • May increase reliability and accuracy • Predictive modeling is also a way to better understand your data accuracy • and conversely where you have problems with your data

46. Challenges of Predictive Modeling • All of the models are more accurate at the aggregate (population) level than at the individual level • Most results published are at the population level • Population level may work well for actuarial • Medical Mgmt is typically focused on the individual • You can adjust (improve) the results by changing the threshold, the specificity, sensitivity, etc. • Models demonstrate better R squared values when outliers are excluded • e.g. Stop-loss amounts • But the outliers may be exactly the members that you are trying to find to have the impact you are looking for

47. Summary & Conclusions • There is no one clearly superior predictive model • Certain approaches may be more valuable for underwriting • Other approaches may be more valuable for managing care • The actionability quotient must also be considered • If you cannot act on the results, the study is merely interesting • Linking models with interventions can help you improve quality and efficiency of care

48. Summary and Conclusions • All predictive models tend to overpredict low utilizers and under predict very high utilizers • Some of this may be mitigated by using a threshold and excluding costs beyond a certain point (typically at a stop-loss amount) • But this can exclude exactly those folks you may want to identify • None of the models can predict “random” events • Trauma • Pregnancy • Catastrophic Claims • Measurement of “success” is very difficult • How do you “unmanage” a case to determine savings? • But the tools are very valuable, getting better, and can be made to work • You will see increasing success over the next several years

49. You can either take action or you can hang back and hope for a miracle. Miracles are great, but they are so unpredictable. Peter Drucker Are There Any Questions?