411 Waverley Oaks Road ■ Suite 330 ■ Waltham, MA 02452-8414

1. Development and Evaluation of CMS-HCCConcurrent Risk Adjustment ModelsPresented byEric Olmsted, Ph.D.Gregory Pope, M.S.John Kautter, Ph.D.RTI InternationalPresented atAcademy HealthJune 26, 2005 411 Waverley Oaks Road ■ Suite 330 ■ Waltham, MA 02452-8414

2. Concurrent Risk AdjustmentIntroduction • Overview • Risk Adjustment/HCC Model • Concurrent v. Prospective • Project Goals and Challenges • Model Development • Model Evaluation • Summary and Conclusion

3. OverviewRisk Adjustment Introduction • Population Risk Adjustment: • The process by which the health status of a population is taken into consideration when setting capitation rates or evaluating patterns or outcomes of practice • Risk adjustment is used to create “apples to apples” comparisons • Risk adjustment removes the effect of health status differences • Reduces or eliminates the problem of selection

4. OverviewRisk Adjustment Model • Model calibrated on 5% national sample of Medicare fee-for-service beneficiaries • Expenditures are regressed on HCC (& demographic) risk markers to estimate incremental impact of each diagnosis on expenditures Annualized Expenditures = Σαi + Σβi + Єi αi = demographic markers βi = HCC markers

5. OverviewHCC Model • Full model contains 184 HCCs • CMS-HCC model contains 70 HCCs • CMS-HCCs: • Cover a broad spectrum of health disorders • Have well-defined diagnostic criteria • Exclude highly discretionary diagnoses • Include conditions with significant expected health expenditures • Demographic Markers • Age, Gender, Medicaid, & Originally Disabled Status • Ensure means for demographic populations correctly estimated

6. OverviewConcurrent vs. Prospective • Prospective risk adjustment uses current year diagnoses to predict next year’s expenditures • Chronic conditions are more important • Concurrent risk adjustment uses current year diagnoses to predict this year’s expenditures • Acute conditions are more important

7. OverviewConcurrent vs. Prospective • AMI: • Prospective Coefficient = $1,838 • Concurrent Coefficient = $12,211 • 63% of HCC coefficients with >$1,000 difference • R-squared: • Concurrent - .4811 • Prospective - .0981

8. Project Goals • Concurrent Risk Adjustment Project Goals: • Develop payment model for Pay-for-Performance demonstration • Develop model for use in profiling physicians • Make model consistent with prospective CMS-HCC model that is being used for MA payment, and its data collection requirements • Improve prediction across the spectrum of patient cost

9. Concurrent Modeling Challenges • Applied standard HCC model • Resulted in negative predictions and coefficients • Concurrent HCC coefficients fit high-cost beneficiaries • This forces age-sex coefficients down and they sometimes become negative • Age-sex coefficients reflect the average beneficiary • Negative age-sex coefficients can lead to negative predictions

10. Model Challenges Standard Regression

11. Model ChallengesSplit Sample Regression

12. Model ChallengesRegression through the Origin

13. Model ChallengesNonlinear Regression

14. Project GoalsModel Selection • Criteria for Model Selection • Avoid negative predictions, which lack face validity • Avoid negative coefficients • Maintain correct age-sex means to prevent age and sex selection by providers • Prefer simple models to complex models • Select model with good ‘performance’ among model evaluation measures

15. Model DevelopmentSample Statistics • 1.4 million FFS Medicare beneficiaries with mean expenditures of $5,214 • Beneficiaries with at least one CMS-HCC represent 61% of the population, but provide 94% of all Medicare expenditures

16. Model Development Standard Models • Full HCC Model • 184 HCCs & demographics • CMS-HCC Model • 70 HCCs & demographics • Interaction and Topcoding Models • Created disease and demographic interactions to tease out high-expense beneficiaries • Created topcoded models to reduce impact of outliers

17. Model DevelopmentAlternative Models • Nonlinear Models • Log model • Square root model • Split Sample Models • Designed separate models for populations with different expected expenditures • Community/Institutional • High Cost/Low Cost HCC • Catastrophic HCC • Multi-stage models including two-part and four-part logit models • Simple two-stage model with demographic multipliers • Segmentation

18. Model EvaluationStandard Model Results • Full HCC model suffers not only from 30% negative predictions, but also contains negative HCC coefficients • CMS-HCC model explains 92% of the variation that the Full HCC model explains • CMS-HCC model eliminates negative HCC coefficients • CMS-HCC model has only 10% negative predictions • Interaction and Topcoding Models • Did not sufficiently reduce negative predictions

19. Model EvaluationAlternative Model Results • Nonlinear Models • Log model and square root model did not produce reasonable predictions • Split Sample Models • Splitting sample by community/institutional did not eliminate negative predictions • Splitting sample by disease burden eliminated negative predictions

20. Model EvaluationMeasures of Model Performance • R2 within .04 for all models • R2 did not differentiate models • Predictive Ratio = Average of model’s predictions Average of actual expenditures • Where each of the two averages is taken over the individuals in the subgroup • Predicted expenditure deciles • Number of HCCs for a beneficiary

21. Model EvaluationPredictive Ratios by Expenditure Percentile

22. Model EvaluationPredictive Ratios by Expenditure Percentile

23. Model EvaluationPredictive Ratios by # of HCCs

24. Model EvaluationPredictive Ratios by # of HCCs

25. Concurrent Model EvaluationModel Summary • High Cost & Catastrophic Models performs well • Some face validity problems with splitting HCCs into “high-cost” and “low-cost” • Still has negative predictions • Four Part Model also performs well • Computationally advanced and hard to interpret intuitively • No negative predictions • Sample Segmentation Model performs very well • Also computationally advanced • Two-Stage Multiplier Model performs adequately • No face validity problems

26. Concurrent Model EvaluationConclusion • Nonlinearities cause difficulties in concurrent risk adjustment model calibration • Negative coefficients and predictions • These difficulties can be addressed with: • Nonlinear models • Split sample models • But nonlinear/split sample models add complexity • Difficult to estimate • Difficult to interpret • Adds instability • Two-Stage Multiplier Model • Good face validity, avoids negative coefficients and predictions • Simpler to estimate and interpret