TREATMENT OPTIMIZATION FOR PATIENTS WITH TYPE 2 DIABETES

1. TREATMENT OPTIMIZATION FOR PATIENTS WITH TYPE 2 DIABETES <Decision Analytics and Optimization in Disease Prevention and Treatment> Chapter 16 Jennifer Mason Lobo Department of Public Health Sciences, University of Virginia, Charlottesville, VA, USA 2018-08-14

2. Introduction • In this paper author describes a Markov decision process (MDP) model to optimize the order and timing of blood pressure and cholesterol medications for patients with type 2 diabetes. • The objective of this model is to determine the optimal order and timing of blood pressure and cholesterol medications to effectively manage the risk of adverse events subject to polypharmacy constraints. • Stroke • Coronary heart disease (CHD)

3. Model formulation --- states • Health state • Defined by patient’s TC, HDL, SBP • Presented by L (low), M (medium), H (high), and V (very high) • Threshold values: • 160, 200, and 240 mg/dL for TC • 40, 50, and 60 mg/dL for HDL • 120, 140, and 160 mmHg for SBP • History of adverse events • defined by the current number of each type of event the patient has incurred up to a maximum of k events of each type. • k=1 • Complete health state:

4. Model formulation --- states • Death states: • Death from stroke: • Death from CHD: • Death from other causes: • Medication states: • Medication iis not currently being taken, • Patient is taking medication i, • Patient remains on that medication, • The entire state space:

5. Model formulation --- actions • The recurring decision at each horizon: • Initiate a medication：I • Do nothing: W • Total potential medication: n • Medication budget: ( ) • Action space:

6. Model formulation --- probabilities • The probability of moving into death states: • Probability of death from other causes: • Probability of fatal stroke: • Probability of fatal CHD: • The probabilities among health states: • Transitions among health states:

7. Model formulation --- rewards • Monetary reward for one QALY: • : willingness-to-pay factor, which represents the value society places on one QALY. • : decrements in quality of life from stroke, CHD, and medication use. • Cost of initial treatment for stroke and CHD: • Follow‐up costs for stroke and CHD: • Medication costs: • Costs of all other treatment for diabetes patients:

8. Model formulation --- rewards • Monetary reward for one QALY: • : willingness-to-pay factor, which represents the value society places on one QALY. • : decrements in quality of life from stroke, CHD, and medication use. • Cost of initial treatment for stroke and CHD: • Follow‐up costs for stroke and CHD: • Medication costs: • Costs of all other treatment for diabetes patients:

9. Model formulation --- value function • The objective of the MDP is to maximize expected total rewards over a patient’s lifetime. • : updated medication state m’ that according to action a .

10. Model performance • Comparison between optimal treatment and US guideline • US guideline: • Blood pressure treatment is initiated when SBP > 130 • Thiazides -> ACE inhibitors/ARBs -> beta blockers -> calcium channel blockers • Cholesterol treatment is initiated when low‐density lipoprotein (LDL)≥100 • Statins -> Fibrates • Comparisons are made for males and females, averaged over all combinations of TC, HDL, and SBP, for • Baseline: case of no treatment

11. Model performance

12. Model performance • Incremental cost‐effectiveness ratio (ICER)

13. Data-Driven Markov Decision Process Approximations for Personalized Hypertension Treatment Planning SAGE journals First Published October 17, 2016 Schell G J, Marrero W J, Lavieri M S, et al. 2018-08-14

14. Background & Contributions • MDPs can be difficult for real-world clinical practice • Require specialized software • requires long computational times ( high dimensional state & action states ) • Developed a Poisson regression model to approximate the optimal treatment decisions from an MDP model. • Enables fast, easily interpretable, and com- parable decision support without a need for specialized software.

15. Statespace • Demographic information • Age, sex, smoking status, diabetes status • Clinical observation • Untreated systolic blood pressure (SBP) • high-density lipoprotein (HDL) • total cholesterol (TC) • 10 states: • healthy • history of CHD but no CHD event this period • history of stroke but no stroke this period • history of CHD and stroke but no adverse event this period • survived a CHD event this period • survived a stroke this period • death from a non–CVD-related cause • death from CHD event this period • death from stroke this period • dead

16. Action space • The physician makes annual treatment decisions in accordance withpolicy. • Physician is able to prescribe between 0 and 5 antihypertensive medications. • Constrain of action sets: prevent from excessive medication • minimum allowable SBP threshold : 120/90 mmHg

17. Rewards • Objective is to maximize the patient’s expected discounted QALYs • : quality of life • :burden from treatment =

18. Poisson regression model--- approximate optimal policies • Response variable • Optimal medication count at each decision period from the MDP policy • Predictor variable • Demographic information and risk factor data • Full Poisson model • Age, sex, smoking status, diabetes status, pretreatment SBP, diastolic blood pressure (DBP), HDL, TC, 5- year CVD risk as predictor. • Risk-only Poisson model • Just considered 5-year CVD risk as predictor

19. Poisson regression model analysis

20. Poisson regression model analysis