Estimating Outcomes in Decision Analysis

Estimating Outcomes in Decision Analysis Miriam Kuppermann, PhD, MPH Associate Professor Departments of Obstetrics, Gynecology & Reproductive Sciences and Epidemiology & Biostatistics University of California, San Francisco

Review—Last Lecture Formulate an explicit question Make a decision tree Estimate probabilities Estimate utilities Compute expected utility of each branch Perform sensitivity analysis

Preview—Where We Are Going with this Analysis? Recall Ms. Brooks and her incidental aneurysm -- to clip or not to clip? We want to: Determine her utility: clip, and not clip Compare incremental utility and cost Compare cost-per-unit of utility across private and public uses of funds.

Overview – Today’s Lecture Quantifying outcomes Measuring utilities Calculating Quality Adjusted Life Years Discounting Prenatal testing example

Quantifying Health Outcomes Mortality Life years Number of expected years of life Significant morbidity Paralysis, loss of sight Quality-Adjusted Life Years (QALYs) Expected life years adjusted for the valuation of the possible states in each year Financial valuation of outcomes Costs to patient, payor, or society Willingness to pay to avoid outcomes, obtain treatment

Health Outcomes – Mortality Mortality Death from disease/accident/procedure e.g. If Ms. Brooks undergoes surgery, one of the possible outcomes is mortality Life Years Calculate an expected value of life years using a probabilistically weighted average of expected life e.g. If Ms. Brooks does not undergo surgery, her life expectancy is less than if she did not have aneurysm, these outcomes are measured in expected life years

Health Outcomes – Morbidity Morbidity Some health state that is less than perfect -Includes underlying condition (e.g. stroke, chronic pain) and treatment side effects (nausea, hair loss) Comparison of morbidities Difficult – apples and oranges problem e.g. which is worse: Blind vs. deaf Deaf vs. paraplegia Paraplegia vs. blind

Outcomes - Utility Utility is the currency of outcomes Scaled from 0 to 1 Commonly Death = 0 Perfect Health = 1 Sounds good, but how can this be measured?

Utility Measurement Utilities are commonly estimated using comparisons to the 0 and 1 anchors Methods are: Visual Analog Scale Standard Gamble Time Trade-off

Utility Measurement Clinical Scenario: Patient in the hospital has infection of the leg Two options: BKA v. Medical Management BKA – below the knee amputation (1% mortality risk) Medical – 20% chance of infection worsening If worse, AKA – above the knee amputation (10% mortality risk) How can we value these outcomes?

Utility Measurement Visual Analog Scale Linear Scale from 0 to 1 (death 0-----------------------------1.0 cure) Where is AKA? (death 0-----------------------------1.0 cure) Where is BKA?

Utility Measurement Visual Analog Scale Advantages: quantitative, easy to understand, visual Disadvantages: may bias values to the middle, seems disconnected from medical reality

Utility Measurement--Standard Gamble Method relies on patients choosing between: 1) a certain outcome 2) a gamble between a better outcome and a worse outcome How it works: Choice 1: You live with a BKA Choice 2: The gamble – you might have a cure; you might die. Goal of method is to find the break-even point. What probability of death would you accept to avoid living with the BKA?

Utility Measurement – Standard Gamble

Utility Measurement – Standard Gamble Standard gamble measurement involves questioning patients to determine the p at which the two outcomes are equivalent Using expected utilities, the value of p [p = P(cure)] gives the utility: Utility (BKA) x Prob (BKA) = Utility(cure) x (p) + Utility(death) x (1-p) [Note that here, P(BKA) = 1.] You can demonstrate that the utility of BKA = p: Utility (BKA) = [Utility(cure) x (p) + Utility(death) x (1-p)] = [1.0 * p + 0 * (1-p)] = p

Utility Measurement – Standard Gamble In utility estimation utilizing the standard gamble measurement, using the simple comparison described, the value of p,the probability of perfect health needed to make the patient indifferent between the two choices is the estimated utility of the outcome. Advantages: the incorporation of risk into the model, comparison or choice between different outcomes. Disadvantages: possible nonrealistic choices patients may be asked to make, the difficulty of understanding the question (especially for non-gamblers), and the difficulty for some in understanding a math equation.

Utility Measurement – Time Trade-off Time Trade-off involves patients choosing between: quality of life vs. length of time alive We want to determine when patients are equivocal between choice: Time A * Utility A = Time B * Utility B e.g. -- If you have a life expectancy of 30 years with a BKA, how much time would you give up to live in your current state? Would you give up 5 years? 3 years? 1 year? 30 years * Utility (BKA) = (30-x) years * 1.0 If you’re willing to give up 3 years, that means the utility of BKA is: = [(30-3)*1/ 30] = 27/30 = 0.9

Utility Measurement – Time Trade-off Time trade off can be used to measure the utility of an outcome in the fashion described. Like standard gamble, patients need to understand subtlety of questions being asked. Advantages: the simplicity of the choice between different outcomes, consideration of long-term outcomes. Disadvantages: fails to incorporate risk, lack of clarity of when time traded occurs, different valuation of time during life, and the theoretical lack of realism of the choice.

Utility Measurement – Health State Excluding Clinical Problem 0 to 1 utility scaling is simple and useful for determining the utility of different outcomes for a single patient. However, comparisons across patients or across programs raises the issue of ‘Is a utility of 1 really available?’ Consider: (a) polio vaccinations for children, and (b) hip replacement surgery for 85-year old patients. Typically (on average) the health state of a polio-free 5-year old will be very close to 1, whereas the health state of an 85-year old even with a perfectly functioning hip will typically be less than one.

Utility Measurement – Chaining The utility of the health state of the 85-year old can be determined by “chaining”. Consider first the clinical situation under review: what is the utility, on a 0 to 1 scale, of a hip with reduced function and some pain? Let us say utility = .8. Consider next the utility of an 85-year old with no hip problems, but some other reduced function or chronic pain. Here, perhaps overall utility = .9. Total utility for the bad-hip case = .8 * .9 = .72 Here, the perfect-hip case utility = 1 * .9 = .9

Utility Measurement – Additional Information Multi-Attribute Health Status Classification System Developed by Health Utilities, Inc. Available at: http://www.healthutilities.com/overview.htm

Utilities in Decision Analysis Now that we have methods to estimate utilities, these can be used in the DA However, our outcomes often include both mortality and morbidity Want a way to add in life expectancy Quality Adjusted Life-Years (QALYs)

QALYs QALYs are generally considered the standard unit of comparison for outcomes QALYs = time (years) x quality (utils) e.g. 30 years after BKA, util (BKA) = 0.9 = 30 x 0.9 = 27 QALYs

QALYs

QALYs Aneurysm Example We said life expectancy is reduced by 2/3, so instead of 35, it is = 35 * .333 = 11.67 Here, we have assigned a utility of .5 to surgery-induced disability, so QALYs = years * utils = 11.67 * .5 = 5.8

Outcomes - Discounting However, are all years considered equal? Consider:Favorite Meal Extreme Pain Lifetime Income

Outcomes - Discounting Generally, present > future One common way to value the different times is discounting Essentially this year is worth δ more than next year δ is commonly set at 0.03 or 3% In order to compare values of all future times, a calculation, net present value, is often used NPV = 1 / (1 + δ)t Where t is number of years in the future

Outcomes - Discounting Aneurysm Example If utility is 0.5 and life expectancy is 3 years NPV would be:  Utility / (1 + δ)t However, when is year 1? Often, since events in year one occur on average half way through, we use 0.5 for year 1 NPV = 0.5 / (1.03)0.5 + 0.5 / (1.03)1.5 + 0.5 / (1.03)2.5 NPV = 0.5 * {(1.03)-0.5 + (1.03) -1.5 + (1.03) -2.5}

Outcomes - Discounting

Outcomes - Discounting Another issue is partial years Can use similar adjustment: Exponent is half way through time period Numerator is multiplied by fraction of year e.g. disability for 1.6 years, utility of 0.5 NPV =  Utility / (1 + δ)t NPV = 0.5 / (1.03)0.5 + 0.5*0.6 / (1.03)1.3

Discounting – Special Topic Issues with exponential discounting: Relatively easy to manipulate However, assumes same difference between any two time periods is the same value difference Consider today : vs. tomorrow and 10 yrs vs. 10 yrs + 1 day Essentially, “today” versus all other time periods is valued higher for many outcomes

Discounting – Special Topic Importance of discount rate chosen: Imagine a benefit of $1,000,000 to be received in 30 years Present value of this benefit at different discount rates: 1% -- $741,923 3% -- $411,987 8% -- $99,377 17% -- $9,004

Overall Review Outcomes Mortality Morbidity Measuring Utilities Visual Analog Standard Gamble Time Trade-off Quality Adjusted Life Years (QALYs) QALYs = time (years) x quality (utils) Discounting NPV =  Utility / (1 + δ)t

Department of Obstetrics, Gynecology, & Reproductive Sciences Informed Decision-Making in Prenatal Testing Miriam Kuppermann, PhD, MPH Associate Professor Decision and Cost-Effectiveness Analysis EPI 213

Prenatal Tests for Chromosomal Disorders Screening Tests (non-invasive) Expanded AFP Nuchal translucency Diagnostic Tests (invasive) Amniocentesis Chorionic villus sampling (CVS)

Women < 35 Screening first Invasive testing only if “positive” results Women > 35 Invasive testing offered Screening as an option Guidelines For Prenatal Testing Are Dichotomized By Maternal Age

Rationale for Guidelines Need to limit access to invasive testing • Inherent risk of procedure • Limited availability of providers, laboratories Age 35 selected as the threshold • Increasing risk with advancing maternal age • Threshold set where risks equal • Cost/benefit considerations Kuppermann, Nease, Goldberg, Washington. Who should be offered prenatal diagnosis? The 35-year-old question. Am J Public Health 1999; 89:160-3

Equal Risk Threshold Risk of Miscarriage = Risk of Down Syndrome Implicit assumption: equally burdensome outcomes Down-syndrome affected infant Procedure-related miscarriage

First Challenge to Guidelines Do women find procedure-related miscarriage and Down-syndrome-affected birth to be equally burdensome?

How Do Women Value Testing Outcomes? • 1082 socioeconomically and age-diverse women • English-, Spanish- or Chinese-speaking • Interviewed <20 weeks pregnant • Preferences for testing outcomes • Demographic/attitudinal questions • Testing behavior

Simplified Decision Tree for Prenatal Testing

Preference Score • “Utility” • Ranges from 0 to 1 • 1 = ideal outcome, 0=death • Measured using “time tradeoff”

Time Tradeoff Preference Elicitation Which do you prefer? Choice A Choice B 40 years of life remaining with affected child 40 years of life remaining with unaffected child (give up 0 years of life)

Time Tradeoff Preference Elicitation Which do you prefer? Choice A Choice B 40 years of life remaining with affected child 30 years of life remaining with unaffected child (give up 10 years of life)

Calculation of Time Tradeoff Scores reduced life expectancy with unaffected child (30 years) UTTO = __________________________________________ full life expectancy with affected child (40 years) = 0.75

On average, women do not equally weight the outcomes of procedure-related miscarriage and Down syndrome-affected birth Median utility for procedure-related miscarriage = 0.86 Median utility for Down-syndrome affected infant = 0.73 P<0.001 by Wilcox sign rank test Kuppermann, Nease, Learman, Gates, Blumberg, Washington. Procedure-related miscarriages and Down syndrome-affected births: implications for prenatal testing based on women’s preferences. Obstet Gynecol 2000; 96:511-6.

Preference Difference Score One way to look at the relative value women assign to procedure-related miscarriage and DS-affected birth Pref score misc – Pref score DS Higher score = greater preference for miscarriage over DS

Preferences Vary Substantially 200 175 150 125 Number 100 75 50 25 0 0 -1 -.75 -.5 -.25 .25 .5 .75 1 Utility misc - Utility DS

First Conclusion • Current guidelines do not adequately reflect the preferences of pregnant women.

Second Challenge to Guideline Old paradigm: COST BENEFIT Benefits (in $$ terms) of program should exceed costs. • Costs of offering testing should be offset by savings accrued by averting the birth of Down-syndrome-affected infants New paradigm: COST EFFECTIVENESS No $$ value assigned to outcomes. • Cost of offering testing should be “worth” the gain in quantity and quality of life.

Estimating Outcomes in Decision Analysis