Preference measurement in CEA: Are we capturing values or creating them?

1. Preference measurement in CEA: Are we capturing values or creating them? Peter A. Ubel, M.D. Program for Improving Health Care Decisions Ann Arbor VAMC University of Michigan Health System

2. A Policy Dilemma • Imagine Medicaid program is choosing a colon cancer screening test • Test #1 • Inexpensive - can offer to everyone • Less effective - saves 1,000 lives • Test #2 • More expensive - can offer to 1/2 of the people • More effective – saves 1,100 lives • Which test would you choose?

3. If choosing according to CEA • You would choose Test #2 • The one that saves 1,100 lives • CEA helps identify health care interventions • That maximize • The average health of a population

4. What do people actually choose? • Test #1 (saving 1,000 lives) chosen by • 55% of general public • 55% of medical ethicists • 45% of CEA experts

5. In This Talk • Present evidence demonstrating that CEA does not capture people’s allocation preferences • Show that people’s allocation preferences • While at odds with CEA • Are often at odds with themselves • Internally inconsistent • Susceptible to irrelevant information • Occasionally downright confused

6. Another Dilemma

7. A transplant allocation choice • Imagine there is a blood test that predicts the outcome of liver transplant • 200 patients • Divided into two groups based on prognosis • Only 100 organs available • Subjects received one of five allocation choices • 80% vs. 70% • 80% vs. 50% • 80% vs. 20% • 40% vs. 25% • 40% vs. 10%

8. Allocation decisions % organs to betterprognostic group 80/70 80/50 80/20 40/25 40/10 Total <50 50 51-75 76-99 100

9. Allocation decisions % organs to betterprognostic group 80/70 80/50 80/20 40/25 40/10 Total <50 3 0 0 9 3 3 50 53 33 26 40 14 33 51-75 22 27 21 14 29 22 76-99 9 6 29 11 37 19 100 13 33 24 26 17 22

10. Allocation decisions % organs to betterprognostic group 80/70 80/50 80/20 40/25 40/10Total <50 3 0 0 9 3 3 50 53 33 26 40 14 33 51-75 22 27 21 14 29 22 76-99 9 6 29 11 37 19 100 13 33 24 26 17 22 100 13 33 24 26 17 22

11. Allocation decisions % organs to betterprognostic group 80/70 80/50 80/20 40/25 40/10 Total <50 3 0 0 9 3 3 50 53 33 26 40 14 33 51-75 22 27 21 14 29 22 76-99 9 6 29 11 37 19 100 13 33 24 26 17 22 50 53 33 26 40 14 33

12. What do these two studies prove? • Colon cancer study • That the public values equity in allocating resources • Disagree with CEA’s emphasis on efficiency • Transplant study • Preference for equity over efficiency is not absolute • “Consensus view” strikes a balance • People’s preferences seem really damn reasonable • When more to gain from efficiency, more preference for it

13. What values should guide allocation decisions?

14. 1. Priority for treating severely ill patients • Imagine an illness A that causes severe health problems • treatment will help patients a little • Imagine illness B that causes moderate problems • treatment will help patients considerably • The cost of treatment is the same in both cases

15. What do you believe? • Most funding should be allocated to illness A, involving severe health problems that improve a little • Most funding should be allocated to illness B, involving moderate health problems that improve considerably

16. Severity Study - Treatment Choices • Most funding should be allocated to illness A, involving severe health problems that improve a little 40 • Most funding should be allocated to illness B, involving moderate health problems that improve considerably 60

17. 1 2 3 4 5 6 7 2. Avoiding Discrimination Against People With Disabilities Disability level Full health Paraplegia Death Death

18. How Valuable is Life with Paraplegia? • Asked People • How many lives of people with paraplegia would need to be saved • To be equally beneficial as saving 100 lives of people who could be returned to perfect health • 65% of people said the number should be 100

19. 3. Age discrimination is OK • Allocating 100 transplantable livers among • 100 35 year olds • 100 65 year olds • Even distribution • Favored by 40% of people • Priority to younger patients • Favored by 57%

20. Fine-tuning CEA

21. Tweaking the numbers • Proposals to adjust numerical weights in CEA to account for people’s values • Nord: severity weights • Williams: age-adjustment of QALYs • Basic idea • Current measure:QALY = P1 U1 + P2 U2 +… • Revised measure: QALY = P1 U1 X1 + P2U2 X2 +…

22. Assumptions underlying the fine-tuning proposals • Public has stable allocation preferences • These preferences are quantifiable • CEA is amenable, methodologically, to incorporating these preferences

23. It’s time to shatter some assumptions

24. 1. People Get Confused

25. Reminder of Transplant Allocation decisions % organs to betterprognostic group 80/70 80/50 80/20 40/25 40/10 Total <50 3 0 0 9 3 3 50 53 33 26 40 14 33 51-75 22 27 21 14 29 22 76-99 9 6 29 11 37 19 100 13 33 24 26 17 22

26. Do people understand their allocation choices? • After people made their allocation choices, we asked them • What distribution of organs would maximize survival • In all cases, correct answer = all 100 organs to group with better prognosis • Majority of people did not give correct answer • E.g. 80/20 group → 80/20 distribution

27. Percent of subjects making choice Those who did not understand (n=96) Those who understood (n=71) Allocation choice <50 0 50 14 51-75 18 76-99 18 100 49 Do confused people make different allocation choices? 3 48 26 20 3

28. Even when people aren’t confused . . . • Suppose 200 transplant candidates can be ranked from 1 – 200 • By prognosis • Based on a blood test • Would you give organs to top 100 patients? • Majority say yes!

29. 2. People hate saying no to a whole group of patients • Blood test ranks people 1 – 200 • Okey Dokey • Blood test divides patients into two groups • No one wants to abandon second group • What if you could ignore blood test? • 41% would choose to do so!

30. 3. People like “easy outs” • Reminder of “severity study” • Most funding should be allocated to illness A, involving severe health problems that improve a little40 • Most funding should be allocated to illness B, involving moderate health problems that improve considerably 60

31. People like “easy outs” • Reminder of “severity study” • Most funding should be allocated to illness A, involving severe health problems that improve a little • Most funding should be allocated to illness B, involving moderate health problems that improve considerably • Equal $ to A and B 10 15 75

32. 4. People often refuse to make tradeoffs • The Person Tradeoff (PTO) preference measure <How many>people with mild shortness of breath would need to be cured to be equally good as Curing 100 people with severe shortness of breath?

33. Types of PTO Refusals • Two types of refusals: • Equality Refusals • The choices are equally good • Curing 100 people of quadriplegia = curing 100 people of foot numbness • High Refusals • Extremely high indifference points • Curing 100 people of quadriplegia = curing 300,000,000 people of foot numbness

34. Frequency of PTO Refusals Most prevalent

35. Is the problem the Decision-maker Perspective? • Imagine that YOU ARE THE EXECUTIVE DIRECTOR of a regional health system …you have only enough money to fund one treatment program….THE FINAL DECISION IS UP TO YOU. • … you must choose between two treatment programs… who would you cure?

36. Study Question • Does perspective matter? • Will a non-decision-making perspective encourage more people to make tradeoffs? • Less negative emotion • Less pressure • Easier

37. Evaluator Perspective • Imagine two regional health systems…the Executive Director of each system had only enough money to fund one treatment program…The health systems …were the same in every way except for the treatment program each Executive Director decided to fund. • The Directors made the following decisions… who made a better decision?

38. Results

39. Results

40. 6. What do people mean by “equity”?

41. Reminder of Colon Cancer Study Design • Imagine Medicaid program is choosing a colon cancer screening test • Test #1 • Inexpensive - can offer to everyone • Less effective - save 1,000 lives • Test #2 • More expensive - can offer to 1/2 of the people • More effective - save 1,100 lives

42. Why Do People Value Equity? • Test 1 can be offered tomorepeople than Test 2 • Test 1 can be offered to everyoneand Test 2 cannot Do people’s preferences for equity over efficiency persist when neither test can be offered to the entire population?

43. Is Equity All or Nothing? AB% 1,000 lives 1,100 lives choosing A 1. 100% 50% 2. 90% 40% 3. 50% 25%

44. Is Equity All or Nothing? AB% 1,000 lives 1,100 lives choosing A 1. 100% 50% 56 2. 90% 40% 27 3. 50% 25% 28

45. Isn’t 100% Arbitrary? Now imagine that the situation has changed in the following way: Because of an unusually weak economy, the number of people poor enough to qualify for Medicaid is doubled. That means twice as many people will be enrolled in Medicaid as had been predicted. However, there’s no change in the budget for colon cancer screening. . . Vice versa

46. Arbitrary Design AB% 1,000 lives 1,100 lives choosing A 4. 100% 50% 50% 25% 5. 50% 25% 100% 50%

47. Arbitrary Results AB% 1,000 lives 1,100 lives choosing A 4. 100% 50% 62 50% 25% 64 5. 50% 25% 24 100% 50% 40

48. Colon Cancer Thoughts • Preferences for equity vs. efficiency are fragile • Preferences depend on whether more effective tests can be offered to 100% of a population • People are only moderately sensitive to the “arbitrariness” with which populations are defined

49. 7. Revisiting attitudes toward paraplegia • Prior result • Saving 100 people with paraplegia • Equally good as saving 100 non-disabled people • Could conclude that • When saving lives, disabilities like paraplegia don’t matter • Or pre-existing disabilities don’t matter

50. Pre-existing versus new paraplegia • Asked about pre-existing paraplegia • 100 non-disabled = 100 paraplegia • Then asked about onset of paraplegia • 100 non-disabled = 126 paraplegia • Conclusion? • Care more about saving lives of people with pre-existing paraplegia? • But still don’t think paraplegia is too bad?