Modeling Preferences

1. Modeling Preferences Farrokh Alemi, Ph.D. Sunday, February 06, 2005

2. Objectives • Model decision makers preferences • Learn how to interact with a policymaker in order to model their values • Create a mathematical model that assigns higher numbers to more preferred options • Test the accuracy/validity of the model • Focused on one decision maker’s values Alemi at George Mason University

3. Historical Basis • Bernoulli • Von Neumann • Edwards • Von Winterfeldt & Edwards Alemi at George Mason University

4. What is a model of values? • Value models help us quantify a person's preferences • Assign numbers to options so that higher numbers reflect more preferred options • Assumptions • Decision makers must select from several options • The selection requires grading the preferences for the options. • Preferences are quantified by examining the various attributes (characteristics, dimensions, or features) of the options. • Overall preference is a function of decision maker’s preferences on each attribute • Electronic Medical Record (EMR) system Alemi at George Mason University

5. Mathematical Model An Additive Multi-Attribute Value Model Overall Value = V (A1) + V (A2) +   ... + V (An) Value on Attribute 1 Value on Attribute 2 Value on Attribute n Alemi at George Mason University

6. Why Model Preferences? • To clarify and communicate • To aid decision making in complex situations • To replace decision maker in repetitive tasks • To measure hard to measure concepts Alemi at George Mason University

7. Misleading Numbers • Model scores are rough approximate measures of preference • Nominal scale • Ordinal scale • Interval scale Alemi at George Mason University

8. Who decides? What must be done? What judgments must be made? How can the model of the judgment be used? Step 1. Would It Help? Alemi at George Mason University

9. Who decides? What must be done? What judgments must be made? How can the model of the judgment be used? Administrator Step1. Would it help? Alemi at George Mason University

10. Who decides? What must be done? What judgments must be made? How can the model of the judgment be used? Administrator Judge efficiency of providers’ practice Step1. Would it help? Alemi at George Mason University

11. Who decides? What must be done? What judgments must be made? How can the model of the judgment be used? Administrator Judge efficiency of providers’ practice Severity of illness Step1. Would it help? Alemi at George Mason University

12. Who decides? What must be done? What judgments must be made? How can the model of the judgment be used? Administrator Judge efficiency of providers’ practice Severity of illness Step1. Would it help? • What plans would change if the judgment were different? • What is being done now? • If no one makes a judgment about the underlying concept, would it really matter, and who would complain? • Would it be useful to tell how the judgment was made, or is it better to leave matters rather ambiguous? • Must we choose among options, or should we let things unfold on their own? • Is a subjective component critical to the judgment, or can it be based on objective standards? Alemi at George Mason University

13. Who decides? What must be done? What judgments must be made? How can the model of the judgment be used? Administrator Judge efficiency of providers’ practice Severity of illness Model can be used repeatedly to analyze medical records Step 1.4 Would It Help? Alemi at George Mason University

14. Step 2. Select Attributes • Introduce yourself and your purpose • Be judicious about pausing • Ask the experts to introduce themselves • Start with tangible examples • Ask directly for additional attributes • Arrange the attributes in a hierarchy • Always use the expert's terminology • Use prompts that feel most natural • Take notes and do not interrupt Alemi at George Mason University

15. Step 2. Select Attributes • Introduce yourself and your purpose • Be judicious about pausing • Ask the experts to introduce themselves • Start with tangible examples • Ask directly for additional attributes • Arrange the attributes in a hierarchy • Always use the expert's terminology • Use prompts that feel most natural • Take notes and do not interrupt Alemi at George Mason University

16. Step 2. Select Attributes • Introduce yourself and your purpose • Be judicious about pausing • Ask the experts to introduce themselves • Start with tangible examples • Ask directly for additional attributes • Arrange the attributes in a hierarchy • Always use the expert's terminology • Use prompts that feel most natural • Take notes and do not interrupt Alemi at George Mason University

17. Step 2. Select Attributes • Introduce yourself and your purpose • Be judicious about pausing • Ask the experts to introduce themselves • Start with tangible examples • Ask directly for additional attributes • Arrange the attributes in a hierarchy • Always use the expert's terminology • Use prompts that feel most natural • Take notes and do not interrupt Alemi at George Mason University

18. Analyst:Can you recall a specific patient with a very poor prognosis?

19. Expert: I work in a referral center, and we see a lot of severely ill patients. They seem to have many illness and are unable to recover completely, so they continue to worsen.

20. Analyst: Tell me about a recent patient who was severely ill.

21. Expert: A 28‑year‑old homosexual male patient deteriorated rapidly. He kept fighting recurrent influenza and died from gastrointestinal cancer. The real problem was that he couldn't tolerate AZT, so we couldn't help him much. Once a person has cancer, we can do little to maintain him.

22. Analyst: Tell me about a patient with a good prognosis, say close to five years.

23. Expert: Well, let me think. A year ago we had a 32‑year‑old male patient diagnosed with AIDS who has not had serious disease since‑‑a few skin infections but nothing serious. His spirit is up, he continues working, and we have every reason to expect he will survive four or five years.

24. Analyst: What key difference between the two patients made you realize that the first patient had a poorer prognosis than the second?

25. Expert: That's a difficult question‑ patients are so different from each other that it's tough to point at one characteristic. But if you really push me, I would say two characteristics: the history of illness and the ability to tolerate AZT.

26. Analyst: What about the history is relevant?

27. Expert: If I must predict a prognosis, I want to know whether he has had serious illness in vital organs.

28.  Analyst: Which organs?

29. Expert: Brain, heart, and lungs are more important than, say, skin.

30. Step 2. Select Attributes • Introduce yourself and your purpose • Be judicious about pausing • Ask the experts to introduce themselves • Start with tangible examples • Ask directly for additional attributes • Arrange the attributes in a hierarchy • Always use the expert's terminology • Use prompts that feel most natural • Take notes and do not interrupt Alemi at George Mason University

31. Step 2. Select Attributes • Introduce yourself and your purpose • Be judicious about pausing • Ask the experts to introduce themselves • Start with tangible examples • Ask directly for additional attributes • Arrange the attributes in a hierarchy • Always use the expert's terminology • Use prompts that feel most natural • Take notes and do not interrupt Alemi at George Mason University

32. Step 2. Select Attributes • Introduce yourself and your purpose • Be judicious about pausing • Ask the experts to introduce themselves • Start with tangible examples • Ask directly for additional attributes • Arrange the attributes in a hierarchy • Always use the expert's terminology • Use prompts that feel most natural • Take notes and do not interrupt Alemi at George Mason University

33. Step 2. Select Attributes • Introduce yourself and your purpose • Be judicious about pausing • Ask the experts to introduce themselves • Start with tangible examples • Ask directly for additional attributes • Arrange the attributes in a hierarchy • Always use the expert's terminology • Use prompts that feel most natural • Take notes and do not interrupt Alemi at George Mason University

34. Step 2. Select Attributes • Introduce yourself and your purpose • Be judicious about pausing • Ask the experts to introduce themselves • Start with tangible examples • Ask directly for additional attributes • Arrange the attributes in a hierarchy • Always use the expert's terminology • Use prompts that feel most natural • Take notes and do not interrupt Alemi at George Mason University

35. Step 2. Select Attributes • Introduce yourself and your purpose • Be judicious about pausing • Ask the experts to introduce themselves • Start with tangible examples • Ask directly for additional attributes • Arrange the attributes in a hierarchy • Always use the expert's terminology • Use prompts that feel most natural • Take notes and do not interrupt Alemi at George Mason University

36. Step 2. Select Attributes • Introduce yourself and your purpose • Be judicious about pausing • Ask the experts to introduce themselves • Start with tangible examples • Ask directly for additional attributes • Arrange the attributes in a hierarchy • Always use the expert's terminology • Use prompts that feel most natural • Take notes, and do not interrupt Alemi at George Mason University

37. Step 3. Do It Again • Use two different perspectives • Survival versus mortality prompts • Check assumptions: • Attributes are exhaustive • Attributes are not redundant • Attributes are important to decision maker • Attributes are not preferentially dependent Alemi at George Mason University

38. Age Race Transmission mode Defining diagnosis  Time since defining diagnosis Diseases of nervous system Disseminated diseases Gastrointestinal diseases Skin diseases Lung diseases Heart diseases Recurrence of a disease Functioning of the organs Co-morbidity Psychiatric co morbidity Nutritional status Drug markers Functional impairment Attributes Judged Important in Measuring Severity of AIDS Alemi at George Mason University

39. Step 4. Set Attribute Levels • Decide on the best and worst • Select a target population and ask the expert to describe the possible range of the attribute. • Avoid adjectives, describe the levels • Decide on levels in between • Fill out the gaps to describe all possible situations Alemi at George Mason University

40. Analyst: I understand that patients on total parenteral treatment have the worst prognosis. Can you think of other relatively common conditions with a slightly better prognosis?

41. Expert: Well, a host of things can happen. Pick up any book . on nutritional diseases and you find all kinds of things.

42. Analyst: Right, but can you give me three or four examples?

43. Expert: Sure. The patient may be on antiemetics or nutritional supplements.

44. Analyst: Do these levels‑include a level with a moderately poor prognosis and one with a relatively good prognosis?

45. Expert: Not really. If you want a level indicative of moderately poor prognosis, then you should include whether the patient is receiving Lomotil or Imodium. 

46. Step 4. Set Attribute Levels • Decide on the best and worst • Select a target population and ask the expert to describe the possible range of the attribute. • Avoid adjectives, describe the levels • Decide on levels in between • Fill out the gaps to describe all possible situations Alemi at George Mason University

47. Levels for Skin Infection • No skin disorder • Kaposi's sarcoma • Shingles • Herpes complex • Candida or mucus • Thrush Alemi at George Mason University

48. Step 5. Assign Values to Single Attributes • Double anchored assessment method • 100 to best • 0 to worst • Rate the remaining levels • Check against different anchors Alemi at George Mason University

49. Analyst: Which among the skin disorders has the worst prognosis?

50. Expert: None is really that serious.