1 / 21

Exploring Prospect Theory in Decision-Making Models

Dive into preference modeling through the lens of Bayesian and MLE methods, analyzing ground truth & uncertainties in today's schedule of decisions. Understand Prospect Theory's impact on selecting preferences under uncertainty.

stanforda
Download Presentation

Exploring Prospect Theory in Decision-Making Models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Preference Modeling Lirong Xia

  2. Today’s Schedule • Modeling random preferences • Random Utility Model • Modeling preferences over lotteries • Prospect theory

  3. Parametric Preference Learning Decisions Ground truth + Statistical model MLE, Bayesian, etc …

  4. Parametric ranking models • A statistical model has three parts • A parameter space: Θ • A sample space: S = Rankings(A)n • A = the set of alternatives, n=#voters • assuming votes are i.i.d. • A set of probability distributions over S: {Prθ(s) for each s∈Rankings(A) and θ∈Θ}

  5. Example • Condorcet’s model for two alternatives • Parameter space Θ={ , } • Sample space S = { , }n • Probability distributions, i.i.d. Pr( | ) = Pr( | ) = p>0.5

  6. Mallows’ model [Mallows-1957] • Fixed dispersion <1 • Parameter space • all full rankings over candidates • Sample space • i.i.d. generated full rankings • Probabilities: PrW(V)∝ Kendall(V,W)

  7. Example: Mallows for Eric • Probabilities: Kyle Stan > > > > > > > > > > > > > > Truth

  8. Random utility model(RUM)[Thurstone 27] • Continuous parameters: Θ=(θ1,…,θm) • m: number of alternatives • Each alternative is modeled by a utility distribution μi • θi: a vector that parameterizes μi • An agent’s latent utilityUi for alternative ci is generated independently according to μi(Ui) • Agents rank alternatives according to their perceived utilities • Pr(c2≻c1≻c3|θ1,θ2, θ3) = PrUi ∼ μi(U2>U1>U3) θ2 θ3 θ1 U3 U1 U2

  9. Generating a preference-profile • Pr(Data|θ1, θ2, θ3) = ∏V∈DataPr(V |θ1, θ2, θ3) Parameters θ3 θ2 θ1 Agent n Agent 1 … Pn= c1≻c2≻c3 P1= c2≻c1≻c3

  10. Plackett-Luce model • μi’s are Gumbeldistributions • A.k.a. the Plackett-Luce (P-L) model[BM 60,Yellott 77] • Alternative parameterization λ1,…,λm • Pros: • Computationally tractable • Analytical solution to the likelihood function • The only RUM that was known to be tractable • Widely applied in Economics [McFadden 74], learning to rank [Liu 11], and analyzing elections [GM 06,07,08,09] • Cons: may not be the best model c1 is the top choice in { c1,…,cm } c2 is the top choice in { c2,…,cm } cm-1 is preferred to cm McFadden

  11. Example 5 4 > > > > > > > > > > > > 1 Truth

  12. RUM with normal distributions • μi’sare normal distributions • Thurstone’s Case V [Thurstone 27] • Pros: • Intuitive • Flexible • Cons: believed to be computationally intractable • No analytical solution for the likelihood function Pr(P | Θ) is known … Um: from -∞ to ∞ Um-1: from Umto ∞ U1: from U2to ∞

  13. Decision making

  14. Maximum likelihood estimators (MLE) Model:Mr • For any profileP=(V1,…,Vn), • The likelihood of θisL(θ,P)=Prθ(P)=∏V∈P Prθ(V) • The MLE mechanismMLE(P)=argmaxθL(θ,P) • Decision space = Parameter space “Ground truth”θ … V1 V2 Vn

  15. Bayesian approach • Given a profile P=(V1,…,Vn), and a prior distribution 𝜋 over Θ • Step 1: calculate the posterior probability over Θusing Bayes’ rule • Pr(θ|P) ∝ 𝜋(θ)Prθ(P) • Step 2: make a decision based on the posterior distribution • Maximum a posteriori (MAP) estimation • MAP(P)=argmaxθPr(θ|P) • Technically equivalent to MLE when 𝜋 is uniform

  16. Example • Θ={ , } • S = { , }n • Probability distributions: • Data P = {10@ + 8@ } • MLE • L(O)=PrO(O)6PrO(M)4 =0.610 0.48 • L(M)=PrM(O)6PrM(M)4 = 0.410 0.68 • L(O)>L(M), O wins • MAP: prior O:0.2, M:0.8 • Pr(O|P) ∝0.2L(O) = 0.2 × 0.610 0.48 • Pr(M|P) ∝0.8 L(M) = 0.8 × 0.410 0.68 • Pr(M|P)>Pr(O|P), M wins Pr( | ) = Pr( | ) = 0.6

  17. Decision making under uncertainty • You have a biased coin: head w/p p • You observe 10 heads, 4 tails • Do you think the next two tosses will be two heads in a row? Credit: Panos Ipeirotis & Roy Radner • MLE-based approach • there is an unknown but fixed ground truth • p = 10/14=0.714 • Pr(2heads|p=0.714) =(0.714)2=0.51>0.5 • Yes! • Bayesian • the ground truth is captured by a belief distribution • Compute Pr(p|Data) assuming uniform prior • Compute Pr(2heads|Data)=0.485<0.5 • No!

  18. Prospect Theory: Motivating Example • Treat lung cancer with Radiation or Surgery • Q1: 18% choose Radiation • Q2: 49% choose Radiation

  19. More Thoughts • Framing Effect • The baseline/starting point matters • Q1: starting at “the patient dies” • Q2: starting at “the patient survives” • Evaluation • subjective value (utility) • perceptual likelihood (e.g. people tend to overweight low probability events)

  20. Prospect Theory Kahneman • Framing Phase (modeling the options) • Choose a reference point to model the options • O = {o1,…,ok} • Evaluation Phase (modeling the preferences) • a value function v: O  R • a probability weighting function π: [0,1]  R • For any lottery L=(p1,…,pk)∈Lot(O) V(L) = ∑ π(pi)v(oi)

More Related