1 / 40

Bayesian Games

Bayesian Games. Bayesian Games. Definitions and Static Bayesian Games Example of solving a static Bayesian game Sender-Receiver Bayesian Games Equilibrium concepts An example of Bayesian games in Computer Science: Ad Hoc Networks: Modelling cooperation. What is a Bayesian game?.

PamelaLan
Download Presentation

Bayesian Games

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bayesian Games © Petteri Nurmi 2003

  2. Bayesian Games • Definitions and Static Bayesian Games • Example of solving a static Bayesian game • Sender-Receiver Bayesian Games • Equilibrium concepts • An example of Bayesian games in Computer Science: Ad Hoc Networks: Modelling cooperation © Petteri Nurmi 2003

  3. What is a Bayesian game? • A strategic form game with incomplete information • What is incomplete information? • Some players don’t know the payoff of the others • Incomplete information is not imperfect information • Imperfect information = Players don’t observe the actions of others correctly • Classic reference: • Harsanyi J., 1967-1968, • Games with incomplete information played by Bayesian players • Management Science 14: 159-182; 320-334; 486-502 © Petteri Nurmi 2003

  4. Harsanyi’s Model • The game is transformed into a game of imperfect information • A prior move by nature • Nature’s move determines player’s ”type” • The equilibrium of this game is the Bayes-Nash equilibrium • John C. Harsanyi (29.5. 1920 – 2000) • Nobel prize in 1994 together with • John F. Nash Jr. • Reinhard Selten • "for their pioneering analysis of equilibria in the theory of non-cooperative games" © Petteri Nurmi 2003

  5. Mathematical Intermezzo • A Bayesian Game consists of the following components • A (finite) set of players N = {1,…, n} • An action set for each player Ai ; A = χi ∈ N Ai • A type set Θi ; Θi = χi ∈ N Θ i • A probability function pi: Θ i Δ (Θ -i) • A payoff function: ui: A x Θ  ℝ © Petteri Nurmi 2003

  6. Type? • In Harsanyi model nature selected player’s type  What is a type? • Any private information (or is not common knowledge) that is relevant to the players’ decision making • Such as? • Player’s payoff function • Player’s beliefs about other players’ payoff functions • Beliefs about what players believe his beliefs are • … and so on © Petteri Nurmi 2003

  7. Type continued • Let us denote Θi player i’s type • Type Θi is observed by player i only • p(Θi | Θ-i ) denotes player i’s conditional probability about his opponents types given his type Θ-i = (Θ1 , …, Θi-1, Θi+1 , …, ΘI ) • We assume that the marginal pi (Θi) is strictly positive. © Petteri Nurmi 2003

  8. Strategy • A pure-strategy space Si represents choices of actions • σ(Θi) is the strategy player i chooses when his type is Θi • Mathematically defined: • Strategy is a mapping from set of types to the set of Actions • σ(Θi) = Ai ; σi : Θi Ai∀ i ∈ N (pure) • σ(Ai | Θi) = α ; σi : Θi Δ(Ai) ∀ i ∈ N (mixed) • α represents the (conditional) probability that player i chooses action i when the type is Θi © Petteri Nurmi 2003

  9. Equilibrium • Bayesian equilibrium = Bayes-Nash equilibrium • Player i maximizes her expected utility conditional on type Θi si (Θi) ∈ arg max  p(Θi | Θ-i) ui (s i’, s-i (Θ-i), (Θi, Θ-i)) si’ ∈ SiΘ-i © Petteri Nurmi 2003

  10. Enter Don’t 0, -1 2, 0 1.5, -1 3.5, 0 B 2, 1 3, 0 2, 1 3, 0 DB LOW Example • Consider the following game • Player I decides whether to build a new factory • Simultaneously player II decides whether to enter or not • Player I’s decision depends on her building cost that is unknown to player II Enter Don’t B DB HIGH © Petteri Nurmi 2003

  11. Example cont. • This can be seen as a Bayesian game • Set of players N = { I, II } • Action sets A1 = { B, DB} ; A2 = {Enter, Don’t } • Type sets Θ1 = { HIGH, LOW } ; Θ2 = { X } • Player II has a singleton set as the type space so we can ignore it. • Let cl be low cost and ch high cost types © Petteri Nurmi 2003

  12. Enter Don’t Enter Don’t 0, -1 2, 0 1.5, -1 3.5, 0 B B 2, 1 3, 0 DB 2, 1 3, 0 DB HIGH LOW Example cont. • A strategy for player I is an action for EACH of it types • For the high-cost type of player I we have a dominant strategy don’t build © Petteri Nurmi 2003

  13. Enter Don’t 1.5, -1 3.5, 0 B 2, 1 3, 0 DB LOW Example cont. • The best-response for player I low-cost type depends on player II’s strategy u1(B; y; cl) = 1.5y + 3.5(1-y) = 3,5 – 2y u1(DB; y; ch) = 2y + 3(1 – y) = 3 – y • Player I’s low-cost type prefers building IF y ≤ ½ © Petteri Nurmi 2003

  14. Example cont. • For player II we must first consider the possibility that the cost is actually low  u2(E;x) = p + (1-p) [-2x + 2 (1 –x)] = 2 – 4(1 – p)x  x ≤ 1 / 2(1 – p) (:=w) • Now we need to compare the best-response correspondences, for player II the correspondence is {1} x < w y*(x) = [0, 1] x = w {0} x > w © Petteri Nurmi 2003

  15. Example cont. • In a similar way we get the correspondence mapping for player I. • Now the Bayesian equilibrium is the intersection of the correspondence functions (with a fixed value for p1) © Petteri Nurmi 2003

  16. © Petteri Nurmi 2003

  17. Sender-Receiver Games • Game has two players a sender and a receiver • Sender sends a signal to receiver who then chooses an appropriate action • Player I has private information about his type • Player II has only one type, which is considered common knowledge • “A move by nature” © Petteri Nurmi 2003

  18. Strategies in S-R games • A pure strategy for the Sender is a one-to-one correspondence mapping m: Θ M • Let σ(m | Θ) be the probability that a type Θ-sender sends message m  mixed-strategy for sender • For receiver let p(a | m) be a mixed-strategy (choose action a if message = m) • On-the-path messages ℳ+(Θ) = {m: ∃ θ ∈ Θσ(m | Θ) > 0 } = supp σ(Θ) © Petteri Nurmi 2003

  19. Payoffs in S-R games • Sender’s (expected) payoff: a∈Aρ(a|m)u(m,a,θ) • For the receiver? • Must consider every type and every message • E(v(a,σ)) = m∈Mθ∈Θρ(θ) σ(m| θ)u(m,a,θ) © Petteri Nurmi 2003

  20. Bayes’ rule in action • For any on-the-path message m, the receiver’s posterior belief that player I is of type θ, is pB(θ |m) pB(θ | m) = p(θ)σ(m|θ) θ’∈Θp(θ’)σ(m| θ) • POSTERIOR rule for updating PRIOR beliefs! © Petteri Nurmi 2003

  21. Bayes Equilibrium in S-R games • In a Sender-Receiver game the Bayesian equilibrium is a triple (σ, ρ, θ) ∈ℳθxAMx(Δ(θ))M satisfying the following conditions: • For all types θ ∈ Θ supp σ(Θ) ⊂M´(ρ, Θ) • For all on-path-messages: ∀ m∈ M+(σ) supp ρ(m) ⊂A´(ρ´, Θ) • The conditional posterior belief system is consistent with Bayes’ rule whenever possible © Petteri Nurmi 2003

  22. Perfect Bayesian Equilibrium • In a Sender-Receiver game the perfect Bayesian equilibrium is a triple (σ, ρ, θ) ∈ℳθxAMx(Δ(θ))M satisfying the following conditions: • For all types θ ∈ Θ supp σ(Θ) ⊂M´(ρ, Θ) • For all messages: ∀ m∈ M(σ) supp ρ(m) ⊂A´(ρ´, Θ) • The conditional posterior belief system is consistent with Bayes’ rule whenever possible © Petteri Nurmi 2003

  23. Other types of Equilibrium • Perfect Bayesian equilibrium in Multi-Stage Games • Posterior beliefs are independent, and all types of player i have the same beliefs • Bayes rule to update beliefs (history information?) • “no signalling what you don’t know” • Posterior beliefs need to be consistent with a common joint distribution • Extensive-Form games: Sequential Equilibrium • See Fudenberg, D., and J.Tirole Game Theory 1991 The MIT Press p. 337-341 • Trembling-Hand Perfect Equilibrium (p. 351-356) • Proper equilibrium (p.356-359) © Petteri Nurmi 2003

  24. Bayesian games in CS • Ad Hoc Networks • Auctions • Social learning • The web search game • Voting © Petteri Nurmi 2003

  25. Ad hoc networks • MANET = Mobile Ad Hoc Networks • A set of mobile hosts, each with a transceiver • No base stations; no fixed network infrastructure • Multi-hop communication • Routing and packet forwarding takes place in a dynamical network topology • Game Theory and MANET? • Routing mechanisms for “selfish cooperation” © Petteri Nurmi 2003

  26. Cooperation in MANET • Reference: Modelling cooperation in Mobile Ad Hoc Networks: A formal description of Selfishness; Urpi A., Bonuccelli M., and Giordano S. • Modelling Ad Hoc Networks with Bayesian games • The nodes are the players • Nodes have to periodically select whether to forward or not • Nodes have incomplete information about the total traffic in the network • Nodes have local information about their neighbourhood © Petteri Nurmi 2003

  27. cont. • Important issues for each node • Energy consumption • The packets are forwarded by someone • “A shared medium” • Packets are send to every node that is within the transmission range • Prior to choosing its next action, a node has an opportunity to analyze its neighbours past behaviour • Node most decide to whom to send packets and to whom to discard packets. © Petteri Nurmi 2003

  28. The Model • Time is discrete and divided into timeslots t1,…, tn • Node i has the following information in the beginning of frame tk • Ni(tk): Set of neighbours, assumed to be fixed during a single frame • Bi(tk): The remaining energy units (in the battery) • Tij(tk): The traffic node i generated as a source and has to send to node j during frame k. (for each node j in node i’s neighbourhood) • Fij(tk-1): The number of packets that j forwarded for i during the previous frame • Rij(tk-1): The number of packets i received from j during the previous frame • Ȓij(tk-1): The number of packets i received from j during the previous frame as a final destination © Petteri Nurmi 2003

  29. The forwarding game • Nodes are the players • Player i’s type is its energy class e(i) = α, where 0 ≤ α≤ 1 • Player i as an action sets Sij(tk) i.e. the number of packets she will send to node j, and Fij(tk) the number of packets received from j during the previous frame she will forward to her • Player i’s payoff is: αe(i) Wi(tk) + (1 – αe(i)) Gi(tk) • Where • Wi(tk) is a measure of the energy spent succesfully • Gi(tk) is a the ratio of sent packets over packets that player i wanted to send. © Petteri Nurmi 2003

  30. The forwarding game cont. • Player i has prior belief for every player j in its neighbourhood, what its energy class is. • A node tries to maximize its payoff function  SELFISHNESS • We need to analyze the game as a repeated (dynamic) game and provide a utility function that makes it profitable to player i to cooperate © Petteri Nurmi 2003

  31. Problems • How to get the forwarding information? • Badly defined utility function and/or policy leads to self destruction • The usage of time slots • There is no synchronization! • Too simple decision space? • Possible other constaints. © Petteri Nurmi 2003

  32. Problems cont. • Malicious and selfish users? • Need a stronger policy • Punishing vs. Encouraging • Punished better suitable because • How to reward agents? (better throughput in a network with no authority?) • Punishing more suitable to both malicious and selfish users, encouraging/rewarding suitable only for encouraging cooperation • Theorem: Cooperation can be enforced in a mobile Ad Hoc network, provided that enough members agree on it and that no node has to forward more traffic than it generates. © Petteri Nurmi 2003

  33. Additional References • Cooperation in wireless ad hoc networks. Srinivasan V., Nuggehalli P., Chiasserini C-F, and Ramesh R. R., In Proceedings of IEEE Infocom 2003 http://citeseer.nj.nec.com/568937.html • Game Theoretic analysis of security in mobile ad hoc networks. Michiardi P., and Molva R. Technical Report RR-02-070, Institut Eurecom 2002. © Petteri Nurmi 2003

  34. Summary © Petteri Nurmi 2003

  35. Types of Bayesian games • Static Bayesian games • Dynamic Bayesian games • Sender-Receiver Games • Extensive Form Games • Multi-Stage Games • Equilibrium concepts • Bayesian Equilibrium = Bayes-Nash Equilibrium • Bayes Equilibrium (in dynamic games) • Perfect Bayes Equilibrium © Petteri Nurmi 2003

  36. Applications of static BayesianG • Packet forwarding in Ad Hoc networks • Voting mechanisms • Auction mechanisms • = MULTI-AGENT SYSTEMS • Requires: • Simultaneous competition • Multiple agents with incomplete information • Can also be non-simultaneous competition if the agents/players don’t know each others’ decisions (but have same beliefs that affect their decision-making). © Petteri Nurmi 2003

  37. Applications of Dyn. BayesianG. • Many economic applications • Design model for network protocols? • Design model for multiprocessor architectures? • Bayesian games are a suitable tool for modelling situations where there is interaction between two or more agents and the prior information is incomplete. © Petteri Nurmi 2003

  38. References • Fudenberg, D., and J.Tirole Game Theory 1991 The MIT Press • Kockesen L., Bayesian Games, http://www.columbia.edu/~lk290/ugbayes.pdf • Ratliff J., Static Games of Incomplete Information • Myatt D. P., Who Am I Playing? Incomplete Information and Bayesian Games, http://malroy.econ.ox.ac.uk/dpm/MPhilGameTheory/IncompleteStrategic.pdf • Urpi A., Bonuccelli M., Giordano S., Modelling cooperation in mobile ad hoc networks: a formal description of selfishness © Petteri Nurmi 2003

  39. Additional Material • Eyster E., and M.Rabin Cursed Equilibrium 2000 • Jackson M., Kalai E., Social Learning in Recurring Games • Khoussainov R., and N. Kushmerick Playing the Web Search Game • Tenneholtz M., Robust Decision-Making in Multi-Agent Systems © Petteri Nurmi 2003

  40. The End Bayesian Games by Petteri Nurmi http://www.cs.helsinki.fi/u/ptnurmi/papers.html © Petteri Nurmi 2003

More Related