Extensive Games with Imperfect Information

1. Extensive Games with Imperfect Information

2. Extensive Games with Imperfect Info • In this section, we model situations in which players move sequentially but may or may not be informed about previous player’s actions. • With perfect information, we needed to specify players, terminal histories, when players had the move, preferences and payoffs. • With imperfect information, we need to add to that list: a specification about each player’s information about the history at every point at which she moves.

3. Extensive Games with Imperfect Info Definition: an information set for a player is a collection of decision nodes satisfying: • The player has the move at every node in the information set, and • When the play of the game reaches a node in the information set, the player with the move does not know which node in the information set has (or has not) been reached

4. Extensive Games with Imperfect Info I Example 314.2 in Osborne: Battle of the Sexes U2 CP II II U2 CP U2 CP 2,1 0,0 1,2 0,0 Nash equilibria? Subgame Perfect Nash Equilibria ?

5. Extensive Games with Imperfect Info Other Examples • 315.1 in Osborne: Card Game (Like problem set question)

6. Extensive Games with Imperfect Info Definition: Strategy in an Extensive game of Imperfect information: A (pure) strategy of player i in an extensive game is a function that assigns to each of i’s information sets, Ii , an action in A(Ii ) (ie, the set of actions available to player i at information set Ii ). Definition: Mixed Strategy in an Extensive game of Imperfect information: A mixed strategy of a player in an extensive game is a probability distribution over the player’s pure strategies.

7. Extensive Games with Imperfect Info Definition: Nash Equilibrium in an Extensive game of Imperfect information: The mixed strategy profile, a*, in an extensive game is a (mixed strategy) Nash equilibrium if, for each player i and every mixed strategy ai of player i, player i’s expected payoff to a* is at least as large as her expected payoff to (ai,a*-i) according to a payoff function whose expected value represent player i’s preferences over lotteries.

8. Extensive Games with Imperfect Info II X Y 320.2 in Osborne (Commitment and Observability) X (1,1) (3,2) I (4,3) (2,4) Y

9. Extensive Games with Imperfect Info II X Y 320.2 in Osborne (Commitment and Observability) X (1,1) (3,2) I (4,3) (2,4) Y NE of Simultaneous game: (Y,Y)

10. Extensive Games with Imperfect Info II X Y 320.2 in Osborne (Commitment and Observability) X (1,1) (3,2) I I (4,3) (2,4) Y X Y NE of Simultaneous game: (Y,Y) II II Y X Y X 3,2 1,1 4,3 2,4 SPNE outcome of extensive game (I moves first, II second) of perfect information: (X,X)

11. Extensive Games with Imperfect Info 320.2 in Osborne (Commitment and Observability) -- Imperfect Information Game: How many information sets? How many subgames? 1,1 4,3 2,4 3,2 Y X X Y II 1-e X e X I X Y Chance Chance Y 1-e Y e II X Y X Y 3,2 1,1 4,3 2,4 • Nash Equilibrium for e < 1/4 ?

12. Player I moves first, but his choice is imperfectly observed by player II. • Strategies: player I: (X, Y) player II: ( (X,X),(X,Y),(Y,X),(Y,Y) ). For player II, a strategy (a,b) means play a if the signal is X and play b if the signal is Y. 1,1 4,3 2,4 3,2 Y X X Y II 1-e X e X I X Y Chance Chance Y 1-e Y e II X Y X Y 3,2 1,1 4,3 2,4 • Nash Equilibrium for e < 1/4 ?

13. Strategic Form: * Payoff calculated as: ( 3(1-e)+1(e), 2(1-e)+1(e) ) = (3-2e,2-e) 1,1 4,3 2,4 3,2 Y X X Y II 1-e X e X I X Y Chance Chance Y 1-e Y e II X Y X Y 3,2 1,1 4,3 2,4 • Nash Equilibrium for e < 1/4 ?

14. Strategic Form: *3-2e>2+2e  -4e>-1  e < 1/4 **4-2e>1+2e  -4e>-3 e < 3/4 1,1 4,3 2,4 3,2 Y X X Y II 1-e X e X I X Y Chance Chance Y 1-e Y e II X Y X Y 3,2 1,1 4,3 2,4 • Nash Equilibrium for e < 1/4 ? Equilibrium = (Y,YY)

15. Extensive Games with Imperfect Info II X Y 320.2 in Osborne (Commitment and Observability) X (1,1) (3,2) I I (4,3) (2,4) Y X Y NE of Simultaneous game: (Y,Y)  outcome = (2,4) II II Y X Y X 3,2 1,1 4,3 2,4 SPNE of extensive game (I moves first, II second) of perfect information: (X,X)  outcome = (3,2) (Bayesian) NE outcome of sequential imperfect information game (Y,YY)  outcome (2,4) Player 1 loses his first-mover advantage!

16. Extensive Games with Imperfect Info Osborne: 317.1 / 322.1 / 323.1: Entry Game Motivates a need for “beliefs” of players.

17. Extensive Games with Imperfect Info • In simultaneous (strategic) games of perfect information, our equilibrium concept was the Nash equilibrium (NE). When we moved to sequential games, we introduced a slightly stronger equilibrium called subgame perfect Nash equilibium (SPNE), primarily to rule out non-credible threats. In static games of imperfect information, we needed a further “refinement” which we called Bayesian Nash Equilibrium (BNE). Finally, in extensive games of imperfect information, we make a further refinement and introduce Perfect Bayesian Nash Equilibrium, or (PBE) for short.

18. Extensive Games with Imperfect Info A note on terminology: Perfect Bayesian Equilibrium or PBE Weak Sequential Equilibrium or WSE (Osborne’s terminology) PBE = WSE

19. Perfect Bayesian Equilibrium* • Gibbons Game, Page 176 I R (1,3) L M II II L’ L’ R’ R’ (2,1) (0,2) (0,0) (0,1)

20. Perfect Bayesian Equilibrium* • Gibbons Game, Page 176 – Strategic Form • So 2 NE in pure strategies: (L,L’) and (R,R’) • Subgame Perfect NE? Since the game has no subgames besides the whole game, NE=SPNE. • But player II playing R’ is based on a non-credible threat since L’ is always optimal if player II gets to move. So we rule out (R,R’) with the following 2 refinements. II L’ R’ L (2,1) (0,0) I M (0,2) (0,1) R (1,3) (1,3)

21. Perfect Bayesian Equilibrium* • Requirement 1: At each information set, the player with the move must have a belief about which node in the information set has been reached by the play of the game. For a nonsingleton information set, a belief is a probability distribution over the nodes in the information set; for a singleton information set, the player’s belief puts probability one on the single decision node.

22. Perfect Bayesian Equilibrium* • Requirement 2: Given their beliefs, the players’ strategies must be sequentially rational. That is, at each information set the action taken by the player with the move (and the player’s subsequent strategy) must be optimal given the player’s belief at that information set and the other players’ subsequent strategies (where a “subsequent strategy” is a complete plan of action covering every contingency that might arise after the given information set has been reached)

23. Perfect Bayesian Equilibrium* • Gibbons Game, Page 176 I R (1,3) L M II II [1-p] [p] L’ L’ R’ R’ (2,1) (0,2) (0,0) (0,1) • (p,1-p) are the beliefs of player II. p is the probability that player II puts on the history that player I has played L and (1-p) is the probability that player II puts on the history that player I has played M.

24. Perfect Bayesian Equilibrium* • Gibbons Game, Page 176 I R (1,3) L M II II [1-p] [p] L’ L’ R’ R’ (2,1) (0,2) (0,0) (0,1) • So, given player II’s beliefs, the expected payoff to player II from playing R’ is 0*p + 1*(1-p) = 1-p. The expected payoff to player II from playing L’ is 1*p + 2*(1-p) = 2-p. • Since 2-p > 1-p, for ALL p, then R’ can never be played in equilibrium. • So (R,R’) does not satisfy the requirement #2.

25. Perfect Bayesian Equilibrium* Requirements 1 and 2 insist that the players have beliefs and act optimally given these beliefs, but not that these beliefs be reasonable. We need further requirements. Definition: for a given equilibrium in a given extensive game, an information set is on the equilibrium path if it will be reached with positive probability if the game is played according to the equilibrium strategies, and is off the equilibrium path if it is certain not to be reached if the game is played according to the equilibrium strategies.

26. Perfect Bayesian Equilibrium* • Requirement 3: At information sets on the equilibrium path, beliefs are determined by Bayes’ rule and the players’ equilibrium strategies. • Requirement 4 :At information sets off the equilibrium path, beliefs are determined by Bayes’ rule and the players’ equilibrium strategies where possible.

27. Perfect Bayesian Equilibrium* • Gibbons Game, Page 176 I R (1,3) L M II II [1-p] [p] L’ L’ R’ R’ (2,1) (0,2) (0,0) (0,1) • So in the SPNE where players play (L,L’), player 2’s belief must be p = 1. “Given player 1’s equilibrium strategy (L), player 2 knows which node in the information set has been reached.” • As a second example, suppose in a mixed strategy equilibrium, player I played L with probability q1, M with probability q2, and R with probability 1-q1-q2. Then, this would force player II to set p = q1/(q1+q2), by Bayes rule.

28. Perfect Bayesian Equilibrium* Definition: A Perfect Bayesian Equilibrium consists of strategies and beliefs satisfying 1 through 4. • Crucial new features of a PBE: beliefs are elevated to the level of importance of strategies in our definition of an equilibrium. An equilibrium consists of strategiesandbeliefs for each player at each information set at which the player has the move. • Players must have reasonable beliefs both on and off the equilibrium path.

29. Signaling Games • Signaling Games: a dynamic game of imperfect information involving a sender and a receiver, S and R. • Nature draws a type for the S from a set of possible types. S observe his type and chooses a message to send to R. R observes the message (not the type) and then choose an action. • Payoffs are then realized.

30. Signaling Games • Sender may be a worker who signals to a potential employer (Reciever) with an education choice based on his type, possibly his productivity level. See Spence (1973): Job Market Signaling.

31. Signaling Games • Consider the following signaling game: a1 a1 Sender t1 m1 m2 a2 a2 z Receiver Nature Receiver 1-z a1 a1 m1 m2 Sender t2 a2 a2 If both types of senders send the same message in an equilibrium, we call this equilibrium “Pooling” If a sender sends a different message depending on his type, we called this equilibrium “Separating.”

32. Signaling Games • Denote (p,1-p) and (q,1-q) denote R’s beliefs at each of his (two) information sets. a1 a1 Sender t1 m1 m2 [p] [q] a2 a2 z Receiver Nature Receiver 1-z a1 a1 [1-p] [1-q] m1 m2 Sender t2 a2 a2 This game has four possible (pure strategy) equilibria: 1) Pooling on m1, 2) pooling on m2, 3) Separating with t1 playing m1 and t2 playing m2, 4) Separating with t1 playing m2 and t2 playing m1.

33. Signaling Games • Lets add payoffs and solve an example. 2,1 1,3 u u Sender t1 L R [p] [q] d d 0,0 4,0 0.5 Receiver Nature Receiver 1,0 2,4 0.5 u u [1-p] [1-q] L R Sender t2 d d 1,2 0,1 Solve for all PBE of this game ...

34. Signaling Games Need to consider the following candidate equilibria: • Pooling on (L,L); • Pooling on (R,R); • Separating on (L,R); • Separating on (R,L). For each type of candidate equilibrium we need strategies of each player and beliefs of any player that is uncertain at any point in the game. In this game, the receiver is uncertain at two of his information sets (the one following the L signal and the one following the R signal). So overall, we need: • A strategy for the sender, (X,Y)  (what he does if he’s type 1, what he does if he’s type 2). • A strategy for the receiver, (X,Y)  (what he does if he sees the L signal, what he does if he sees the R signal). • A belief for the receiver at the left side information set, (p,1-p)  (his belief that the L signal came from a type 1 sender, his belief that the L signal came from a type 2 sender). • A belief for the receiver at the right side information set, (q,1-q)  (his belief that the R signal came from a type 1 sender, his belief that the R signal came from a type 2 sender).

35. Signaling Games Pooling on (L,L). This means that the sender plays L if he’s type 1 or type 2. On the left, the receiver doesn’t get any additional information about the sender’s type when he sees L being played (because both types are playing L). So the receivers belief should be p = ½, the prior probability distribution of types. What does the receiver do if he sees the L signal? We see he always plays u on the left for any beliefs (including ½, ½). What does the receiver do if he sees the R signal? We see he would like to play u on the top, d on the bottom, but he cannot distinguish top from bottom. He can find beliefs (q,1-q) to make him want to do either. Now lets see if the sender really wants to player (L,L). Playing (L,L) results in the receiver playing u, so a type 1 sender gets a payoff of 1 and a type 2 sender gets a payoff of 2. If a type 1 sender deviates to R, his payoff is either 2 (if u) or 0 (if d). So in order for this to be a PBE, a type 1 sender requires that the receiver have beliefs such that he wants to play d if the receiver sees the R signal. Otherwise, the type 1 sender would want to deviate from L (and get 1) to R (and get 2). So a type 1 sender requires Erec[d|R] >= Erec[u|R]  0q+2(1-q) >= 1q+0(1-q)  q <= 2/3. If a type 2 sender deviates to R, his payoff is either 1 (if u) or 1 (if d). These are both less than what he gets by playing L (2), so we say that a type 2 sender puts no requirement on the beliefs of the receiver following the R signal. So we have a PBE at {(L,L), (u,d), (p,1-p), (q,1-q) | p = ½, q <= 2/3 }

36. Signaling Games Pooling on (R,R). This means that the sender plays R if he’s type 1 or type 2. On the right, the receiver doesn’t get any additional information about the sender’s type when he sees R being played (because both types are playing R). So the receivers belief should be q = ½, the prior probability distribution of types. What does the receiver do if he sees the R signal? With q = ½, Erec[u|R] = 1(1/2) + 0(1/2) = ½, while Erec[d|R] = 0(1/2) + 2(1/2) = 1. So the receiver should play d if he sees R. What does the receiver do if he sees the L signal? We see he always plays u on the left, no matter his beliefs (it’s a dominant strategy). Now lets see if the sender really wants to player (R,R). Playing (R,R) results in the receiver playing d, so a type 1 sender gets a payoff of 0 and a type 2 sender gets a payoff of 1. If a type 1 sender deviates to L, his payoff is 1, higher than he gets by playing R, so he would want to deviate and (R,R) cannot be part of a PBE. We could stop here … If a type 2 sender deviates to L, his payoff is 2, again higher than what he gets by playing R, so he would also want to deviate. So there does not exist a PBE involving the sender pooling on R.

37. Signaling Games Separating on (L,R). This means that a type 1 sender plays L and a type 2 sender player R. Given our definition of a PBE, beliefs are determined by equilibrium strategies. Since both information sets of the receiver are ON the equilibrium path, the only beliefs that are consistent with (L,R) is p = 1 and q = 0. I.e., if only type 1 senders play L, then if a receiver sees L, he should believe that it came from a type 1 sender with probability 1. If only type 2 senders play R, then if a receiver sees R, he should believe that it came from a type 2 sender with probability 1. So, the receiver’s optimal strategy is (u,d). Now lets see if the sender really wants to player (L,R). Playing (L,R) results in the receiver playing (u,d), so a type 1 sender gets a payoff of 1 and a type 2 sender gets a payoff of 1. Should a type 1 sender deviate? By deviating to R, his payoff becomes 0, so he doesn’t want to deviate. Should a type 2 sender deviate? By deviating to L, his payoff becomes 2, so he does want to deviate. So there does not exist a PBE involving the sender separating on (L,R).

38. Signaling Games Separating on (R,L). This means that a type 1 sender plays R and a type 2 sender player L. Given our definition of a PBE, beliefs are determined by equilibrium strategies. Since both information sets of the receiver are ON the equilibrium path, the only beliefs that are consistent with (R,L) is p = 0 and q = 1. I.e., if only type 1 senders play R, then if a receiver sees R, he should believe that it came from a type 1 sender with probability 1. If only type 2 senders play L, then if a receiver sees L, he should believe that it came from a type 2 sender with probability 1. So, the receiver’s optimal strategy is (u,u). Now lets see if the sender really wants to player (R,L). Playing (R,L) results in the receiver playing (u,u), so a type 1 sender gets a payoff of 2 and a type 2 sender gets a payoff of 2. Should a type 1 sender deviate? By deviating to L, his payoff becomes 1, so he doesn’t want to deviate. Should a type 2 sender deviate? By deviating to R, his payoff becomes 1, so he doesn’t want to deviate. So there exists a PBE at {(R,L), (u,u), (p,1-p), (q,1-q) | p = 0, q = 1 }

39. Signaling Games In general, • Pooling on (L,L)  p= the prior • Pooling on (R,R)  q= the prior • Separating on (L,R)  p=1, q=0 • Separating on (R,L)  q=1, p=0

40. The Market for Lemons • Akerlof models used car market. • Buyers value good cars at bG and lemons at bL. • Sellers value good cars at sG and lemons at sL. • Sellers know the true quality of the car and offer a price, P. • Buyers only know a certain proportion of cars, q, are good and a proportion, 1-q, are lemons.

41. The Market for Lemons • Safe assumption: bG>bL and sG > sL. • Assume that under perfect information, trade would take place for all cars: • bL > sL • bG > sG • Under imperfect information, buyers buy if E[V] = q*bG+(1-q)*bL P • In a separating equilibrium, owners of good cars do not sell their cars and owners of lemons sell their cars. This happens if: • bL = P < sG • this is easily satisfied (see homework 5) • How about a pooling equilibrium where all cars are sold at the price P. • Buyers Require: P  q*bG+(1-q)*bL • Sellers Require: P  sG • So: sGq*bG+(1-q)*bL • this is hard to satisfy (see homework 5) especially if q is small. • For reasonable valuations and probabilities, a pooling equilibrium will not exist. bG sG bL If q is small, q*bG+(1-q)*bL might be here sL

42. Card Game • 2 Players. Each player antes $1 into a pot. • Player 1 picks a card from a deck that is either High or Low with equal probability. Player 2 does not observe the card. • Player 1 can then See or Raise. If he Sees and the card is High, player 1 wins the pot for a payoff of (+1,-1). If he Sees and the card is Low, player 2 wins the pot for a payoff of (-1,+1). • If player 1 Raises, then player 2 can either Pass or Meet. Again player 2 does not know if the card is High or Low. • If player 2 Passes, player 1 wins the pot for a total payoff of (1,-1) no matter if the card is High or Low. • If player 2 Meets, then if the card is High, player 1 wins the pot with a total payoff of (2,-2) and if the card is Low, player 2 wins the pot with a total payoff of (-2,2). • What is the Nash Equilibrium (not PBE) of this game?

43. Card Game Nature High Low • Extensive Form Game: 1 See See +1,-1 1 -1,+1 Raise Raise 2 Pass Meet Meet Pass +1,-1 +1,-1 +2,-2 -2,+2 • Player 1 has two information sets. Player 2 has one. • Strategies of player 2: (Pass or Meet) • Strategies of player 1: (Raise,Raise),(Raise,See),(See,Raise),(See,See)  i.e., what player 1 does if the card is High and what player 1 does if the card is Low.

44. Card Game Strategic Form:

45. Card Game Strategic Form: • So no pure strategy NE. • Mixed? Notice (S,S) is strictly dominated by for player 1 by an equal mixture of (R,R) and (R,S).

46. Card Game Strategic Form: • Note that if q=1  player 1 would mix on p1 and p3 with p2= 0  q=0  contradiction so q<1. • Note that if q=0  player 1 would only mix on p2 with p1=p3=0  q=1  contradiction so q>0. • So player 2 is mixing with positive weight on both pass and meet. • Also player 1 can’t be playing a pure strategy, because player 2 would want to respond with a pure strategy, but we already know that there are no PSNEs.

47. Card Game Strategic Form: • Could player 1 be mixing on all 3 remaining strategies? We require: q=1/2(1-q)  q=1/2 and ½ =? 1/2-1/2(1-1/2)  ½ 1/4. So this cannot be a NE. • So player 1 must be mixing on 2 strategies. Given the conditional expected payoffs, E[R,R] can never equal E[S,R]. E[R,S] = R[S,R] if q = 1/2 , but E[R,R] = ½ > E[R,S] = E[S,R]. So (S,R) is never played with positive probability. Player 1 must be mixing on (R,R) and (R,S).

48. Card Game Strategic Form: • E[R,R] = E[R,S]  q=1/2(1-q)  q=1/3. • E[Pass] = E[Meet]  -p = -1/2(1-p)  ½=3/2p p=1/3. • So the unique NE of this game is {(1/3,2/3,0,0),(1/3,2/3)} Player 1 “bluffs” with probability 1/3.

49. Why PBE Still Isn’t Enough… I R L M II II [p] [1-p] 1,1 X Y X Y 2,-1 -4,-2 0,-1 -1,-2

50. Why PBE Still Isn’t Enough… I R L M II II [p] [1-p] 1,1 X Y X Y 2,-1 -4,-2 0,-1 -1,-2