Extensive-form games

1. Extensive-form games

2. Extensive-form games with perfect information • Players do not move simultaneously • When moving, each player is aware of all the previous moves (perfect information) • A (pure) strategy for player i is a mapping from player i’s nodes to actions (when at X do s1) Player 1 Player 2 Player 2 Player 1 2, 4 5, 3 3, 2 Leaves of the tree show player 1’s utility first, then player 2’s utility 1, 0 0, 5

3. Backward induction • When we know what will happen at each of a node’s children, we can decide the best action for the player who is moving at that node • Substitute the result of the computation at the parent node. Player 1 Player 2 Player 2 Player 1 1,0 3, 2 2, 4 5, 3 1, 0 0, 5

4. What happens with backwards induction here?

5. A limitation of backward induction • Tree may be too long to evaluate. • If there are ties, then how they are broken affects what happens higher up in the tree • Colors show multiple possible strategies. • Multiple equilibria… Player 1 .12345 .87655 Player 2 Player 2 1/2 1/2 3, 2 2, 3 4, 1 0, 1

6. Conversion from extensive to normal form player 1 is row player LR = Left if 1 moves Left, Right if 1 moves Right; etc. Player 1 LL LR RL RR Left Right Left Player 2 Player 2 Right L L R R • Nash equilibria of this normal-form game include (Right, LL), (Right, RL), (Left, RR) + infinitely many mixed-strategy equilibria • In general, normal form can have exponentially many strategies 3, 2 2, 3 4, 1 0, 1 Underscores mean, given you know what the other person will do, what will you do?

7. Conversion from extensive to normal form player 1 is row player for player 2 LR = Left if 1 moves Left, Right if 1 moves Right; etc. Player 1 LL LR RL RR Left Player 2 Player 2 Right One concern, why is column player playing LL? It doesn’t appear to be a smart strategy. Will player 1 believe him or is he claiming that strategy to get an advantage? 3, 2 2, 3 4, 1 0, 1 Underscores mean, given you know what the other person will do, what will you do?

8. Theorem 5.1.3 (SLB) Every finite perfect-information game in extensive form has a pure-strategy equilibrium. Why?

9. Converting another game to normal form Player 1 Do at your seats… Player 2 Player 2 Player 1 2, 4 5, 3 3, 2 1, 0 0, 1

10. Converting another game to normal form LL LR RL RR Player 1 LL LR RL Player 2 Player 2 RR • Pure-strategy Nash equilibria of this game are (LL, LR), (LR, LR), (RL, LL), (RR, LL) • But the only backward induction solution is (RL, LL) • Why? Player 1 2, 4 5, 3 3, 2 1, 0 0, 1

11. Subgame perfect equilibrium • Each node in a (perfect-information) game tree, together with the remainder of the game after that node is reached, is called a subgame • A strategy profile is a subgame perfect equilibrium if it is an equilibrium for every subgame LL LR RL RR LL Player 1 LR RL RR Player 2 Player 2 *L *R ** *L *L *R *R Player 1 • (RR, LL) and (LR, LR) are not subgame perfect equilibria because (*R, **) is not an equilibrium as player 1 shouldn’t play R at second node. • (LL, LR) is not subgame perfect because (*L, *R) is not an equilibrium • *R for player 2 is not a credible threat 2, 4 5, 3 3, 2 1, 0 0, 1

12. Imperfect information • Dotted lines indicate that a player cannot distinguish between two (or more) states • A set of states that are connected by dotted lines is called an information set. We consider moves from anyplace in the set as a unit • Reflected in the normal-form representation Player 1 At seats, what is normal form? Show both extensive form and normal form as want to show comparisons between methods Player 2 Player 2 0, 0 -1, 1 1, -1 -5, -5 • Any normal-form game can be transformed into an imperfect-information extensive-form game this way

13. Imperfect information • Dotted lines indicate that a player cannot distinguish between two (or more) states • A set of states that are connected by dotted lines is called an information set • Reflected in the normal-form representation Player 1 L R L R Player 2 Player 2 Note at each node of player 2, you must make the same choice. Why? 0, 0 -1, 1 1, -1 -5, -5 • Any normal-form game can be transformed into an imperfect-information extensive-form game this way

14. A poker-like gamePractice turning the game tree into the normal form. It is a bit tricky “nature” 2/3 1/3 1 gets King 1 gets Jack cc cf fc ff player 1 player 1 1/3 bb bet stay bet stay bs 2/3 player 2 player 2 sb call fold call fold call fold call fold ss 2 1 1 1 -2 1 -1 1 Remove dominated strategies Do mixed strategy on the others. Reward for player 1 is shown Zero sum game

15. Computing mixed NE Column player selects q so that row player has no preference between row1 or row 2. row 1 gets 0q + 1(1-q) row 2 gets .5q + 0(1-q) to have no preference, set them equal q = 2/3 Solving for p is similar. Notice that in mixed strategies, the whole strategy is given a probability. In behavioral strategies, each node can have a different mix.

16. Subgame perfection and imperfect information • How should we extend the notion of subgame perfection to games of imperfect information? Player 1 • We cannot expect Player 2 to play Right after Player 1 plays Left, and Left after Player 1 plays Right, because of the information set, player 2 doesn’t know which state he is in. Player 2 Player 2 1, -1 -1, 1 -1, 1 1, -1 Let us say that a subtree is a subgame only if there are no information sets that connect the subtree to parts outside the subtree

17. Subgame perfection and imperfect information… Player 1 Player 2 Player 2 Player 2 4, 1 0, 0 5, 1 1, 0 3, 2 2, 3 • One of the Nash equilibria is: (R, RR) • Also subgame perfect (the only subgames are the whole game, and the subgame after Player 1 moves Right) • But it is not reasonable to believe that Player 2 will move Right after Player 1 moves Left/Middle (not a credible threat) • There exist more sophisticated refinements of Nash equilibrium that rule out such behavior

18. Computing equilibria in the extensive form • Can just use normal-form representation • Misses issues of subgame perfection, etc. • Another problem: there are exponentially many pure strategies, so normal form is exponentially larger • Even given polynomial-time algorithms for normal form, time would still be exponential in the size of the extensive form as you have to build the table first. • There are other techniques that reason directly over the extensive form and scale much better • E.g., using the sequence form of the game

19. Commitment • Consider the following (normal-form) game: • How should this game be played? • Now suppose the game is played as follows: • Player 1 commits to playing one of the rows, • Player 2 observes the commitment and then chooses a column • What is the optimal strategy for player 1? • What if 1 can commit to a mixed strategy?

20. Commitment as an extensive-form game • For the case of committing to a pure strategy: Player 1 Up Down Player 2 Player 2 Left Right Left Right 2, 1 4, 0 1, 0 3, 1

21. Commitment as an extensive-form game • For the case of committing to a mixed strategy: Player 1 (1,0) (=Up) (0,1) (=Down) (.5,.5) … … Player 2 Left Right Left Right Left Right 3, 1 2, 1 4, 0 1.5, .5 3.5, .5 1, 0 • Infinite-size game; computationally impractical to reason with the extensive form here

22. Solving for the optimal mixed strategy to commit to[Conitzer & Sandholm 2006; see also: von Stengel & Zamir 2004, Letchford, Conitzer, Munagala 2009] • For every column t separately, we will solve separately for the best mixed row strategy (defined by ps) that induces player 2 to play t • maximize Σs ps u1(s, t) • subject to • for any t’, Σs ps u2(s, t) ≥Σs ps u2(s, t’) • Σs ps = 1 • (May be infeasible, e.g., if t is strictly dominated) • Pick the t that is best for player 1

23. Visualization (0,1,0) = M C Shows what happens to utility as probability of each choice changes. A 3D version of the best response graph. R L (1,0,0) = U (0,0,1) = D