Chapter 4 Sequential Games

1. Chapter 4 Sequential Games

2. T H H T T H (4,0) (1,2) (2,1) (2,1) Extensive Form Games Any finite game of perfect information has a pure strategy Nash equilibrium. It can be found by backward induction. Chess is a finite game of perfect information. Therefore it is a “trivial” game from a game theoretic point of view.

3. Extensive Form Games - Intro • A game can have complex temporal structure. • Information • set of players • who moves when and under what circumstances • what actions are available when called upon to move • what is known when called upon to move • what payoffs each player receives • Foundation is a game tree.

4. Big Monkey and Little Monkey eat warifruit, which dangle from the extreme tip of a lofty branch of the waritree. • A waritree produces only one fruit. To get the warifruit, at least one of the monkeys must climb the tree and shake the branch bearing the fruit until the fruit comes loose and falls to the ground. • A warifruit is worth 10 calories of energy. Climbing the tree uses 2 calories for Big Monkey, but uses no energy for Little Monkey, who is smaller. If Little Monkey climbs the tree and shakes it down, Big Monkey will eat 90% of the fruit (or 9 calories) before Little Monkey climbs back down, and Little Monkey will get only 10% of the fruit (or 1 calorie). • If Big Monkey climbs the tree and Little Monkey waits, Little Monkey will get 40% of the fruit and Big Monkey will get 60%. If both monkeys climb the tree, Big Monkey will get 70% of the fruit and Little Monkey will get 30%. Assume each monkey is simply interested in maximizing his caloric intake. • Each monkey can decide to climb the tree or wait at the bottom. • a. What is likely to happen if Big Monkey makes his decision first? • b. What is likely to happen if Little Monkey must decide first? • c. What if they both decide simultaneously?

5. Fundamental Tools • Big Monkey (BM) – Little Monkey (LM) • Warifruit from waritree (only one per tree) = 10 Calories • Climb the tree to get the fruit • Cost to get the fruit : • 2 Calories for Big Monkey • zero for Little Monkey • Payoff : • Both climb : BM 7 Calories – LM 3 Calories • BM climbs : BM 6 Calories – LM 4 Calories • LM climbs : BM 9 Calories – LM 1 Calories • What will they do to maximize payoff taking into account cost?

6. Fundamental Tools Extensive form games--Definition • An extensive form game G consists of : • Players • Game tree • Payoffs • Terminal node t : i(t)   (payoffs) • G has tree property : only 1 path from root to any terminal node • Occurrence of stochastic event : fictitious player Nature probability assigned to each branch of which Nature is head node

7. Fundamental Tools Extensive form games—Illustration (BM-LM) • 3 possibilities : • BM decides first what to do • LM decides first what to do • Both decide simultaneously • BM decides first : Big Monkey w c Little Monkey Little Monkey w c w c 0,0 9,1 4,4 5,3

8. Fundamental Tools Extensive form games—Illustration (BM-LM) • Strategies : • BM : • Wait (w) • Climb (c) • LM : Actions are ordered, depending on (w,c) of BM • Climb no matter what BM does (cc) • Wait no matter what BM does (ww) • Do the same thing BM does (wc) • Do the opposite of what BM does (cw) • A series of actions that fully define the behavior of a player = strategy. • A strategy for a player is a complete plan of how to plan the game and prescribes his choices at every information set (in this case, node).

9. Fundamental Tools Extensive form games—Illustration (BM-LM) • LM decides first : • The strategies are conversed Utility=(LM, BM) Little Monkey w c Big Monkey Big Monkey w c w c 0,0 4,4 1,9 3,5

10. Fundamental Tools Extensive form games—Illustration (BM-LM) • They choose simultaneously : • Information Set : a set of nodes at which : • The same player chooses • The player choosing does not know which node represents the actual choice node – represented by dotted line LM Big Monkey w c BM Little Monkey w c w c 0,0 9,1 4,4 5,3

11. The key to representing information in a game tree is realizing the connection between nodes and history. • If you know which node you have reached, you know precisely the history of the play. • To express uncertainty, we use concept of information set (set of nodes you could be in at a given time).

12. Composition of information sets • Each decision node is in exactly one information set • all nodes of an information set must belong to same player • every node of an information set must have exactly the same set of available actions. • If every information set of every player is a singleton, we have a game of perfect information.

13. Fundamental Tools Normal form games--Definition The n-player normal form game consists of : • Players i = 1,…,n • A set Si of strategies for player i = 1,…,n. We call s = (s1, …, sn) where si Si for i = 1,…,n, a strategy profile for the game. Each si is a strategy for player i. • A functioni : S   for player i = 1,…,n, where S is the set of strategy profiles, so i(s) is player i’s payoff when strategy profile s is chosen.

14. Fundamental Tools Normal form games--Illustration • Another way to depict the BM-LM game (where BM chooses first) : LM : Actions are ordered, depending on (w,c) of BM • Climb no matter what BM does (cc) • Wait no matter what BM does (ww) • Do the same thing BM does (wc) • Do the opposite of what BM does (cw) LM BM

15. Fundamental Tools Normal form games--Illustration • Don’t get rid of weakly dominated, as lose equilibrium LM BM

16. Sequential games • If players take turns to move then we have a sequential game (sometimes called a dynamic game) • We model a sequential game by using a ‘game tree’ (or an ‘extensive form representation’)

17. It can be shown that every strategic form game can be represented by an extensive game form game and vice versa • But strategies that are in equilibrium in strategic form games are not necessary equilibrium strategies in extensive form games games. Ex. Monopolist. Made sense to fight as an equilibrium, not not if other has already entered. • – We need to define the concept of an equilibrium in extensive form games

18. Problems with Nash equilibrium • Sequential nature of the game is lost when representing extensive form games in strategic form • Some Nash equilibria rely on playing actions that are not rational once that action node has been reached. In other words, a choice only makes sense if you know what the opponent will do. • Nash equilibrium does not distinguish between credible and non-credible threats

19. Solving sequential games • To solve a sequential game we look for the ‘subgame perfect Nash equilibrium’ • For our purposes, this means we solve the game using ‘rollback’ • To use rollback, start at the end of each branch and work backwards, eliminating all but the optimal choice for the relevant player (Technical point – you can only use this trick if there are no information sets. If you don’t know where you are, it may be too difficult to decide.)

21. Subgame • Its game tree is a branch of the original game tree • The information sets in the branch coincide with the information sets of the original game and cannot include nodes that are outside the branch. • The payoff vectors are the same as in the original game.

22. Subgame perfect equilibrium & credible threats • Proper subgame = subtree (of the game tree) whose root is alone in its information set • Subgame perfect equilibrium • Strategy profile that is in Nash equilibrium in every proper subgame (including the root), whether or not that subgame is reached along the equilibrium path of play

23. On October 22, 1962, after reviewing newly acquired intelligence, President John F. Kennedy informed the world that the Soviet Union was building secret missile bases in Cuba, a mere 90 miles off the shores of Florida. • After weighing such options as an armed invasion of Cuba and air strikes against the missiles, Kennedy decided on a less dangerous response. • In addition to demanding that Russian Premier Nikita S. Khrushchev remove all the missile bases and their deadly contents, Kennedy ordered a naval quarantine (blockade) of Cuba in order to prevent Russian ships from bringing additional missiles and construction materials to the island. • In response to the American naval blockade, Premier Khrushchev authorized his Soviet field commanders in Cuba to launch their tactical nuclear weapons if invaded by U.S. forces. • Deadlocked in this manner, the two leaders of the world's greatest nuclear superpowers stared each other down for seven days - until Khrushchev blinked. On October 28, thinking better of prolonging his challenge to the United States, the Russian Premier conceded to President Kennedy's demands by ordering all Soviet supply ships away from Cuban waters and agreeing to remove the missiles from Cuba's mainland. After several days of teetering on the brink of nuclear holocaust, the world breathed a sigh of relief.

24. Example: Cuban Missile Crisis - 100, - 100 Nuke Kennedy Arm Khrushchev Fold 10, -10 -1, 1 Retract Pure strategy Nash equilibria: (Arm, Fold) and (Retract, Nuke) Pure strategy subgame perfect equilibria: (Arm, Fold) Conclusion: Kennedy’s Nuke threat was not credible.

25. Backwards induction • Start from the smallest subgames containing the terminal nodes of the game tree • Determine the action that a rational player would choose at that action node • At action nodes immediately adjacent to terminal nodes, the player should maximize the utility, This is because she no longer cares about strategic interactions. Regardless of how she moves, nobody else can affect the payoff of the game. Replace the subgame with the payoffs corresponding to the terminal node that would be reached if that action were played • Repeat until there are no action nodes left

26. (MD,BK) payoff

27. The predation game • Nasty Guys is an incumbent firm producing bricks • SIC (Sweet Innocent Corporation) is a potential new entrant in the brick market. • Nasty Guys says that if SIC enters then it will “squish them like a bug”. • What should SIC do?

28. The predation game Don’t enter SIC=0, NG=100 Fight SIC SIC = -10, NG = -10 NG Enter Don’t fight SIC = 30, NG = 30

29. The predation game Don’t enter SIC=0, NG=100 Fight SIC SIC = -10, NG = -10 NG Enter If SIC actually enters, then ‘fighting’ is an incredible threat – it hurts SIC but also hurts NG. So SIC knows the threat is just bluff Don’t fight SIC = 30, NG = 30

30. The predation game Don’t enter SIC=0, NG=100 Fight SIC SIC = -10, NG = -10 NG Enter So the equilibrium is: SIC will enter NG will not fight Don’t fight SIC = 30, NG = 30

31. Credible commitments • When Cortes arrived in Mexico he ordered that his ships should be burnt • This seems silly • His troops were vastly outnumbered • Surely it is better to keep an ‘escape route’ home?

32. Think of Cortes trying to motivate his own soldiers Fight Hard C = 100, S = 0 Keep Ships S Be careful C = 0, S = 10 Fight Hard C = 100, S = 0 C S Burn ships Be careful C = -100, S = -100

33. If no retreat possible, will fight hard or die. But if retreat is possible, may fight less hard and ‘run away’ Fight Hard C = 100, S = 0 Keep Ships S Be careful C = 0, S = 10 Fight Hard C = 100, S = 0 C S Burn ships Be careful C = -100, S = -100

34. So Cortes wants to burn his ships. It is a credible commitment not to retreat – and this alters how his own troops behave. Fight Hard C = 100, S = 0 Keep Ships S Be careful C = 0, S = 10 Fight Hard C = 100, S = 0 C S Burn ships Be careful C = -100, S = -100

35. Hold up • Hold up occurs if one party has to incur sunk costs before they bargain with another party • For example, hardware manufacturers and software developers • Hardware manufacturers want software manufacturers to make applications for their hardware • But most of the cost of software is sunk • So if bargain after the software is designed, the hardware manufacturer can seize most of the benefits

36. Holdup: in equilibrium, no-one designs softwarepayoffs = (software, nintendo) Bargain hard (= Pay low price) (-$50,000; $250,000) Nintendo design Software Designer Bargain “soft” ($100,000; $100,000) Don’t design (0, 0)

37. Strategies in extensive form A strategy in an extensive form game is a complete description of the actions that player performs at any action node at which it is her turn to move turn to move Key points –It is not sufficient to specify responses only at those action nodes that are arrived at via some particular sequence of plausible play –A strategy must prescribe an action at any action node where that player moves. node where that player

38. Definition: The strategy set of agent i is the cartesian product of the sets of children nodes of each information set belonging to i. Definition: An information setI is a subset of the nodes in a game tree belonging to player P such that - All iÎI belong to P - For i,jÎI there is no path from i to j - All iÎI have the same number of outgoing edges

39. Sequential Prisoner’s Dilemmadotted line means P2 doesn’t know which state he is in P1 Confess Deny P2 P2 Confess Deny Confess Deny (-5,-5) (0,-10) (-10,0) (-1,-1)

40. With perfect information – each information set is a singleton (as you always know which state you are in) • A strategy profile (s1,s2,…sn) determines a unique path from the root to some terminal node. (where s1 states what player 1 will do in every situation) • We say this unique path is supported by the strategy profile. A path supported by a Nash equilibrium will be called an equilibrium path. • A Nash equilbrium in sequential game (perfect or imperfect): • U(si*, s-i*) >U(si, s-i*) for all i. • Note, there can be two different strategy profiles which have the same path. • Every path from the root to a terminal node is supported by at least one strategy profile.

41. Example 4.9 RL” is best path Stategies ({R}, {R’,L”}) and ({R}, {L’,L”}) both support it ({R}, {R’,L”}) means P1 always takes R, P2 takes R’ if at node B and L” if at node C [Note: Notation is confusing; you always have to read to get meaning.] P1 A L R P2 P2 B C R’ L’ L” R” D E F G (2,1) (0,3) (4,1) (1,0)

42. Theorem (Kuhn): Every sequential game with perfect information has a Nash equilibrium (use backwards induction). P1 A P2 P2 B C D E F (1,0) (0,1) (2,2)

43. Example 4.12 Stackelberg Duopoly Model Stackelberg duopoly (like a monopoly, but with exactly two players) corresponds to a sequential game, first leader (firm 1) chooses how much to produce, then follower (firm 2) chooses can be solved by backward induction: for each quantity q1, the follower chooses its best response q2 i (q1, q2) = qi[p(q) -ci] where q = q1+q2 p(q) = A-q is the market clearing price when the total output in the market is q ci is the marginal cost of the production of the product by firm i. That is, the profit for each firm is i (q1, q2) = qi[A-q1-q2 -ci]

44. Solving by backwards induction • This is a two person game sequential game with two stages and perfect information. • Find best response for each choice of q1 2 (q1, q2*) = max q2[A-q1-q2 –c2] 2 (q1, q2) = -(q2)2 + q2[A-q1 –c2] = -2q2 +A-q1-c2 Second derivative = -2 So the maximizer is (A-q1-c2)/2

45. Continuing • Thus, firm 1 should anticipate this result and choose q1 to maximize 1(q1, q2*) = q1[A-q1-(A-q1-c2)/2 –c1] • = ½(-q12 + (A+c2-2c1)q1) • = -q1 +1/2(A+c2-2c1) • q1= ½((A+c2-2c1) and q2 = ¼*(A+2c1-3c2)

46. Type of games

47. subgame perfect equilibrium • A strategy profile of a sequential game is a subgame perfect equilibrium if it is a Nash equilbrium for every subgame of the original game. In other words, the strategy is perfect even if the play never goes to that part of the tree. • An imperfect equilibrium is like a strategy that wouldn’t be optimal if the other player did something different.

48. Imperfect information • 􀂄􀂄 SPE is not an appropriate equilibrium concept because most games with imperfect information have too few proper subgames to rule out extraneous Nash equilibria of the game • Alternative equilibrium concepts • – Bayesian Equilibrium/Perfect Bayesian Equilibrium • – Sequentially rational equilibrium • – Forwards induction • – Trembling hand equilibrium Topic of active research

49. A Nash equilibrium that fails to be subgame perfect is also known as Nash equilibrium supported by noncredible behavior. • To find subgame perfect equilibrium, use backward induction on the subgames of the original problem.

50. Bob and Betty • Bob and Betty must cook, wash dishes, and vacuum. Bob can't cook very well and just doesn't like to wash dishes, so they have concocted the following game for allocating the tasks. Betty moves first and she chooses between cooking and doing the dishes. • If she chooses dishes, then Bob chooses to Go Out or Cook. • On the other hand, if Betty chooses to cook, then they simultaneously choose between the remaining two tasks; vacuuming and doing the dishes. The payoffs are at the end of the tree. • You may conclude whatever you want about the relationships between the payoffs and the preferences of Bob and Betty for doing chores and being together.