Evolution of cooperation in Stackelberg games

1. Evolution of cooperation in Stackelberg games Raimo P. Hämäläinen Ilkka Leppänen Systems Analysis Laboratory Aalto University

2. Main theme • To learn about cooperative behavior in repeated interactions • In a Stackelberg setting, the players are in asymmetric positions. How does this affect cooperation? • How do people act in these games when they have possibility to cheap talk / cheat / second-play? • We study cheap talk with a second-play model

3. Earlier results from repeated settings • Cournot duopoly: random pairs converge to Nash, fixed pairs collude (Holt 1985) • Sequential prisoner’s dilemma: cooperation decreases over repetition (Clark and Sefton 2001) • Market games (auctions): convergence to equilibrium prices (Roth et al. 1991) • Ultimatum games: offers and rejections become lower over time (Roth et al. 1991, Camerer 2003) • Public goods games: cooperation deteriorates and free-riding increases, but cooperation is maintained with punishment (Fehr and Schmidt 1999)

4. Stackelberg game Basic setting: • Leader decides: knows both payoffs and takes into account the best response reaction of the follower • Follower reacts to the leader’s decision Second-play Stackelberg setting (Hämäläinen 1981): • Leader decides and announces his decision to the follower • Follower reacts to the leader’s decision • Leader takes into account follower’s decision and decides again (re-optimizes / cheats)

5. Stackelberg game • Leader has the opportunity to better payoffs because he decides first • Second-play is possible when follower does not have full information about leader’s payoffs • Stackelberg games have not received much attention in the experimental games literature

6. Stackelberg game with second-play In the second-play Stackelberg game, the leader’s announcement should not affect the follower’s choice In theory, the leader is the follower Second-play announcement introduces a possibility of cheap talk and cheating Does this affect cooperation, and is this possibility used? Cheap talk literature: cheap talk signaling helps cooperation in social dilemma situations (Crawford, 1998)

7. Experiment of Huck, Müller, and Normann (2001) • Stackelberg game in a market context • Players acting as quantity-choosing firms • 13 × 13 payoff matrix, pen and paper • 92 subjects • Repeated for 10 rounds, with • fixed pairs where each player meets the same pair repeatedly • random pairs where each player meets a new randomly selected pair every round • Compensated from the payoffs of two randomly selected rounds out of the ten rounds (average DM 15.67)

8. Follower Leader JO = joint optimum (6,6) L = Stackelberg leader (12,6) LS = second-play Stackelberg leader (9,6) F = Stackelberg follower as leader (6,12) N = Cournot-Nash equilibrium (8,8)

9. Our experiment • Same payoff matrix and market context (similar instructions) as Huck et al. (2001) • Computerized: SAL experimental game platform • 210 student subjects from engineering faculties • Repeated for 20 to 24 rounds, players did not know the number of rounds • Compensated from the payoffs of two randomly selected rounds out of ten final rounds (average 6.77 €)

10. 8 different games, only one for each subject (*) Games studied by Huck et al. (2001) Subjects remained in their roles leader/follower for the whole session

11. SAL experimental game platform • Web-based to allow independence of location • Our experiments arranged in a regular computer classroom

12. Evolution of cooperation: Stackelberg fixed pairs F knows payoffs of L F does not know payoffs of L

13. Evolution of cooperation: Stackelberg random pairs F knows payoffs of L F does not know payoffs of L

14. Evolution of cooperation: second-play fixed pairs F knows payoffs of L F does not know payoffs of L

15. Evolution of cooperation: second-play random pairs F knows payoffs of L F does not know payoffs of L

16. Stackelberg games result: gains and losses in cooperation • Reference outcome is the Stackelberg outcome (72,36), payoff difference 36 • Fixed pairs: joint optimum (72,72), leader’s loss 0, follower’s gain 36 • Random pairs: Cournot-Nash (64,64), leader’s loss 8, follower’s gain 28

17. Second-play games result: gains and losses in cooperation Reference outcome is the second-play Stackelberg outcome (81,54), payoff difference 27 Fixed pairs: joint optimum (72,72),leader’s loss 9, follower’s gain 18 Random pairs: Cournot-Nash (64,64), leader’s loss 17, follower’s gain 10 In second-play games, the evolution to cooperation is driven by the leader

18. Comparison to Huck et al. (2001) Frequencies of leader choices over all rounds, Stackelberg games with complete information Joint-optimum Cournot-Nash Stackelberg Fixed pairs Random pairs Explanation for the different leader behavior?

19. Evolution: Stackelberg fixed pairs, F knows payoffs of L Convergence to the joint optimum Vertical axis: number of given outcomes

20. Evolution: Stackelberg fixed pairs, F does not know payoffs of L Convergence to both the joint optimum and the Cournot-Nash Vertical axis: number of given outcomes

21. Evolution: Stackelberg random pairs, F knows payoffs of L Cournot-Nash Vertical axis: number of given outcomes

22. Evolution: Stackelberg random pairs, F does not know payoffs of L Cournot-Nash Vertical axis: number of given outcomes

23. Evolution: Second-play fixed pairs, F knows payoffs of L Joint optimum Vertical axis: number of given outcomes

24. Evolution: Second-play fixed pairs, F does not know payoffs of L Joint optimum Vertical axis: number of given outcomes

25. Evolution: Second-play random pairs, F knows payoffs of L Convergence to equal payoffs Vertical axis: number of given outcomes

26. Evolution: Second-play random pairs, F does not know payoffs of L Convergence to equal payoffs Vertical axis: number of given outcomes

27. Outcomes from last five rounds

28. Questionnaires Leaders drive cooperation • In stackelberg games, 10 out of 11 leaders answered that they tried either to maximize the sum of both payoffs or aim for equal payoffs; in second-play, 8/14 leaders Followers signal for cooperation, but if leaders do not notice it, cooperation does not result • In stackelberg games, 10 out of 11 followers answered that with their choices, they tried to signal the intention to get better payoffs for both; in second-play, 12/14 followers • In pairs that do not converge to the joint optimum, only 1 leader out of 6 has noticed the follower’s signaling; in second-play 2/7 leaders Punishments are used only in pairs which are not cooperating

29. Summary Strong other-regarding behavior:fixed pairs converge to joint optimum, random pairs Cournot-Nash, both result in equal payoffs On average 51% of outcomes have equal payoffs No Stackelberg outcomes Leader drives cooperation in second-play games with cheap talk Leader has no threat of loss of payoffs (as in the ultimatum game) and still chooses cooperative strategy, even in random pairs Frequencies of cooperation: • Our leaders: 35,5% • Huck leaders: 21% • Typical ultimatum game fair offers: 71% (Fehr and Schmidt 1999)

30. Neural correlates to other regarding behavior? Are there differences in the neural areas activated for self regarding and other regarding players? Compare to Sanfey et al. (2003) fMRI observations from receiving unfair ultimatum proposals: • Activation in the bilateral anterior insula, ”emotional goal of resisting unfairness” • Activation in the dorsolateral prefrontal cortex, ”cognitive goal of accumulating money” • Activation in the anterior cingulate cortex, ”motivational conflict between fairness and self-interest”

31. References • Camerer, C. 2003. Behavioral Game Theory, Princeton University Press. • Crawford, V. 1998. A survey of experiments on communication via cheap talk. Journal of Economic Theory, Vol. 78. • Fehr, E. and Schmidt, K.M. 1999. A theory of fairness, competition, and cooperation. Quarterly Journal of Economics, Vol. 114. No. 3. • Holt, C. 1985. An experimental test of the consistent-conjectures hypothesis. The American Economic Review, Vol. 75, No. 3. • Huck, S., Müller, W., and Normann, H-T. 2001. Stackelberg beats Cournot: on collusion and efficiency in experimental markets. The Economic Journal, Vol. 111. • Hämäläinen, R. 1981. On the cheating problem in Stackelberg games. Int J Syst Sci, Vol. 12, No. 6. • Roth, A., Prasnikar, V., Okuno-Fujiwara, M. and Zamir, S. 1991. Bargaining and Market Behavior in Jerusalem, Ljubljana, Pittsburgh, and Tokyo: An Experimental Study. The American Economic Review, Vol. 81, No. 5. • Sanfey, A.G., Rilling, J.K., Aronson, J.A., Nystrom, L.E., Cohen, J.D. 2003. The neural basis of economic decision-making in the ultimatum game. Science, Vol. 300.

32. More Huck and Müller et al. references Huck, S., Müller, W. and Normann, H-T. 2002. To commit of not to commit: Endogenous timing in experimental duopoly markets. Games and Economic Behavior, Vol. 38, No. 2.. Huck, S., and Wallace, B. 2002. Reciprocal strategies and aspiration levels in a Cournot-Stackelberg experiment. Economics Bulletin, Vol. 3., No. 3. Müller, W. 2006. Allowing for two production periods in the Cournot duopoly: experimental evidence. Journal of Economic Behavior & Organization, Vol. 60, No. 1. Müller, W., and Tan, F. 2010. Team versus individual play in a sequential market game. Manuscript, http://www.tilburguniversity.edu/research/institutes-and-research-groups/center/phd_stud/tan/Mueller_Tan.pdf