Finite Iterated Prisoner’s Dilemma Revisited: Belief Change and End Game Effect

1. Finite Iterated Prisoner’s Dilemma Revisited: Belief Change and End Game Effect Jiawei Li (Michael) & Graham Kendall University of Nottingham

2. Outline • Iterated prisoner’s dilemma (IPD) • Probability vs. uncertainty • A new model of IPD • End game effect • Conclusions Jubilee Campus, University of Nottingham

3. Iterated Prisoner’s Dilemma (IPD) • Prisoner’s Dilemma Two suspects are arrested by the police. They are separated and offered the same deal. If one testifies (defects from the other) for the prosecution against the other and the other remains silent (cooperates with the other), the betrayer goes free and the silent accomplice receives the full 10-year sentence. If both remain silent, both prisoners are sentenced to only six months in jail for a minor charge. If each betrays the other, each receives a five-year sentence. Each prisoner must choose to betray the other or to remain silent.

4. Iterated Prisoner’s Dilemma (IPD) Nash equilibrium: (Defect, Defect)

5. Iterated Prisoner’s Dilemma (IPD) • Finite IPD • n-stages; • n is known; • End game effect; • Backward induction; • Nash equilibrium;

6. Iterated Prisoner’s Dilemma (IPD) • Finite IPD experiments Each of 15 pairs of subjects plays 22-round IPD. Average cooperation rates are 44.2% in known and 55.0% in unknown. (from Andreoni and Miller (1993))

7. Iterated Prisoner’s Dilemma (IPD) • Incomplete information (Kreps et al (1982)) • ‘Tit for Tat (TFT)’ type or ‘Always defect (AllD)’ type; • Assign probability  to TFT type and 1- to AllD type; • Mutual cooperation can be sequential equilibrium; • End game effect;

8. Iterated Prisoner’s Dilemma (IPD) • A new model • Assumption that both players may be either AllD or TFT type; • Repeated game with uncertainty; • Changeable beliefs;

9. Probability vs. uncertainty • Probability: Tossing a coin; 50% -- 50% • Uncertainty: • Tossing a what? • 50% -- 50%?

10. Probability vs. uncertainty • Probability • Tossing a coin repeatedly; • 50% -- 50% • Uncertainty • Tossing the dice repeatedly; • Expected probability may change according to the outcome of past playing.

11. Probability vs. uncertainty • Repeated games with uncertainty • Bayes theorem: P(Head) = 0.5; p(Head|1Head) = 0.683; p(Head|1Tail) = 0.317; p(Head|2Head) = 0.888; p(Head|2Head+1Tail) = 0.565; ......

12. A new model of IPD • Let ROW and COL denote the players in a N-stage IPD game. where R, S, T, and P denote, Reward for mutual cooperation, Sucker’s payoff, Temptation to defect, and Punishment for mutual defection, and

13. A new model of IPD • We refer to COL’s belief (Row’s type) by that denote the probabilities that, if ROW is a TFT player, ROW chooses to cooperate at the 1,…,n stage. Similarly, we define ROW’s belief about COL’s type by that denotes the probabilities that COL chooses to cooperate at each stage if COL is a TFT player. {δi} and {θi} represent the beliefs of player COL and ROW respectively.

14. A new model of IPD Two Assumptions: • A1: • A2: If either player chooses to defect at stage i, there will be

15. A new model of IPD

16. A new model of IPD

17. A new model of IPD

18. A new model of IPD (D, D) is always Nash equilibrium at stage i; but it is not necessarily the only Nash equilibrium. When (2) is satisfied, both (C, C) and (D, D) are Nash equilibrium. (2) is a necessary and sufficient condition.

19. A new model of IPD A sufficient condition for (C,C) to be Nash equilibrium: This condition denotes a depth-one induction, that is, if both players are likely to cooperate at both the current stage and the next stage, it is worth each player choosing to cooperate at the current stage. For example, when T=5, R=3, P=1, and S=0, the condition for (C,C) to be Nash equilibrium is,

20. End game effect • ? • Unexpected hanging paradox. • A condemned prisoner; • Will be hanged on one weekday in the following week; • Will be a surprise;

21. End game effect

22. End game effect P1 = 1 P2 = 1 P3 = 1 P4 = 1 P5 = 1

23. End game effect

24. End game effect • Process of belief change P1 = 1/5 P2 = 1/5 P3 = 1/5 P4 = 1/5 P5 = 1/5 P1 = 0 P2 = 1/4 P3 = 1/4 P4 = 1/4 P5 = 1/4 P1 = 0 P2 = 0 P3 = 1/3 P4 = 1/3 P5 = 1/3 P1 = 0 ... P4 = 1/2 P5 = 1/2 P1 = 0 ... P5 = 1

25. End game effect • Backward induction is not suitable for repeated games with uncertainty because uncertainty can be decreased during the process of games. • End game effect has limited influence on the players’ strategies since it cannot be backward inducted.

26. End game effect • Why does the rate of cooperation in finite IPD experiments decrease as the game goes toward the end?

27. Conclusions • 1. We develop a new model for finite IPD that takes into consideration belief change. • 2. Under the new model, the conditions of mutual cooperation are deduced. The result shows that, if the conditions are satisfied, both mutual cooperation and mutual defection are Nash equilibrium. Otherwise, mutual defection is the unique Nash equilibrium.

28. Conclusions • 3. This model could also deal with indefinite IPD and infinite IPD. • 4. The outcome of this model conforms to experimental results.

29. Conclusions • 5. Backward induction is not suitable for repeated games with uncertainty when the beliefs of the players are changeable. • 6. This model has the potential to apply to other repeated games of incomplete information.

30. Thank you.