Indirect Reciprocity in the Selective Play Environment

1. 08/07/2003 Indirect Reciprocity in the Selective Play Environment Nobuyuki Takahashi and Rie Mashima Department of Behavioral Science Hokkaido University

2. How can we account for altruism? Altruism = Giving resources (helping others, unilateral cooperation) at a cost of oneself • Kin selection altruism among genetically related individuals • Reciprocal altruism (TFT strategy)  altruism among 2 players in repeated interactions Previously proposed answers How can we account for altruism among unrelated individuals without repeated interaction?

3. Indirect reciprocity is the answer! • If A gives to B, B does not give back to A. Instead, C, a third party, may give to A. • Equivalent to pure-generalized exchange in sociology (Takahashi, 2000) Can we think of a strategy that is equivalent to TFT in direct reciprocity?

4. Review of previous research Giving game and the Image Scoring Strategy(Nowak and Sigmund, 1998a, b) Framework of giving game • A pair of donor and recipient is randomly chosen from a population. • A donor decides whether to give his resources to his recipient with a cost of c (recipient receives the benefit b: b>c).  In this game, of course a rational donor should never give his resources to a recipient.

5. Errors • Errors in strategy execution(occurs with a probability έ) • performing an action different from the one prescribed by the strategy. • 2) Errors in perception (occurs with a probability δ) • misperceiving an action performed by another individual Framework of Giving Game (continued) Adding a score Score • Each player has a reputation score swhich has two values: “good” or “bad”. • A donor gives if he considers the recipient’s score “good”. • A donor doesn’t give if he considers the recipient’s score “bad”. How to assign a score is regulated by a strategy.

6. Solution 1: Image Scoring Strategy(Nowak and Sigmund, 1998a, b) • Evolutionary computer simulation and mathematical analysis • Image Scoring Strategy makes indirect reciprocity possible. • Image Scoring Strategy is a TFT-like strategy: It assigns “good” to previous givers, whereas it assigns “bad” to previous non-givers.  It gives only to givers.

7. The expected payoff: All-C > Image Scoring ⇒All-C increases, and All-D can invade. Problem of Image Scoring(Leimar & Hammerstein, 2001; Panchanathan & Boyd, 2003) When matched with a “bad” recipient who did not give… ◇Image scoringdoesn’t give to the recipient. →becomes being perceived as “bad” by another image scoring. →loses chances to be given by others in the future. ◇ All-C always gives to a recipient. →keeps his “good” score and chances to be given by others.

8. Solution 2: The Standing Strategy(L & H, 2001; P & B, 2003) Standing strategy uses the2nd order information. ◇Standing assigns “good” to previous givers. ◇If the recipient did not give to the previous recipient, standing checks whether or not it was justified as a punishment toward a “bad” recipientby using the information of the score of the recipient’s previous recipient. ◇If it is a justifiable not giving, the standing assigns “good”, whereas if it is an unjustifiable not giving, it assigns “bad”.

9. 1st and 2nd order information (1)The recipient’s previous behavior (2)The score of the recipient’s previous recipient (2) (1) gave good or or bad did not give Current recipient Current donor Current recipient’s previous recipient

10. justified defection as a punishment unjustified defection Representation of the Standing Strategy Standing distinguishes between justified and unjustified not-giving, so that standing can punish non-giverswithout losing his “good” score. Current recipient’s previous recipient’s score Standing = GGBG

11. Problem of the Standing Strategy(Mashima and Takahashi, 2003) • If there are no errors in perception, the standing strategy can make indirect reciprocity possible. • The expected payoff: Standing > All-C Standing saves the cost of giving when matched with a “bad” recipient.  All-D cannot invade.

12. Problem of the Standing Strategy(Mashima and Takahashi, 2003) • If there are errors in perception, it is not the case, however. Suppose the current recipient didn’t give to a “good” person.  The current donor considers the current recipient “bad” and does not give. Previous recipient Current recipient Current donor Next donor didn’t give doesn’t give will give good bad good

13.  The next donor considers the current recipient “good.” good Problem of the Standing Strategy(Mashima and Takahashi, 2003) However, suppose that actually the current donor misperceived the current recipient’s behavior: the current recipient actually gave to a “good” person. Previous recipient Current recipient Current donor Next donor gave good

14.  The next donor will consider the current donor “bad” since he does not give to a “good” person. bad Problem of the Standing Strategy(Mashima and Takahashi, 2003) Nevertheless, the current donor does not give to the current recipient. Previous recipient Current recipient Current donor Next donor gave doesn’t give good good

15.  When there are errors in perception, the expected payoff: All-C > Standing  All-D can invade. Previous recipient Current recipient Current donor Next donor gave doesn’t give will not give good good bad Problem of the Standing Strategy(Mashima and Takahashi, 2003) The next donor will not give to the current donor.

16. Solution 3: Strict Discriminator (GBBB) (Mashima and Takahashi, 2003) • Only an individual who was matched with and gave to a “good” recipient is considered “good” by Strict Discriminator (GBBB). • It sometimes considers All-C “bad” since All-C gives even to All-D.  It drives out not only All-D but also All-C. Image Scoring and Standing always consider All-C “good.”

17. Solution 3 (continued) The results of evolutionary computer simulation showed that…… Strict discriminator could always dominate the population and make indirect reciprocity possible.

18. Remaining issue of Mashima and Takahashi (2003) ---- (1) • The strict discriminator strategy (GBBB) could maintain indirect reciprocity when there can be only 3 strategies: ALL-C, ALL-D, and strict discriminator  What if other strategies are also present?

19. Remaining issue (2) Too strict! The strict discriminator strategy gives only to the recipient who was matched with a “good” recipient and gave to her on the previous round.  He does not give even to his own kind (i.e., the strict discriminator itself) who did not give on the previous round because she was unfortunately matched with a “bad” recipient. Isn’t this realistic as a behavioral pattern of real people?

20. Remaining issue (2) (continued) This unrealistic interpretation of the result is required because the above simulation used the random matching paradigm (i.e., on each round a pair of a donor and a recipient is chosen randomly from a population). However, if a donor knows every other player’s reputation score, why not choose a “good” player as a recipient and avoid being paired with a “bad” player?

21. Selective Play Paradigm • It was proposed in the 90s in social psychology (e.g., Yamagishi and Hayashi 1996, Takahashi 2000). Forced play --Each player has to interact with the designated partner. (e.g., repeated PD, random matching giving game) VS. Selective play --Each player has a choice of selecting a desirable partner.

22. Potential advantage of Selective Play • Remaining issue (2)  The unrealistic nature ofGBBB may be resolved since the situation in which the 4th gene matters occurs rarely.

23. Potential advantage of Selective Play • It is easier for conditionally altruistic strategies (e.g., image scoring, standing, strict discriminator) to target their giving to givers in the selective play environment.  ”Punishing” non-givers that is sometimes costly since players might lose their “good” score is unnecessary. The emergence of indirect reciprocity may be easier in the selective play environment. 

24. New series of simulations Purpose: To examine the remaining 2 issues shown above. (1) 16 strategies can be present simultaneously. (2) Comparing forced play situation (random matching) with selective play situation (a donor randomly chooses one of the “good” players as his recipient)

25. Strategies in the forced play environment 16 strategies (2222=16) are available. GGGG=All-C BBBB=All-D GGBB=image scoring GGBG=standing GBBB=strict discriminator Current recipient’s previous recipient’s score Current recipient’s previous behavior

26. Strategies in the selective play environment • When a potential recipient gave to someone on the previous round, a donor decides her score as he would do in the random matching environment. • When a potential recipient did not give, if a donor believes that there were “good” players, the recipient is considered to have not given to a “good” recipient. If a donor believes that there was no “good” player, the recipient is considered to have not given to a “bad” recipient.

27. Parameters Evolutionary computer simulation • On each round, Random matching condition – a pair of donor and recipient is chosen randomly Selective playcondition – a donor is chosen randomly and selects a desirable recipient • 1500 rounds per generation (m=1500) • At the end of each generation, Selection and Mutation occur (mutation rate:μ=0.0001). • Group size (n)=300, Errors in perception (έ ) = 0.025, Errors in behavior (δ)=0.025, Benefit/cost ratio = 2, 4, 6, 8, 10.

28. Questions to be asked • Does indirect reciprocity emerge when 16 strategies can be present simultaneously? • Is the emergence of indirect reciprocity easier in the selective play environment? • What is the composition of a population when indirect reciprocity emerges?

29. Result (1) Figure 1. Frequency of maintained indirect reciprocity (mutual cooperation) after 10000 generations

30. Result (2) Figure 2. An example of the history of giving rate

31. Indirect reciprocity is more attainable in the selective play environment • When 16 strategies are possible, although indirect reciprocity failed in most cases in the random matching environment, it was maintained in most cases in the selective play environment.  Which strategy maintained indirect reciprocity in the selective play environment?

32. Result (3) Figure 3. An example of the history of evolution when indirect reciprocity was maintained

33. Result (4) Composition of population • When indirect reciprocity was maintained in the selective play environment, two strategies dominated the population in most cases. • They are GBBB (strict discriminator)and GBBG (extra standing). Table 1. Frequency of domination by each strategy

34. What makes the extra standing strategy win in the selective play environment? • Extra standing (GBBG) gives to a recipient who gave to a “good” player or who did not give to a “bad” player.

35. What makes the extra standing strategy win in the selective play environment? • Extra standing (GBBG) gives to a recipient who gave to a “good” player or who did not give to a “bad” player. • In the random matching environment, it cannot maintain indirect reciprocity because of the 4th gene since it sometimes gives even to All-D. • In the selective play environment, the 4th gene rarely matters. Almost always there are some “good” players in a population.

36. Characteristics of the strategies that maintain indirect reciprocity in the selective play environment • The 4th gene matters little •  • Give to a player who gave to a “good” player • Do not give to a player who did not give to a “good” player or who gave to a “bad” player •  GBBG (extra standing) and GBBB (strictdiscriminator)

37. Discussion (1) The emergence of indirect reciprocity is more attainable in the selective play environment. (2) Two strategies are promising for the emergence of indirect reciprocity in the selective play environment: strict discriminator (GBBB) and extra standing (GBBG).  Their behaviors are almost identical. We do not have to adopt the unnatural interpretation of GBBB in the random matching environment.  Selective play is a promising alternative environment in order to investigate indirect reciprocity.

38. Discussion (continued) (3) The strategies that have been proposed to account for indirect reciprocity (e.g., image scoring, standing, strict discriminator, extra standing) are all similar in the sense that only discriminating altruism can hold (Hardin, 1981).  In order for altruism (indirect reciprocity) to emerge, it must take the form of discriminating altruism that have a mechanism that targets its altruistic acts predominantly toward other discriminating altruists.

40. Still remaining issues • In order for indirect reciprocity to emerge, is the 2nd order information really necessary? • As long as we use this type of “score,” we have a problem of infinite regress, since we cannot define the scores on the first round.  How can we avoid this problem?