Acqui Terme, 2 September 2010 Cooperation from a Game Theory perspective

1. Acqui Terme, 2 September 2010Cooperation from a Game Theory perspective Marco Dardi UniversityofFlorence marco.dardi@unifi.it

2. In common language cooperation means mutual assistance in order to get benefits that would be unavailable to agents acting non-cooperatively. GT can be applied to analyzing problems that arise in the effort to reap this cooperative surplus. In the language of GT the terms cooperative/non-cooperative have also a technical meaning connected with the formal specification of solution concepts. This lecture investigates the relationship between the two levels at which GT deals with cooperation: as an object problem and as a formal definition. Acqui T. 2 September 2010

3. 1. GT, conflict, cooperation • In a nutshell: GT elaboratesmethodsfor • describing the structureofinteractivesituations, focusing on the choicesavailabletoindividualagents, on theirknowledge and preferences (theoryofrepresentations) • prescribingwaysofbehaving in interactivesituationsthatcomplywithnotionsofintelligentpursuitofeachagent’s owninterests (theoryofsolutions) • First, representation. The normal or strategicformofso-callednon-cooperativegames (on these, later on) is the mostconvenientrepresentationfor the purposesofthislecture. Itconstitutes a sortof compromise between the extensiveformrepresentation, that can bereducedtonormalform at the costoflosingrelevantdetail, and the coalitionalformrepresentationofso-called cooperative games, thatcan bederivedfrom the normalform, againwith some loss ofdetail. • The normalformrepresentationconsistsof: • a set ofagents or players (in numbern  1) • foreachagent, a set ofstrategies Strategicspace • a set ofoutcomes • foreachagent, preferences on the outcomes • representedby von Neumann utility numbers Acqui T. 2 September 2010

4. Accordingto the normalformrepresentation, a game form G is a mapping G: strategicspace  outcomes A game g is a game form + a specificationof the preferencesof the agentsinvolved. Formally, with a numericalrepresentationofpreferences, a game is a composite mapping g = U(n) • G, with U(n) the n-vectorof the players’ utility functions. Hence, g: strategicspace  n Interpretation: G describes the “physical” rulesthatapply in a situation (distributionof the agents’ powersover the outcomes). Each G generates a classofgames, oneforeachspecificationof the preferencesof the agentsinvolved (thinkof a planoforganization and all the firmsorganizedaccordingtoit; of a legal code, and all the specificsituationsregulatedbyit; etc) Warning: the mainbehavioralfeatureimplicit in GT representationsisnotself-interest or egoism, but the instrumentalityof the choicesmade in playing a game. Players are notinterested in the actionsthey or the otherplayerschoose, but in the outcomesbroughtaboutbytheseactions. The motivationslyingbehindtheirpreferences on the outcomesneedn’t bemadeexplicit. Altruismisasgood a motivationasanyother. Acqui T. 2 September 2010

5. Just looking at the imagesof g in nwegetan idea ofwhatconflict and cooperation look like in a GT framework. Case 1: the vectorinequality (strong Paretopreference) doesnotapply in the rangeof g  all the outcomes are Paretooptima  g describes a pure conflict situation Case 2: The rangeof g iscompletelyorderedby   Pareto and individualpreferences are in agreement  conflictisnon-existent Case 3: The rangeof g isorderedby  butonlypartially  some (more thanone) outcomes are Paretooptima, some are Pareto-dominated  conflict and common interest co-exist In 1 there are no gainsfromcooperation (e.g., zero-sum games). In 2, the gains can bereapedunproblematicallybecauseindividual and common interestsaccordwitheachother (the onlyproblemmaybecoordination). The only case of interest for GT turns out tobe3, whereconflictsof interest mayprevent the playersfrompicking the potentialgainsfromcooperation. Note: the conflict/cooperation mix doesnotdepend on the game formbut on “who” the players are Acqui T. 2 September 2010

6. 2. Cooperationproblems: basicpatterns • Togetan idea, take the simplestpossible case: twoagents, A and B, eachwithtwostrategies, c = follow a cooperative lineofconduct, d = don’t followsuch a line (whateverthismaymean). No player can makehis/herchoiceconditionalupon the other player’s choice (no information leaks). • Game form (outcomes in greekletters) B c d c d •  •   A Two games ( = A’s preference; = B’s preference) B B c d c d c d •  •   c d •  •   A A “PRISONER” “HAWK-DOVE” Acqui T. 2 September 2010

7. In both games we can observe: gains from cooperation, symmetrical incentives to defect, each player prefers the other to cooperate. PRISONER unconditionally prefers not to cooperate; H-D prefers to cooperate if the other doesn’t. Consequently, gains from cooperation are greater in H-D than PRISONER as revealed by Pareto preference B B c d c d c d •  •   c d •  •   A A PRISONER HAWK-DOVE In order to avoid the non-cooperative, Pareto-dominated outcome  some sort of agreement is needed. Any agreement in PRISONER is liable to defection. So is agreement  in H-D, while agreements  and , although secure against defection, may be refused on grounds of justice. The analysis suggests remedies that in all cases, barring the possibility of changing the agents’ preferences, require modifications of the game form… Acqui T. 2 September 2010

8. In PRISONER: expand the game so as to add a post-play stage in which players have a possibility of sanctioning the agreement by means of penalties. The expansion may consist of a number of repetitions of the game, provided no repetition is known with certainty to be the last one. In H-D: introduce correlated randomization (NB: not the independent randomization known as “mixed strategies”) of the outcomes. In some cases this will require changes of the game form through the introduction of an umpire or an external information system (with suitable utility numbers the best fair agreement requires prob() = 0 and  to be drawn with the same probability as  and ). These examples provide a clue to one of the most thriving lines of research in applied GT during the 1980s and 1990s: how to design a game form such that the cooperative gains latent in a situation do not go unexploited (implementation theory (IT) or “mechanism design”). IT provides a new basis for the theory of contracts, industrial organization, imperfect markets and other microeconomic applications. The object of IT is: given the agents’ preferences, re-design the situation as a game such that sticking to the agreement to cooperate in reaping the existing Pareto gains is the only intelligent line of conduct for all the players in the game. A preliminary step is of course clarifying what an intelligent line of conduct in a game is. This leads to the particular theory of solutions on which IT depends… Acqui T. 2 September 2010

9. 3. Stability • In the theory of solutions a solution concept is a rule that selects a subset of strategy profiles out of the strategicspace of the game accordingto some criterionofrationality, • solution  strategicspace • withsolution beingdefinedby some common propertyof the strategyprofilesincluded. • The mainpropertyused in GT solutionconceptsisthatofstability. A strategyprofileisstablewheneverfor each player the following statement holds: “ifnobodyhasanyreasontorefuseto do his/her part in thisprofile, I have no reasoneither”. • A cooperative agreement has a chance ofbeingeffectiveonlyifitprescribesstrategieswhichmake up a profilethatbelongsto a stablesolution. Thus, cooperative agreementsshouldbestablesolutionsofanappropriatelydesigned game. • Stability may mean various things depending on the way “reasons to refuse” in the above statement are specified. Here for the first time we have to deal with the technical distinction in GT between so-called non-cooperative and cooperative solutions. Acqui T. 2 September 2010

10. Fromnow on: cooperative/non-cooperative in the technicalsenseof GT solutionswillbemarked off with a *. A stablesolutionis non-cooperative* if “reasonstorefuse” are referredexclusivelytoindividualplayers. An individual player hasreasontorefuseto do his/her part in a prescribedstrategyprofileif, in the hypothesisthat the others do theirs, he/shehas the powerofbringingabout a preferredoutcome. If no player hassuchreasons, then the prescribedstrategyprofileis a Nash non-cooperative* equilibrium (NE). Hence, a stable non-cooperative* solution coincideswith the set of Nash non-cooperative* equilibria. Note: bybasing cooperative agreements on NE, asisusuallydone in IT, wehave a theoryofcooperation in which the stabilityofagreementsreliesentirely on a non-cooperative* solutionconcept. Far frombeing a paradox, thisis the essenceofthe so-called “Nash program” (Nash 1951) for reducing all cooperative solutions to non-cooperative* equilibrium analysis. But it is to be noted that Von Neumann and Morgenstern refused to take Nash’s cooperative*/non-cooperative* partition into consideration. The statement “the general, typical game – in particular all significant problems of a social exchange economy – cannot be treated without these devices [of cooperation]” remained unchanged in the third edition of their work (1953, p. 44), after they had taken cognizance of Nash’s papers on non-cooperative* games. Acqui T. 2 September 2010

11. Stable cooperative* solutions differ from NE in that “reasons to refuse” may be referred not only to individuals but also to groups of individuals acting cooperatively (coalitions). Refusing is not necessarily an individual affair, individuals may refuse by forming a coalition in order to get a better outcome. A coalition has reason to refuse to do its part in a prescribed strategy profile if, in the hypothesis that the other players do theirs, it has the power of bringing about an outcome which is preferred by all its members. That a strategy profile belongs to NE is a necessary but not a sufficient condition for it to be stable with respect to coalitions. Generally, cooperation in refusing restricts the domain of stability. If all possible coalitions are considered to be equally feasible, the relevant cooperative* solution concept is (from Edgeworth) the CORE. A strategy profile belongs to the CORE if and only if no individual or coalition has reason to refuse to do its part in it on condition that all the others do theirs. Obviously, CORENE. Basing a cooperative agreement upon a cooperative* solution concept turns out to be more difficult than relying on a non-cooperative* solution such as NE. In particular cases the CORE may even be empty (for example, PRISONER has NE = (d,d) but no CORE). Acqui T. 2 September 2010

12. However, not all the conceivable coalitions are generally equally feasible. More sophisticated concepts of cooperative* stability refer to the stability of the coalitions themselves. A strategy profile s, which is liable to be refused by a coalition, may be considered to be stable all the same if the agreement within the potentially refusing coalition can in turn be challenged by some other coalition. Members of the former, knowing that the latter could thwart their plans, could be dissuaded from forming it. Thus, although not in the CORE, s may remain unchallenged. These considerations open the way to a variety of sophisticated cooperative* solution concepts (BARGAINING SET, KERNEL, SHAPLEY VALUE etc.). The specific definitions depend on the way that the notion of challenging coalitions and counter-coalitions is modelled. In general, all these concepts are more permissive than the CORE. Von Neumann & Morgenstern proposed a cooperative* solution concept that, while of a sophisticated kind, lies on a completely different line from those we have considered so far. Their concept of “stable set” (SSET) was defined not on the basis of a common property of the strategy profiles that belong to it, but on the basis of a structural property of the set itself. They insisted on stability in social theories being “a property of the system as a whole and not of the single imputations [here, read “strategy profiles”] of which it is composed” (1953, p. 36). Acqui T. 2 September 2010

13. A set ofstrategyprofilesis a SSETif and onlyif • Foreachprofileincluded in it: if a coalitionhasreasontorefuseit, thismustbe in favourof a profileexcluded; • Foreachprofileexcludedfromit: thereis at least a coalitionthathasreasontorefuseit in favourof a profileincluded • SSETmaybeempty (as in PRISONER). In the same game theremaybe more thanoneSSET (as in the purelyconflictual game knownas “matchingpennies”, withemptyNE and CORE ). If more thanoneSSETexist, thesehave no intersection. And, ofcourse, in generalCORE SSET, whilethereis no generalrelationshipbetweenSSET and NE. • Von Neumann & MorgensterninterpretedSSETas a formalizationof the notionofan “establishedorderof society” or “accepted standard ofbehavior”. Itdescribes a varietyofmodesofbehavior, none ofwhichisabletounsettle the others. Some ofthemmaybeunsettledby some non-conformingmodesofbehavior, butallof the latter are unsettledbyone or anotherof the acceptedones. Lastly, the same game or social situation may express more thanonesuch “order” or standard. However, forallits evocative power, thisnotionhashadlittleapplication in economics and in social theory in general. Acqui T. 2 September 2010

14. 4. Final remarks Cooperative* solution concepts have been little used in economic applications. Perhaps the very variety of concepts available, with the ensuing feeling of ad hoc constructions, has been an obstacle to generalized adoption. Thus, cooperation on a non-cooperative* basis, in the sense explained above, constitutes the unifying methodological framework of great part of contemporary microeconomics (Moulin, 1995: “Cooperation in the economic tradition is mutual assistance between egoists”). The “Nash program” has prevailed over Von Neumann & Morgenstern’s more “social” approach. But note that Moulin’ s reference to “egoism”, as remarked above (slide 4), is anyway wide of the mark, since acting on the basis of individual preferences has no necessarily egoistic implications. Moreover, we should not be induced to view the non-cooperative* approach to cooperation as an expression of an inevitably atomistic social philosophy. All the arguments that try to justify NE as the only solution concept consistent with individual rationality resort to some kind of “communality of thought” that, as Schelling (1960) and Lewis (1969) have pointed out, presupposes that some social convention is in force. It cannot be an exclusively individual affair. Acqui T. 2 September 2010

15. Recall the premise of the conditional statement underlying individual stability (slide 9): “if the others have no reason to refuse to do their part in this profile, I have no reason either”. Doing my part is rational if the premise is true. But why should I believe it to be true? A moment’s reflection shows that the basis of this belief is the belief that it is shared by everybody, that it is believed to be shared by everybody, and so on ad infinitum: briefly, it must be, in a specially strong sense, a “common” belief. Common beliefs presuppose conscience that on some matters there is something like communality of thought, thought that does not need to be communicated. This would seem to be an unlikely phenomenon in a rigorously atomistic society because it implies that individuals do not think independently of each other on all matters, and therefore some “accepted standards” of thought (in Von Neumann & Morgenstern’s language) must be well-established. Individualism itself must be an expression of such a standard. Thus, non-cooperative* foundations of cooperation (as in IT) must in turn be founded on sociological premises lying at a deeper level than individual rationality. Nash program is at best half a program for GT-based social research; the other half would require relating solution concepts to types of social culture. Acqui T. 2 September 2010

16. 5. Selectedreferences For the basicsof GT, IT, coalitions: accordingtomy taste and teachingexperience, the best (althoughby no meanselementary) introductionisOsborne & Rubinstein, A course in GT (1994) MIT Press, chapters 1-5, 10, 13-14. For the game-theoreticoutlook on cooperation in economics: seeMoulin, Cooperative microeconomics (1995) Prentice Hall. Forstability, cooperative/non-cooperativegames, and the Nash program: see von Neumann & Morgenstern, Theoryofgames and economicbehavior (3rdedition, 1953) Princeton UP, chapts. I.4, V, XI; Nash, “Noncooperativegames”, AnnalsofMathematics (1951); Myerson, “Nash equilibrium and the historyofeconomictheory”, JEL (1999). For the foundationsofnon-cooperativeNE: Schelling, The strategyofconflict (1960) Harvard UP; Lewis, Convention: A philosophicalstudy (1969) Blackwell; Bacharach, “A theoryofrationaldecision in games”, Erkenntnis (1987). Forallmattersrelatedto GT: see the relevantchapters in the Handbookof GT witheconomicapplications, Aumann & Harteditors, 3 vols., 1992-1994-2002: North-Holland. Acqui T. 2 September 2010