Cooperative/coalitional game theory

1. Cooperative/coalitional game theory A composite of slides taken from Vincent Conitzer and Giovanni Neglia (Modified by Vicki Allan)

2. Prisoners Dilemma rules • Binding agreements are not possible. Note in Prisoners dilemma, if binding agreements were possible, there would be no dilemma • Utility is given directly to individuals as a result of individual actions. So, I am not worried about collective utility. Solution: cooperative game theory

3. Cooperative Games • Coalitions – set of agents • Grand coalition – all agents work together • Characteristic function: v:2Ag R subsets of agents are assigned a value. • Simple coalitional game: a coalition has value 0 or 1. A voting system can be thought of as a simple game (have enough votes to win or do not)

4. Three Parts of Cooperative Action • Coalition Structure Generation – who should work together. Know only characteristic function. Want to earn a lot, but so does everyone else. • Optimization Problem – given coalitions and tasks, which coalition should do each problem. • Dividing the utility- What is fair? What is stable?

5. In Multi-Agent Systems • Agents create their own coalitions rather than have a centralized decision.

6. Scenario 1 – Bargain Buy(supply-demand) • Store “Bargain Buy” advertises a great price • 300 people show up • 5 in stock • Everyone sees the advertised price, but it just isn’t possible for all to achieve it

7. Scenario 2 – hiring a new PhD(strategy) • Universities ranked 1,2,3 • Students ranked a,b,c Dilemma for second tier university • offer to “a” student • likely rejected • delay for acceptance • “b” students are gone

8. Scenario 3 – selecting a spouse(agency) • Bob knows all the characteristics of the perfect wife • Bob seeks out such a wife • Why would the perfect woman want Bob?

9. Scenario 4 Blowhard (trust) What if one person talks a good story, but his claims of skills are really inflated? He isn’t capable of performing. the task.

10. Scenario 5 Power in diversity You consult local traffic statistics to find a good route home from work But so does everyone else

11. The coalition is completed and rewards are earned. How are they fairly divided among agents with various contributions? If organizer is greedy, why wouldn’t others replace him with a cheaper agent?

12. Consider the Stable Marriage Problem – small coalition • In a small town there are n men and n women who wish to be wed. Each person would be happy to be married to any of the people of the opposite sex but has a definite preference ranking of the possible marriage partners. • If marriages are arranged arbitrarily, some of the marriages can be unstable in the following sense. Suppose Alice marries Bob and Carol marries Dave. If Alice prefers Dave to Bob and Dave prefers Alice to Carol, then Alice and Dave will leave their partners; in this situation, we say that the marriages of Alice and Dave are unstable. • Goal is stability not just the most partners: the difference between local and global utility.

13. Core • Ex: software engineering group project. Who would you choose to work with? Who would also choose you? Coalition doesn’t form unless everyone is happy with it. • Stability is necessary but not sufficient for a coalition to form. If it isn’t stable, someone will defect. If there are multiple stable coalitions, another may form instead. • core: set of feasible distributions of payoff to members of the grand coalition so none will defect. • We require the outcome (payments) to be both feasible (able to pay) and efficient (all utility distributed). • Pareto efficiency is the efficiency referred to.

14. We say a coalition objects to (or blocks) an outcome if every member of the coalition is strictly better off in some other feasible outcome. • Asking if the grand coalition is stable means, “Is the core non-empty?” • The point of the core is to study stability not fairness.

15. Example • v{a} = 1 v{b} = 3 v{a,b} = 5 • How should we divide the profit if they work together? Which are feasible, fair, efficient, stable? • <a=5,b=0> • <a=4,b=1> • <a=3,b=2> • <a=2,b=3> • <a=1,b=4> • <a=1.5,b=3.5> • <a=0,b=5> • <a=2,b=2> • <a=2,b=4>

16. So what are concerns? • What if there are no outcomes in the core? • What if there are multiple outcomes in the core, how do you pick? • Looking at all possible distributions of utility is exponential as you have 2n possible objecting subsets to consider.

17. Example • v{1} = 2 v{2} = 2 v{3} = 2 • v{1,2} = 5 v{2,3} = 5 v{1,2,3} = 6 • How should we divide the profit if they work together? Which are feasible, fair, efficient, stable? • <2,2,2> • <3,3,0> • <1,2,3> • <1,2.5,2.5> • <2,2.5, 1.5>

18. More Examples Emptiness & multiplicity • Example 1: Let us modify the above example so that agents receive no additional utility from being all together (and being alone gives 0) • v({1, 2, 3}) = 6, • v({1, 2}) = v({1, 3}) = v({2, 3}) = 6, • v({1}) = v({2}) = v({3}) = 0

19. More Examples Emptiness & multiplicity • Example 1: Let us modify the above example so that agents receive no additional utility from being all together (and being alone gives 0) • v({1, 2, 3}) = 6, • v({1, 2}) = v({1, 3}) = v({2, 3}) = 6, • v({1}) = v({2}) = v({3}) = 0 • Now the core is empty! Notice, the core must involve the grand coalition (giving payoff for each).

20. More Examples Emptiness & multiplicity • Example 2: • v({1, 2, 3}) = 18, • v({1, 2}) = v({1, 3}) = v({2, 3}) = 10, • v({1}) = v({2}) = v({3}) = 0

21. More Examples Emptiness & multiplicity • Example 2: • v({1, 2, 3}) = 18, • v({1, 2}) = v({1, 3}) = v({2, 3}) = 10, • v({1}) = v({2}) = v({3}) = 0 • Now lots of outcomes are in the core – (6, 6, 6), (5, 5, 8), …

22. Fairness • Could we get people to join a coalition by saying, “Will you agree to a fair solution even though it is not in the core?” • In example v{1} =0 v{2} = 0 v{3} = 0 v{1,2} = 5 v{2,3} = 5 v{1,2,3} = 6 What would a fair but unstable solution be? • If we had extra (that no one could really demand), how would we divide any surplus?

23. Terms • Marginal Contribution (value added): μi(C) = v(C U {i}) – v(C) • shi is what i is given (its share) • Symmetry: Agents that make the same contribution should get the same payoff • Dummy player – never increases the value of the coalition beyond what it could earn alone μi(C) = v(C U {i}) – v(C) = v{i} for every C • Alternate definition of Dummy: μi(C) = 0 for every C • Additivity: if we add two games defined by v and w by letting (v+w)(S) = v(S) + w(S), then the utility for an agent in v+w should be the sum of her utilities in v and w • most controversial axiom (for example, participant i’s cost-share of a runway and terminal is it’s cost-share of the runway plus his cost-share of the terminal)

24. Superadditivity • v is superadditive if for all coalitions A, B with A∩B = Ø, v(AUB) ≥ v(A) + v(B) • Informally, the union of two coalitions can always act as if they were separate, so should be able to get at least what they would get if they were separate. • There is a synergy – if not, coalitions make no sense. • Usually makes sense • Previous examples were all superadditive • Given this, always efficient for grand coalition to form • Without superadditivity, finding a core is not possible.

25. The Shapley value [Shapley 1953] • In dividing the profit, sometimes agent is given its marginal contribution (how much better the group is by its addition) • The marginal contribution scheme is unfair because it depends on the ordering of the agents • One way to make it fair: average over all possible orderings • Let MC(i, π) be the marginal contribution of i in ordering π • Then i’s Shapley value is ΣπMC(i, π)/(n!) • The Shapley value is always in the core for convex games • … but not in general, even when core is nonempty, e.g. • v({1, 2, 3}) = v({1, 2}) = v({1, 3}) = 1, • v = 0 everywhere else

26. Example: v({1, 2, 3}) = v({1, 2}) = v({1, 3}) = 1,v = 0 everywhere else Compute the Shapley value for each. Is the solution in the core?

27. Axiomatic characterization of the Shapley value • The Shapley value is the unique solution concept that satisfies: • (Pareto) Efficiency: the total utility is the value of the grand coalition, Σi in Nu(i) = v(N) • Symmetry: two symmetric players (add the same amount to coalitions they join) must receive the same utility • Dummy: if v(S {i}) = v(S) for all S, then i must get 0 • Additivity: if we add two games defined by v and w by letting (v+w)(S) = v(S) + w(S), then the utility for an agent in v+w should be the sum of her utilities in v and w.

28. Additivity Example • Game a • Game b

29. a+b Notice how their value in the combined game is the sum of their values in the original games

30. Shapley Result • Satisfies all three fairness axioms (the last three listed) • If you came up with some other method that satisfied all fairness axioms, it would be Shapley. In other words, the Shapley value is the UNIQUE value that satisfies all fairness axioms.

31. Computing a solution in the core • How do you even represent the characteristic function which defines the problem? exponential in the number of agents. • Can use linear programming: • Variables: u(i) • Distribution constraint: Σi in Nu(i) = v(N) • Non-blocking constraints: for every S, Σi in Su(i) ≥ v(S) • Problem: number of constraints exponential in number of players (as you have values for all possible subsets) • … but is this practical?

32. Convexity • v is convex if for all coalitions A, B, v(AUB)-v(B) ≥ v(A)-v(A∩B) • In other words, the amount A adds to B (in forming the union) is at least as much it adds to the intersection. • One interpretation: the marginal contribution of an agent is increasing in the size of the set that it is added to. The term marginal contribution means the additional contribution. Precisely, the marginal contribution of A to B is v(AUB)-v(B) • Example, suppose we have three independent researchers. When we combine them at the same university, the value added by A is greater if the set is larger.

33. Example • v{1} = 0 v{2} = 0 v{3} = 0 • v{1,2} = 8 v{2,3} = 8 v{1,2,3} = 12 • Let A = {1,2} and B={2,3} • v(AUB)-v(B) ≥ v(A)-v(A∩B)

34. Convexity • In convex games, core is always nonempty. (Core doesn’t require convexity, but convexity produces a core.) • One easy-to-compute solution in the core: agent i gets u(i) = v({1, 2, …, i}) - v({1, 2, …, i-1}) • Marginal contribution scheme- each agent is rewarded by what it adds to the union. • Works for any ordering of the agents

35. Theory of cooperative games with sidepayments • It starts with von Neumann and Morgenstern (1944) • Two main (related) questions: • which coalitions should form? • how should a coalition (which forms) divide its winnings among its members? • The specific strategy the coalition will follow is not of particular concern... • Note: there are also cooperative games without sidepayments

36. Example: Minimum Spanning Tree game • For some games the characteristic form representation is immediate • Communities 1,2 & 3 want to be connected to a nearby power source • Possible transmission links & costs as in figure 1 40 100 40 3 40 source 20 50 2

37. 1 40 100 40 3 40 source 20 50 2 Example: Minimum Spanning Tree game • Communities 1,2 & 3 want to be connected to a nearby power source • v(void) = 0 • v(1) = 0 • v(2) = 0 • v(3) = 0 • v(12) = -90 + 100 + 50 = 60 • v(13) = -80 + 100 + 40 = 60 • v(23) = -60 + 50+ 40 = 30 • v(123) = -100 + 100 + 50+ 40 = 90 • A strategically equivalent game. • We show what is gained from the coalition. How to divide the gain?

38. The Core • What about MST game? We use value to mean what is saved by going with a group. • v(void)= v(1) = v(2) = v(3)=0 • v(12) = 60, v(13) = 60, v(23) = 30 • v(123) = 90 • Analitically, in getting to a group of three, you must make sure you do better than the group of 2 cases: • x1+x2>=60, iff x3<=30 • x1+x3>=60, iff x2<=30 • x2+x3>=30, iff x1<=60 1 2

39. 1 40 100 40 3 40 source 20 50 2 The Core • Let’s choose an imputation in the core: x=(60,25,5) • The payoffs represent the savings, the costs under x are • c(1)=100-60=40, • c(2)=50-25=25 • c(3)=40-5=35 FAIR?

40. The Shapley value: computation • MST game • v(void) = v(1) = v(2) = v(3)=0 • v(1,2) = 60, v(1,3) = 60, v(2,3) = 30, v(1,2,3) = 90 Value added by Coalitions

41. The Shapley value: computation • MST game • v(void) = v(1) = v(2) = v(3)=0 • v(12) = 60, v(13) = 60, v(23) = 30, v(123) = 90 Value added by Coalitions

42. The Shapley value: computation • MST game • v(void) = v(1) = v(2) = v(3)=0 • v(12) = 60, v(13) = 60, v(23) = 30, v(123) = 90 Value added by Coalitions Discount per person

43. The Shapley value: computation • A faster way • The amount player i contributes to coalition S, of size s, is v(S)-v(S-i) • This contribution occurs for those orderings in which i is preceded by the s-1 other players in S, and followed by the n-s players not in S • ki = 1/n! S:i in S (s-1)! (n-s)! (v(S)-v(S-i))

44. The Shapley value has been used for cost sharing. Suppose three planes share a runway. The planes require 1, 2, and 3 KM to land. Let’s label those planes 1, 2, and 3. Thus, a runway of 3 must be build, but how much should each pay? Instead of looking at utility given, look at how much increased cost was required.

45. The Shapley value has been used for cost sharing. Suppose three planes share a runway. The planes require 1, 2, and 3 KM to land. Thus, a runway of 3 must be build, but how much should each pay? Instead of looking at utility given, look at how much increased cost was required.

46. An application: voting power • A voting game is a pair (N,W) where N is the set of players (voters) and W is the collection of winning coalitions, s.t. • the empty set is not in W (it is a losing coalition) • N is in W (the coalition of all voters is winning) • if S is in W and S is a subset of T then T is in W • Also weighted voting game can be considered in which each player has w voting weight an a quota is required to win. • The Shapley value of a voting game is a measure of voting power (Shapley-Shubik power index) • The winning coalitions have payoff 1 • The losing coalitions have payoff 0

47. All powers sum to 1. What is power? • {99,99,1: 100} Any two win, so all have equal power 1/3. • {6,4,2: 10} Having non-zero weight does not guarantee power. 2 has zero power. It is a dummy. • If the same agent exists in all winning coalitions, the core is non-empty (as no subgroup can pull out and win). • Simple game: any coalition is either winning (1) or losing (0) • It seems clear that some simple games could not be formulated in these terms. (incomplete)

48. An application: voting power • The United Nations Security Council in 1954 • 5 permanent members (P) • 6 non-permanent members (N) • the winning coalitions had to have at least 7 members, • but the permanent members had veto power • A winning coalition had to have at least seven members including all the permanent members • The seventh member joining the coalition is the pivotal one: he makes the coalition winning • Is this a fair voting system?

49. An application: voting power • 11! possible orderings • Power of non permanent members • (PPPPPN)N(NNNN) • 5 ways to pick which N comes before and 4! possible arrangements for those that come after. (5! total choices) • 6! possible arrangements for (PPPPPN) • The total number of arrangements in which an N is pivotal is 6!5! • The power of all non permanent members is 6!5!/11!, each getting .0022 • The power of each permanent members is.1974 • The ratio of power of a P member to a N member is 91:2 • In 1965 • 5 permanent members (P) • 10 non-permanent members (N) • the winning coalitions has to have at least 9 members, • the permanent members keep the veto power • Similar calculations lead to a ratio of power of a P member to a N member equal to 105:1 (.19627:.001865)

50. How to represent the characteristic function (in condensed way)? • Induced subgraphs Node represents agents. Arc represents added benefit if source/target agents are both in the same coalition