Computing Shapley Values, Manipulating Value Distribution Schemes, and Checking Core Membership in Multi-Issue Domains

1. Computing Shapley Values, Manipulating Value Distribution Schemes, and Checking Core Membership in Multi-Issue Domains Vincent Conitzer and Tuomas Sandholm

2. Agenda • Introduction to coalitional games • Multi-Issue Domains • How to distribute the gains • The Core • Shapley Value • Other marginal contribution schemes • Computing the Shapley value • Manipulating contribution schemes • Checking core membership

3. Coalitional Games • Coalition formation is a key part of automated negotiation between self-interested agents • Several of companies can unite into a virtual organization to take more diverse orders and gain more profit • Truck delivery companies can share truck space, as the cost is mostly dependant on the distance rather than on the weight carried • Coalition formation has been studied extensively in game theory, and solution concepts were adopted in multi agent systems

4. Coalitional Games Solutions • Given a coalitional game we want to find the distribution of the gains of the coalition between the agents • Different solution concepts have different objectives • The Core promotes stability • The Shapley value promotes fairness • Game theory has studied these solution concepts for quite some time, but the computational aspect has received little attention

5. Some Questions to Keep in Mind • How much should each of the employees of the company be paid to make sure a group of them won’t be bought away by another company? • Get a value division in the core • A few truck delivery companies unite to carry a high load of deliveries. How can the profits be divided fairly? • The Shapley value division

6. Coalitional Games With Side Payments • The game is presented as a characteristic function • Let A be the set of agents (players) • Each potential coalition S has a value v(S) • The value is independent of what the non members of the coalition do • The characteristic function: • Typically it is increasing:

7. Super additivity • The characteristic function is super additive if for all disjoint sets of a S,T we have: • This means every two subsets can do better if they unite • Finally we would get the grand coalition of all the agents • This does not always hold: • Hard optimization problem to decide what to do united • Anti trust laws

8. Multi Issue Domains • The characteristic function is the sum of values of independent issues • We have sub-games • The characteristic function (for every subset S of A) is • Every coalition gets the sum of what it gets in all the sub games • If a game is a decomposition to increasing (super additive) sub games, it is also increasing (super additive)

9. Games Concerning a Subset of the Agents • We say only concerns a subset of the agents if • Assuming that each of the sub games concerns only a small subset of the agents we can improve our calculations • Our representation of the characteristic function now only requires a small fraction of the space it once took:

10. Solution Concepts • On a super additive game, the grand coalition is likely to form, and the coalition gets v(A) • How much does each agent gets? • We want a value division • We want to divide all the gains:

11. The Core • The best known solution concept • Proposed by Gillies (1953) and von Neumann & Morgenstein (1947) • A value division is in the core if no sub coalition has an incentive to break away • A value division d is blocked by a sub coalition S if • If d is blocked by S, it is not in the core • Some coalitional games have an empty core

12. Player Types • Dummy players add nothing to all coalitions: • Equivalent players add the same to any coalition that does not contain any of the two players:

13. The Shapley Value (Cont.) • A well know value division scheme • Aims to distribute the gains in a fair manner • A value division that conforms to the set of the following axioms: • Dummy players get nothing • Equivalent players get the same • If a game v can be decomposed into two sub games, an agent gets the sum of values in the two games: • Only one such value division scheme exists

14. The Shapley Value • Given an ordering of the agents in A, we define to be the set of agents of A that appear before a in • The Shapley value is defined as the marginal contribution of an agent to its set of predecessors, averaged on all possible permutations of the agents:

15. A Simple Way to Compute The Shapley Value • Simply go over all the possible permutations of the agents and get the marginal contribution of the agent, sum these up, and divide by |A|! • Extremely slow • Can we use the fact that a game may be decomposed to sub games, each concerning only a few of the agents?

16. Computing the Shapley Value • If v can be decomposed to several sub games, we know (from the axioms) that • If only concerns then for any player a, we have

17. Computing the Shapley Value • We do not really need to sum over all possible orderings, but rather on all possible subsets of agents that arrive before player a • For each such sub set we get the same marginal contribution of player a. • If the sub set S has n agents, there are n! ordering on the players inside. There are then (|A|-n-1)! ways to complete this ordering to an ordering on all agents. We get:

18. Computing the Shapley Value Quickly in Multi Issue Domains • Compute the Shapley value for each sub game, using the previous formula, only taking into account the concerning agents, then sum these up • If we assume computation of factorials, multiplication and addition in constant time we get an time complexity of or less precisely

19. Marginal Contribution Based Value Division Schemes • A marginal contribution scheme is a scheme that chooses some ordering of the players, and distributes the gains to the players according to their marginal contribution • If on the chosen orderings you add much to the value of the coalition of the players before you on the ordering, you deserve a nice share of the profits

20. Marginal Contribution Based Value Division Schemes • For the Shapley value we have considered an average on all possible orders • If we consider just one of the possible orderings, the value an agent gets depends on it location in the ordering • Obviously, the agent has a specific location it wants to be in • If the game is convex (you add to a coalition at least as much as you add to any of its subsets), you want to be last in the ordering

21. Marginal Contribution Based Value Division Schemes (Cont.) • If we randomly choose a permutation the expectancy of the value distribution for an agent is its Shapley value • This requires a trusted source of randomness / cryptography • Another way is to show that even if an agent has total control on the ordering chosen, it would still be computationally intractable for that agent to find the optimal ordering for him • The computational complexity is used as a barrier for manipulation

22. Maximal Marginal Contribution • Let v be a game decomposed as follows: and the game only concerns • We are given an agent a and a number k, and are asked if there is some such that the value • We want to see if we can find a subset of the agents to which a’s marginal contribution is at least k • These would be the agents before a in the ordering a would choose

23. NP-Completeness of Max-Marginal-Contribution • Conitzer and Sandholm have shown that Max-Marginal-Contribution is NP-Complete, even in the case that and all ‘s take values in {0,1,2} • The problem is in NP since for a given subset of agents we can simply calculate the marginal contribution of a to this subset

24. NP-Completeness of Max-Marginal-Contribution • NP-hardness is proven by reducing an arbitrary MAX2SAT instance to a Max-Marginal-Contribution instance • In MAX2SAT we are given a set V of Boolean variables and a set of clauses C, each with 2 literals, and a target number r of satisfied clauses • For each variable v in V there is an agent Av • We also have an agent a, whose contribution we want to maximize • For every clause c there is a sub game (issue) tc, that only concerns the agents a and the agents representing the variables in the clause c

25. NP-Completeness of Max-Marginal-Contribution • The sub game results are as follows: • 1 point for having a in the coalition • 1 point for having all the agents representing the negative literals • But, if you want to get 2 points, you also have to have one of the agents representing the positive literals • The marginal contribution we want is k=r

26. NP-Completeness of Max-Marginal-Contribution • If there is a solution to MAX2SAT with r satisfied clauses, take V+ - the variables set to true • What is the marginal contribution of a to this subset? • Hint: you either satisfied the clause by setting one of the negative literals to false, or if you didn’t, you’ve set one of the positive literals to true • Given a solution S to max-marginal-contribution, look at the assignment of true to everything in S, false otherwise • If a sub game tc has increased the value by 1 due to adding a, what can you say about the clause? • Open question: we have used increasing games here, so the problem is NP-Complete even if the game is known to be increasing. What is the complexity for super additive games?

27. Checking Core Membership • Let v be a game decomposed as follows: and the game only concerns • We are given a value division that may not even be feasible • If it isn’t we can increase only the value of the grand coalition to the point where it is (the help of an outside benefactor for the stability) • We are asked if the division is in the core, or if there is no blocking sub coalition for it

28. NP-Completeness of Checking Core Membership • Conitzer and Sandholm have shown that checking core membership (CHECKE-IF-BLOCKED) is NP-Complete • The problem is in NP since for a given subset of agents we can simply calculate the sum of their values in the division and see if it is less than v(S)

29. NP-Completeness of Checking Core Membership • NP-hardness is proven by reducing an arbitrary VERTEX-COVER instance to a core membership problem • We have an agent for each vertex, av, and another special agent a • We have a sub game for each edge, that only concerns agent a and the agents of the edge’s vertices • The value of the sub game is 1 if the coalition contains agent a and at least one of the edge’s vertices (we have agent a, and the edge is covered) • The value distribution to check:

30. NP-Completeness of Checking Core Membership • If there is a vertex cover with W vertices • What is the value of the coalition of these vertices and agent a? • How much do they get according to the value distribution? • If a set of agents is a blocking coalition • It has to contain agent a (or they get nothing) • Consider the set of vertices of the agents in the blocking coalition, W • How much do they get according to the value distribution? • Can the number of vertices in W be smaller than r? • To block, v(S) must be greater than v(a), since a is in the blocking coalition • But then we have to get 1 for every sub game, so we have covered all the edges, with r vertices or less

31. Conclusions • Coalitional games important for automated negotiation between agents • Such games can be decomposed to sub games (issues) which only concern some of the agents • We can quickly compute the Shapley value in some of these cases • Other marginal contribution value distribution schemes can be manipulated, but the general case is hard (an NP-complete problem) • So such distribution schemes are acceptable in some cases, even if some of the agents have control on the chosen ordering • Checking if a value distribution is stable (in the core) is hard (and NP-Complete problem in the general case)

32. Open Questions • NTU games (no side payments) • Finding value divisions the are even harder to manipulate (eg. PSPACE-hard) • Finding stability concepts that take into account the complexity of finding a beneficial deviation • The complexity of other (longer term) solution concepts