Cooperation and Efficiency in Utility Maximization Games

1. Cooperation and Efficiency in Utility Maximization Games Milan VojnovićMicrosoft Research Joint works with Yoram Bachrach, Vasilis Syrgkanis, and ÉvaTardos MSR SVC, October 1, 2013

2. This talk based on… • Y. Bachrach, V. Syrgkanis and M. V., Incentives and Efficiency in Uncertain Collaborative Environments, WINE 2013 • Y. Bachrach, V. Syrgkanis, E. Tardos, and M. V., Strong Price of Anarchy and Coalitional Dynamics, working paper, 2013

3. Two Main Questions • Q1: What social efficiency can be guaranteed by simple local value sharing rules in uncertain environments? • Abilities and effort budgets are private information • Q2: What social efficiency can be guaranteed in presence of coalitional deviations?

4. Contribution Incentives • Rewards for contributions • Credits • Social gratitude • Monetary incentives • Online services • Ex. Quora, Stackoverflow, Yahoo! Answers • Other • Scientific authorship • Projects in firms

5. Question Topic Site

6. Another Example: Scientific Co-Authorship o random

7. Utility Maximization Games (): • : set of players • : strategy space, • : utility of a player,

8. Project Contribution Games 1 1 Special: total value functions 2 2 Share of value i j n m

9. Two Main Questions • Q1: What social efficiency can be guaranteed by simple local value sharing rules in uncertain environments? • Abilities and effort budgets are private information • Q2: What social efficiency can be guaranteed in presence of coalitional deviations?

10. Incomplete Information • Player type is private information , for • Production output: ) • Assumed to be increasing concave in effort • Effort budget: • Utility: • Efficiency:

11. Marginal Contribution Condition • A game is said to satisfy marginal contribution condition if for every player and : • k-approximate marginal contribution:] • Locally at each project:

12. A Sufficient Condition for Marginal Contribution Condition • Suppose that each project value function is a function of the total investment in this project that is increasing, concave and • Then, proportional value sharing satisfies marginal contribution condition

13. Proof Sketch • concave and , for every • Take and to obtain

14. Efficiency and Marginal Contribution • Suppose that local sharing rules satisfy the marginal contribution condition, and project value functions satisfy diminishing marginal returns. Then, every mixed-strategy Bayes-Nash equilibrium of the incomplete information game has the social value that is at least ½ of the optimal social value • Same guarantee holds for every coarse correlated equilibrium of the complete information game

15. Proof Sketch • Marginal contribution condition -universally smooth game • -universally smooth game efficiency of at least for every mixed Bayes-Nash equilibrium (and coarse correlated equilibrium) • Hence,efficiency of 1/2

16. Universal Smoothness [Roughgarden, Syrgkanis 2012] • An incomplete information game is -universally smooth if for every there exists a strategy profile such that for all and • If a game is -universally smooth then every mixed Bayes-Nash equilibrium of the incomplete information game has the expected social value of at least of the maximum social value • Same holds also for every coarse correlated equilibrium of the complete information game

17. Marginal Contribution and Universal Smoothness • Let be a socially optimal outcome, and let

18. } Tightness of ½ 1 1 2 2 • Proportional allocation and two types of projects there exists a pure-strategy Nash equilibrium in which all players invest all their efforts to project 1

19. Tightness of ½ (cont’d) } 1 1 2 2 • Nash equilibrium: • Social optimum:(players invest in distinct projects) , large

20. Contribution Order Value Sharing • A rank-order sharing assigns a fixed share depending on the rank of the investment with respect to the marginal contribution • Suppose that player with -th largest marginal contribution is allocated a share proportional to • Then, the social value in any Bayes-Nash equilibrium (and coarse correlated equilibrium) is at least of the optimal social value.

21. Proof Sketch • (marginal contribution t-th largest) (telescope formula + diminishing incr.)

22. Soft Budget Constraints • A1: is continuously differentiable, concave in and • Ex. holds for proportional allocation and project value functions of total investment that are continuously differentiable, concave and zero at zero • A2: is convex increasing in

23. Efficiency: Bad News First • For the class of convex production costs, the worst-case efficiency can be arbitrarily small • Consider a simple example with one project with value function , proportional allocation, and symmetric linear production costs • Recall that in this case with hard budget constraints the efficiency is at least 1/2 Efficiency =

24. Efficiency: Good News • For any concave sharing rule and the elasticity of the cost functions of at least the expected social welfare in any Bayes-Nash equilibrium is at least of the optimal social welfare. • Obs. • Constant factor efficiency independent of the number of players for any • Budget constraints may be seen as a limit of a sequence of convex cost functions whose elasticities go to infinity • For (linear production costs) the result gives a zero efficiency bound Elasticity of at :

25. Two Main Questions • Q1) What social efficiency can be guaranteed by simple local value sharing rules in uncertain environments? • Abilities and effort budgets are private information • Q2) What social efficiency can be guaranteed in presence of coalitional deviations?

26. Cooperative Nash Equilibrium Concepts • Strong Nash equilibrium [Aumann, 1959] • No coalition deviation exists that would benefit each member of the coalition • Strong correlated Nash equilibrium • Coalitional sink equilibrium • Steady-state of a Markov dynamics where in each step a coalition is picked and the members of the coalition deploy a coalitional deviation

27. New Concept: Coalitional Smoothness • A utility maximization game is -coalitionally smooth if there exists a strategy profile such that , for all and where = permutation of

28. Coalitional Smoothness and Price of Anarchy • Suppose that a utility maximization game is -coalitionally smooth, then whenever a strong Nash equilibrium exists it has a social value of at least of the maximum social value • Same holds for every strong coarse correlated equilibrium

29. Proof Sketch • Let be a strong Nash equilibrium and a socially optimal outcome • If all players coalitionally deviate to then there exists a player who is blocking the deviation, i.e. , say this player is 1 • If players coalitionally deviate from to then there exists a player who is blocking the deviation, say player 2 • Thus, where for • Combining with coalitional smoothness:

30. Marginal Contributions and Coalitional Smoothness • Suppose that in a monotone valid utility game each player is guaranteed a share of at least of his marginal contribution to the social value, then the game is -coalitionally smooth • Thus, the efficiency of at least • Ex. 1/2 if

31. Potential Games • Potential function: • is said to be -close to social value function if: • Suppose that a utility maximization game with non-negative utility function has a potential function that is -close to the social value function, then the game is -coalitionally smooth

32. Coalitional Best-Response Dynamics • For each iteration : • Pick the coalition size by sampling from distribution • Pick coalition uniformly at random from the set of all possible coalitions of size • Let players in deviate to a joint strategy profile that maximizes the total utility of the coalition conditional on the current strategy deployed by players outside of • All players in deviate to their strategy in the above optimal • Update

33. Efficiency of Coalitional Best-Response Dynamics • Suppose that the utility maximization game with non-negative utilities is -coalitionally smoothand the coalition size is sampled from distribution .Then, the expected social value in every coalitional sink equilibrium of the coalitional best-response dynamics is at least of the maximum social value

34. Proof Sketch = =

35. Conclusion Local sharing rules • Showed efficiency bound of ½ for a large class of project contribution games under incomplete information about abilities and effort budgets of players • Showed that this holds as well for correlated equilibrium in the complete information game Coalitional deviations • Introduced novel concept of coalitional smoothness • Showed how this new concept implies efficiency bounds in strong Nash equilibrium, correlated strong Nash equilibrium, and coalitional sink equilibrium

36. Some Related WorkSee papers for more complete list • Vetta, Nash Equilibria in Competitive Societies, with applications to Facility Location, Traffic Routing, and Auctions, FOCS 2002 • Roughgarden, The Price of Anarchy in Games of Incomplete Information, EC 2012 • Syrgkanis, Bayesian Games and the Smoothness Framework, ArXiv e-prints, 2012 • Anshelevich and Hoefer, Contribution Games in Networks, Algorithmica, 2011 • Goemans, Mirrokni, and Vetta, Sink Equilibria and Convergence, FOCS 2005