Price of Anarchy

1. Price of Anarchy GeorgiosPiliouras

2. Games (i.e. Multi-BodyInteractions) • Interacting entities • Pursuing their own goals • Lack of centralized control Prediction?

3. Games (review) • n players • Set of strategies Si for each player i • Possible states (strategy profiles) S=×Si • Utility ui:S→R • Social Welfare Q:S→R • Extend to allow probabilitiesΔ(Si), Δ(S) ui(Δ(S))=E(ui(S)) Q(Δ(S))=E(Q(S))

4. Zero-Sum Games & Equilibria (review) Existence, Uniqueness of Payoffs [von Neumann 1928] 1/3 1/3 1/3 Paper Scissors Rock 1/3 Rock Paper 1/3 Scissors 1/3 Nash: A product of mixed strategies s.t. no player has a profitable deviating strategy.

5. General Games & Equilibria? Borel conjectured the non-existence of eq. in general Paper Scissors Rock Rock Paper Scissors Nash: A product of mixed strategies s.t. no player has a profitable deviating strategy.

6. Prediction in GamesIdea 1 • Nash Equilibrium (1950): A strategy tuple (i.e. one for each agent) s.t. no agent can deviate profitably. • For finite games, it always exists when we allow agents to randomize • Proof on the board

7. Equilibria & Prediction NE

8. Implicit assumptions • The players all will do their utmost to maximize their expected payoff as described by the game. • The players are flawless in execution. The players have sufficient intelligence to deduce the solution. • The players know (can compute) the planned equilibrium strategy of all of the other players. • The players believe that a deviation in their own strategy will not cause deviations by any other players. There is common knowledge that all players meet these conditions, including this one. • Uniqueness

9. Games & Equilibria 1/3 1/3 1/3 Paper Scissors Rock 1/3 Rock Paper 1/3 Scissors 1/3 Nash: A product of mixed strategies s.t. no player has a profitable deviating strategy.

10. Equilibria & Prediction Stag Hare Stag Hare Multiple Nash: Which one to choose?

11. Prediction in GamesIdea 2a • Koutsoupias and Papadimitriou (1999) • If there exist several Nash Equilibria, then be pessimistic and output the worst possible one. (worst case analysis) • Worst in terms of what? Social Welfare • Examples of Social Welfare: • Sum of utilities, maxmin utility, median utility

12. Metrics of Social WelfareExamples Sum of latencies (sec) maxmin utility ($) Throughput bottleneck (bit/sec)

13. Prediction in GamesIdea 2b • Koutsoupias and Papadimitriou (1999) • If there exist several Nash Equilibria, then be pessimistic and output the worst possible one. (worst case analysis) • Normalization Social Cost (worst Equilibrium) Price of Anarchy = Social Cost (OPT) ≥1

14. Price of Anarchy PoA = ≥ 1 Social Cost (worst Equilibrium) Social Cost (OPT) B PoA = 4/3 delay (x) = x 10 A D 0 10 agents AD PoA≤5/2, for all networks x 10 [Koutouspias, Christodoulou 05] C

15. Equilibria & Prediction NE NE PoA

16. Advantages of PoA Approach • Simplicity • Widely Applicable (conditions?) • Allows for cross-domain comparisons (e.g. routing game vs facility location game) • Analytically tractable? • Several variants: Price of Stability, Price of Total Anarchy, Price of X,… YES, 1000+ citations

17. BREAK Q: Any other ways to make predictions in multi-body problems? How do you do it in real life situations?

18. Recap + Plan • Games + Worst Case Analysis + Normalization • PoA = • To do: • PoA Analysis (when welfare = sum utility) • Beyond Nash equilibria PoA Social Cost (worst Equilibrium) Social Cost (OPT)

19. Congestion Games • n players and m resources (“edges”) • Each strategy corresponds to a set of resources (“paths”) • Each edge has a cost function ce(x) that determines the cost as a function on the # of players using it. • Cost experienced by a player = sum of edge costs x x x x Cost(red)=6 Cost(green)=8 2x x x 2x

20. Potential Games • A potential game is a game that exhibits a function Φ: S→R s.t. for every s ∈ S and every agent i, ui(si,s-i) - ui(s) = Φ(si,s-i) - Φ(s) • Every congestion game is a potential game: Why? • This implies that any such game has a pure NE. Why?

21. PoA≤ 5/2 for linear latencies [Koutouspias, Christodoulou 05], [Roughgarden 09] • Definition: A game is (λ,μ)-smooth if • iCi(s*i,s-i) ≤ λcost(s*) + μ cost(s) for all s,s* • Then: POA (of pure Nash eq) ≤ λ/(1-μ) Proof: Let s arbitrary Nash eq. cost(s) = iCi(s)[definition of social cost] ≤ iCi(s*i,s-i) [s a Nash eq] • ≤ λcost(s*) + μ cost(s) [(λ,μ)-smooth]

22. PoA≤ 5/2 for linear latencies [Koutouspias, Christodoulou 05], [Roughgarden 09] • Technical lemma: A linear congestion game is (5/3,1/3)-smooth. • Proof : • Step 0: Matlab simulations to get a hint about what is the best possible (λ,μ) s.t. game is (λ,μ)-smooth. • Step 1: Verify hypothesis (On the board).

23. Tight Example • N agents, • 2N elements (x1, x2,…, xN) (y1, y2,…, yN) c(x)=x for all of them • Each agent i has 2 strategies : (xi ,yi) or (xi, yi-1, yi+1) x1 y1 xN yN x2 y2 … …

24. BREAK 2 Q: What about PoA of mixed NE?

25. Recap + Plan • Games + Worst Case Analysis + Normalization • PoA = • To do: • PoA Analysis (when welfare = sum utility) • Beyond Nash equilibria PoA Social Cost (worst Equilibrium) Social Cost (OPT)

26. Other Equilibrium Notions 1/3 1/3 1/3 Rock Paper Scissors 1/3 Rock Paper 1/3 Scissors 1/3 Nash: A product of mixed strategies s.t. no player has a profitable deviating strategy.

27. Other Equilibrium Notions 1/3 1/3 1/3 Rock Paper Scissors 1/3 Rock Paper 1/3 Scissors 1/3 Choose any of the green outcomes uniformly (prob. 1/9) Nash: Aprobability distribution over outcomes, that is a product of mixed strategies s.t. no player has a profitable deviating strategy.

28. Other Equilibrium Notions 1/3 1/3 1/3 Rock Paper Scissors 1/3 Rock Paper 1/3 Scissors 1/3 Nash: Aprobability distribution over outcomes, s.t. no player has a profitable deviating strategy. Coarse Correlated Equilibria (CCE):

29. Other Equilibrium Notions Rock Paper Scissors Rock Paper Scissors Aprobability distribution over outcomes, s.t. no player has a profitable deviating strategy. Coarse Correlated Equilibria (CCE):

30. Other Equilibrium Notions Choose any of the green outcomes uniformly (prob. 1/6) Rock Paper Scissors Rock Paper Scissors Aprobability distribution over outcomes, s.t. no player has a profitable deviating strategy. Coarse Correlated Equilibria (CCE):

31. Other Equilibrium Notions Is this a CE? NO Choose any of the green outcomes uniformly (prob. 1/6) Rock Paper Scissors Rock Paper Scissors Aprobability distribution over outcomes, s.t. no player has a profitable deviating strategy even if he can condition the advice from the dist. . Correlated Equilibria (CE):

32. Other Equilibrium Notions NE CE CCE Pure NE

33. Smoothness bounds extend to CCE • Definition: A game is (λ,μ)-smooth if • iCi(s*i,s-i) ≤ λcost(s*) + μcost(s)for all s,s* • Then: POA (of pure Nash eq) ≤ λ/(1-μ) Proof: Let s arbitrary Nash eq. cost(s) = iCi(s)[definition of social cost] ≤ iCi(s*i,s-i) [s a Nash eq] • ≤ λcost(s*) + μ cost(s) [(λ,μ)-smooth]

34. Smoothness bounds extend to CCE • Definition: A game is (λ,μ)-smooth if • iCi(s*i,s-i) ≤ λcost(s*) + μ cost(s)for all s,s* • Then: POA (of pure Nash eq) ≤ λ/(1-μ) Proof: Let s arbitrary CCE. E[cost(s)] = E[iCi(s)][definition of social cost] • ≤ E[ iCi(s*i,s-i)][s a CCE] • ≤ λE[cost(s*)] + μE[cost(s)]

35. Criticism of PoA Analysis • What happens in we add 10^10 to the utilities of each agent? • Tightness is achieved over classes. • Holds only for sum of utilities • Sensitive to noise

36. Open Questions • Choose your favorite class of games. Attempt (λ,μ)-smoothness analysis. • Possible problems • Technique gives trivial upper bounds • Still need to identify lower bounds • What about uncertainty? • Other (hidden) assumptions? • [Balcan,Blum,Mansur’09] [Balcan,Constantin,Ehrlich‘11]

37. Recap • Nash always exists (fixed point) but not unique • PoA addresses non-uniqueness • (λ,μ)-smoothness general technique for proving PoA bounds, extends to other notions, and can provide tight bounds

38. Thank You