Load Balancing, Multicast routing, Price of Anarchy and Strong Equilibrium

1. Load Balancing, Multicast routing, Price of Anarchy and Strong Equilibrium Computational game theory Spring 2008 Michal Feldman

2. Load Balancing Model: Unrelated Machines machines • Set of machines M = {M1,…,Mm} • Set of jobs N = {1,…,n} • Unrelated machines model:Job (player) i has load wij on machine j • Strategy: select a machine • Cost of a job = total load on selected machine • Objective: minimize makespan (max load) • Special cases: • Identical machines: wij=wij’ for all j,j’ • Related machines: each machine j has a speed sj, and each job i has load li, and wij=li/sj jobs 4 5 3 M1 M2 L1(s)=9 L2(s)=3

3. (pure) equilibrium existence • Potential function • Identical machines: sum of squares (why?) • Unrelated machines: • Does sum of squares work? • No ! • Before migration: 10, after migration: 9, so cost decreased • Yet, sum of squares increased from 102+52 to 92+92 1 4 10 5

4. Lexicographic order • Definition: a vector (l1,…lm) is smaller than (l1’,…,lm’) lexicographically if for some i, li < li’ and lk = lk’ for all k<I • Definition: A joint action s is smaller than s’ lex. (ss’) if the vector of machine loads L(s), sorted in non-decreasing order, is smaller lex. than L(s’) s ss’ s’

5. (Pure) NE Existence • Lemma: if a job i improves its cost by migration, then the lexicographic order decreases • Proof sketch: • a job migrating from blue machine to red machine • Only the load on these two machines change (blue decreases, red increases) • But if the migrating job improves, red (in post-migration) must be lower than blue (in pre-migration) • Thus after migration, both blue and red are lower than blue prior to migration • Thus profile decreases lexicographically • Conclusion 1: load balancing game admit a Nash equilibrium in pure strategies • Conclusion 2: price of stability of any load balancing game is 1

6. Price of Anarchy for identical machines • Theorem: in any load balancing game on identical machines, it holds that • Proof: • Let s be a NE and let s* be OPT • Let i’ be a machine with highest cost in s, and let j’ be job with lowest weight on machine i’ • wlog, at least 2 jobs on machine i’ (why?), thus w j’≤ ½ cost(s) • Since s is a NE, for any machine i≠I’ (job j’ stays) • li ≥ li’ – wj’ ≥ cost(s) – ½ cost(s) = ½ cost(s)

7. Convergence time of best response for identical machines • Max-weight best response policy: • activate jobs, always activating job of max-weight among unsatisfied jobs • activated job migrates to its best machines (i.e., performs a best-response) • Theorem: for any load balancing game on identical machines, the max-weight best response policy converges to a NE, after each agent was activated at most once (from any initial profile)

8. Convergence time of best response for identical machines • Proof sketch: • Claim: once a job was activated, it never gets unsatisfied again • Proof of claim is based on two observations (for identical machines): • Job is satisfied IFF assigned to machine with minimal load (other than itself) • Best response never decreases the minimal load among the machines (why?) • Thus, a job can become unsatisfied only if another job migrated to its own machine • Thus, sufficient to show that a migration of a job of lower weight into one’s machine cannot make it unsatisfied • Proof in class.. • Note: order is crucial. Under “min-weight best response policy”, there may be instances with an exponential number of steps

9. 1 1 e e Machine 2 Machine 1 Machine 1 Machine 2 Price of anarchy for unrelated machines • POA for unrelated machines is unbounded Machine 1 Machine 2 Job 1 Job 2 Social optimum Nash equilibrium makespan=e makespan=1 PoA=1/e

10. Allowing Coordination in Equilibrium • Strong Equilibrium [Aumann’59] • No coalition can deviate and strictly improve the utility of all of its members • very robust concept • may be a better prediction of rational behavior • most games do not admit Strong Eq. • usually applied to pure Eq with pure deviations

11. Example 1: Prisoner’s Dilemma cooperate defect Unique Nash Eq. cooperate defect Strong Eq. ? Prisoner’s dilemma does not admit any Strong Eq.

12. Price of Anarchy (PoA) [KP00]: Strong Price of Anarchy Strong Price of Anarchy (SPoA): • Determining SPoA requires two parts: • Proving existence of Strong Eq • Bounding the worst ratio • SE  NE  SPoA ≤ PoA

13. k-Strong Equilibrium S=S1x…xSn • A joint action sS is not resilient to a pure deviation of a coalition Gif there is a pure action profile g of G such that ci(s-G ,g)<ci(s)for any i G • e.g., (defect,defect) in Prisoner’s dilemma • A pure Nash Eq sS is resilient to pure deviation of coalitions of size k if there is no coalition G of size at most k such that s is not resilient to a pure deviation by G • A k-Strong Equilibrium is a pure Nash Eq that is resilient to pure deviation of coalitions of size at most k

14. Strong Equilibrium Hierarchy = NE 1-SE 2-SE =SE [Aumann] n-SE

15. Related Work • Existence of Strong Equilibrium • monotone decreasing congestion games [Holzman+Lev-tov 1997, 2003] • monotone increasing congestion games + correlated SE [Rosenfeld+Tennenholtz 2006] • Related solution concepts • Coalition-proof Eq. [Bernheim 1987] • Group-strategyproof mechanisms [Moulin+Shenker 2001] • Coalitions with transferable utilities [Hayrapetyan et al 2006] NE CPE SE

16. 5 3 3 5 4 4 5 5 10 7 7 3 4 4 3 s 9 6 9 s’ Existence of Strong Equilibrium in load balancing games • Is every Nash Eq. on identical machines also a Strong Eq ? • NO ! (for m ≥ 3) Coalition: 5,5,3,3

17. Strong Eq. Existence • Theorem: in any load balancing game, the lex. minimal joint action s is a k-SE for any k

18. Recall Lexicographic Order • Definition: a vector (l1,…lm) is smaller than (l1’,…,lm’) lexicographically if for some i, li < li’ and lk = lk’ for all k<I • Definition: A joint action s is smaller than s’ lex. (ss’) if the vector of machine loads L(s), sorted in non-decreasing order, is smaller lex. than L(s’) s ss’ s’

19. Proof of SE Existence • Suppose in contradiction that s (lex. minimal) is not a SE, and let G be the smallest coalition (deviating to s’). • Claim: the same set of machines are chosen by G in s and in s’ (denote it M(G)) • If a job migrates TO some machine, another jobmigrates FROM it • else contradicting s is NE • If a job migrates FROM some machine, another jobmigrates TO it • else contradicting minimality of G • Since all jobs in G must benefit, all loads of M(G) in s’ must be smaller than max load of M(G) in s • Contradicting minimality of s

20. makespan= 1 1 1 makespan= e e e e e M1 M2 M1 M2 Price of Anarchy (PoA) • Recall: for unrelated machines, PoA may be unbounded Machine 1 Machine 2 Job 1 Objective: min makespan Job 2 Social optimum Nash equilibrium Nash equilibrium Strong equilibrium PoA=1/e Strong equilibrium SPoA=1

21. Strong Price of Anarchy • Theorem: for any job scheduling game with m unrelated machines and n jobs, SPoA ≤ m

22. Proof for SpoA ≤ m • Claim 1: L1(s) ≤ OPT • else: coalition of all jobs to OPT L1(s) L1(s) OPT OPT Mm Mi Mi-1 M1 Mm Mi Mi-1 M1

23. Proof for SpoA ≤ m • Claim 1: L1(s) ≤ OPT • else: coalition of all jobs to OPT • Claim 2:  iLi(s)-Li-1(s) ≤ OPT • else: consider s’, where all jobs on machines i..m go to OPT. For all J  G: • cJ(s) > Li-1(s) + OPT • cJ(s’) ≤ Li-1(s) + OPT (since all J  G together add at most OPT) Li(s) > OPT OPT Li-1(s) L1(s) Mm Mi Mi-1 M1 Mm Mi Mi-1 M1 Lm(s) ≤ m OPT

24. Lower Bound (m machines) • Theorem: there exists a job scheduling game with m unrelated machines for which SPoA ≥ m • Proof:  OPT = 1 SE makespan=m 

25. Identical Machines • Theorem: there exists a job scheduling game with m identical machines and n jobs, such that OPT 1+1/m 1 2 SE 1/m m-1 m 1 m-2

26. Mixed Deviations and Mixed Strong Eq • Nash Eq – unilateral deviations • pure and mixed deviations are equivalent • Strong Eq – coordinated deviation • Pure and mixed deviations are not equivalent • Given a mixed deviation, there might be no single pure deviation which is good J2 mixed deviation cJ1=cJ2=15/8 J1 J2 ¾ ¼ ½ ½ J2 Unique Nash Eq cJ1=cJ2=2 J1 J3 J1 J3 M1 M2 M1 M2

27. Mixed Deviations and Mixed Equilibrium • However, in many cases, allowing mixed deviations by a coalition eliminates all Nash Eq. • Theorem: for m≥5 identical machines, and n>3m unit jobs, there is no 4-Strong Eq when mixed deviations are allowed • Based on a lemma that shows that the support of any two “mixing” jobs must be disjoint

28. Strong equilibrium in multicast routing Theorem:There exists a multicast routing game that does not posses a strong equilibrium. Proof: s Unique NE: c1(S) = c2(S) = 2/2+1=2 2 1+3ε deviation: ci(S) < 2 No SE in game 1 1 1-2ε t1 t2 ½-3ε ½-3ε 2+2ε

29. Strong Price of Anarchy Theorem :The strong price of Anarchy of a multicast routing game with n players is at most H(n). Proof: • Let S be a SE, and SΓbe the induced profile of players in Γ, and let S* be OPT • For k=n,…,1, since S is SE, there exists a player “k” Gk={1,…,k} that does not benefit from coal. deviation. i.e., Potential function:

30. Proof (cont’d) • We got for every k: • Summing over all players:

31. Lower Bound • OPT: all users use indirect edge, c(OPT)=1+e • Unique NE and SE: each user uses direct edge to ti, c(NE)=c(SE)=H(n)  PoA = SPoA = PoS = SPoS = H(n) s 1 1+e t1 t2 t3 tn-2 tn-1 tn