Selfish Algorithms

1. Selfish Algorithms James Newell CS 598IG – Scattered Systems December 7, 2004

2. The Tragedy of the Commons Garret Hadrin 1968

3. Non-technical Solutions • “It is our considered professional judgment that this dilemma [nuclear war] has no technical solution” • J.B. Wiesner and H.F York • A non-null class of problems • Nuclear war • Tic-tac-toe • Commons overuse • Population growth

4. Tragedy of the Commons • Written by Mathematician William F. Lloyd in 1833 • Example of a break-down in fundamental free-enterprise economic theory (invisible hand) • Adam Smith Wealth of Nations (1776) • Laissez Faire approach to most problems.

5. Tragedy of the Commons • Olde Europe traditionally had commons open to all farmers • Each farmer focuses on maximizing his personal utility (benefit) • Benefit of additional animal is solely enjoyed by farmer • Cost of overgrazing is shared among all farmers • Every farmer reaches the same conclusion: Add more animals!

6. Tragedy of the Commons • Result • Commons is heavily over-grazed • Every farmer is worse off than optimal • Still in the interest of the farmer to add more animals • Problem Analysis • System of unlimited consumption in a limited world • Freedom in a commons brings ruin for all!

7. Real Tragedy of the Commons • Over-farming • Over-grazing • Over-fishing • National Parks • Pollution • Population growth • Peer-to-peer systems?

8. Population Growth • Population is a “common” because human families are not self-dependent. • No negative consequences • Overbreeding benefits family, hurts society • No laws to directly prohibit over-breeding from a certain group or family • Can’t appeal to conciseness for self-control • The over-breeders will eventually dominate • Strong incentive to not to conform • Causes confusion/mixed-signals

9. Mutual Coercion • Examples • Bank-robbing (Prohibited) • Parking • Taxes • Agreed upon by the majority that it affects • Preferred over commons but not perfect • Necessary under certain conditions • Reduction of “personal” freedoms • Typically accepted by later generations • Any criminal act can be considered a freedom

10. Conclusion • Freedom to breed will eventually need to be reconsidered • Add incentives to break “commons” scenario • Prevent Malthusian predictions of lower income per capita • Any situation with “Tragedy of the Commons” characteristics will have no technical solution.

11. How Bad is Selfish Routing? Tim Roughgarden and Éva Tardos Cornell University November 2000

12. Introduction • Optimal routing on large-scale networks is very difficult • Difficult to compute • Load dependent latencies • Selfish routing • Each user allowed to route on own interests • Users are rational and not malicious • View as a non-cooperative game to minimize network flow How bad is selfish routing compared to optimal routing?

13. Non-cooperative Game • Players contribute negligible fraction of the network load • Players act selfishly • Always choose route with lowest latency • Not trying to hurt other’s performance • Steady-state is a Nash equilibrium • No player can improve its situation by switching to a different route (strategy) • All flow paths have equal latency • Not always optimal equilibrium

14. Analysis Roadmap • Latency linear function • Nondecreasing and continuous latency function • Direct comparison • Bicriteria results • Assumption analysis • Perfect latency measurements • Minimal traffic generated per node

15. S1 T1 Computation Model • We call triple (G, r, l) an instance • Directed graph: G = <V, E> • k Source-Dest pairs: {s1,t1} … {sk,tk} • k rates of traffic: ri[from sito ti ] • Latency function: le(•) Simple Example le(x) = x k = 1 r1 = 1 le(x) = 1

16. Paths and Flow • Simple si-ti paths set: Pi[from sito ti] • All Paths: P = UPi • Flow function: f : P→ R+ • Flow per path: fp[from sito ti] • Flow per edge: fe = P:eP fP Simple Example V x 1 ½ T S 0 ½ x fe = ½ + ½ le(f) = 1 1 W

17. S1 S1 T1 T1 Cost • Latency per path: lP(f) =eP le(fe) • Cost function: C(f) = pP lp(f) · fp Simple Example le(x) = x 1 Cost: = 1(1) + 0(1) = 1 le(x) = 1 le(x) = x ½ Cost: = ½(½) + ½(1) = ¾ ½ le(x) = 1

18. Model Assumptions • Latency function is nonnegative, differentiable, and nondecreasing • Agents all acts in a selfish but non-malicious manner • There are many agents each contributing a negligible fraction of the network traffic • Agents can choose only one path of the network (pure strategy)

19. Nash Equilibrium • Flow is in Nash Equilibrium (Nash Flow) if no agent can improve its latency by changing its path • Nash Equilibrium Definition: for all i  {1,…,k}, P1,P2 Pi, and   [0, fP1], we have lP1(f)  lP2(f) fP -  if P = P1 fP = fP +  if P = P2 fP otherwise ~ ~

20. Wordrop’s Principle • If f is at Nash equilibrium, then all si - ti flow paths have equal latency: Li(f) • Now the cost of a flow in Nash Equilibrium: C(f) =Li(f) · ri • Also, a Nash equilibrium exists and is essentially unique [Beckman et al.]

21. Optimal Flow • Min ( C(f) ) = Min ( eE ce(fe) ) [ce (fe) = le (fe) · fe] • Special case of a non-linear program: • PPi fP = ri i  {1,…,k} • fe = PP :eP fp e  E • fP≥ 0  P P • Whenever the NLP objective function is convex, • Local minima = Global minima

22. Central Question • Selfish routing lacks coordination • To what extent does a Nash flow optimize social welfare or… What is the cost of the lack of coordination of a flow in Nash equilibrium compared to an optimal flow?

23. ½ traffic on both routes (3/2) S wants to route 1 unit of traffic to T New Equilibrium (2 > 1.5) Braess’s Paradox • Adding an innocuous link can cause negatively impact all agents V x 1 T S 0 x 1 W

24. S1 T1 Linear Latency • Theorem 1: With a linear latency function: le(x) = aex + be • In the worst case, a Nash flow = (4/3) * optimal flow le(x) = x Ratio: ¾· (4/3) = 1 Optimal Flow Nash Flow ½ 1 ½ 0 le(x) = 1

25. Corollary • Lemma: Let (G,r,l) be an instance with edge lantecy functions le(x) = aex+be for each e E Then, • A flow f is at Nash Eq. in G iff for each source-sink park i and P,P’ Pi with fP > 0 • A flow f* is globally optimal in G iff for each source-sink pair i and P,P’ Pi with fP* > 0 • Corollary: If le(x) = aex, then a flow f is optimal iff it is at the Nash Equilibrium

26. Sketch of Proof • Generally: • At rate r/2 → cost of f/2: C(f/2) ≥ ¼·C(f) • Cost of changing from rate r/2 to r ≥ ½·C(f) • Hence, C(f*) ≥ ¾ C(f) *See paper for proof details Cost of increasing rate from r/2 to r Cost of Optimal at rate r Cost of Optimal at rate r/2 = +

27. S1 T1 General Latency • Theorem 2: With a general latency function: le(x) • In the worst case, the ratio between the Nash flow and optimal is unbounded! • Bad Example: • Cost of optimal flow = (1 - i · (i + 1))-(i+1)/i • As i→ ∞, C(optimal)→ 0 le(x) = xi Optimal Flow Nash Flow 1-ε 1 ε 0 le(x) = 1 (i+1)-1/i

28. Bicriteria Results • All is not lost… Theorem 3: with continuous non-decreasing latency function: • The cost of a Nash Flow with rate r is at most the cost of an Optimal Flow with rate 2r • Proof: define a new latency function ℓ as:

29. Sketch of Proof • Evaluating optimal flow with new latency function increases cost by at most C(f) • The cost of optimal flow with new latency function can be bounded to 2C(f) • Combining these two:

30. Extensions • Agents can often only evaluate path latency approximately, rather than exactly. • Restrict to a finite number of agents, each controlling a strictly positive amount of (splittable) flow

31. Conclusions • If le(x) = aex, C(Nash) = C(Optimal) • If le(x) = aex + be, C(Nash)  (4/3) * C(Optimal) • If le(x) = aexi+ …, ratio is unbounded • C(Nash(G, r, l))  C(Optimal(G, r·2, l)) in general case • Results do not deviate greatly when removing agent assumptions

32. Characterizing Selfishly Constructed Overlay Routing Networks B. Chun, R. Fonseca, I. Stoica, and J. Kubiatowicz IEEE Infocom 2004

33. Overlay Routing • Fundamental for traditional peer-to-peer systems • Typically assume cooperative topology construction • In reality, nodes may assign links to maximize personal benefit without regard to global welfare • Do these selfishly-constructed networks have desirable global properties?

34. Roadmap • Simulate two games • Simplistic model • Realistic model • Analyze the Nash Equilibrium • Stretch • Resilience • Node degree distribution • Modifying cost function and game parameters

35. D A B C Motivation 6 Hops → 2 Hops Is it worth A to create a direct link to D? ∆ Hops > Link Cost

36. Game Model • View the overlay network as a non-cooperative game on n users • Each user is a node • Employs a pure strategy • Maximize its benefit • Minimize its link-costs • Link-construction strategy • Low-cost paths to other nodes benefit the node • Too many links could be costly for the node

37. Cost Function • The cost to connect to node j is a function and not a constant • The distance between two nodes maybe not be number of hops • Any node can connect to a neighbor subset of all possible nodes. Cost incurred to connect to node j Distance between nodes i and j Relation between two terms

38. Game Process • Each round, every player attempts to minimize Ci(s) • Exhaustive search over strategy space • Randomized local search • Exhaustive Search is NP-hard • 2(n-1) strategies to scan per node per round • Feasible for only small topologies • Randomized local search • More realistic for normal topologies • May not always produce correct Nash Eq.

39. Randomize Local Search Greater than  Less than  * Uses Link State protocol

40. Simple Case Study • Physical network: complete graph with all edge weights = 1 • Distance function = number of hops • 20 nodes in network • Exhaustive search of strategy space • Explored different cost functions • Unit: tj = 1 • Exponential distribution: tj = cj (mean = 1) • Node-degree: tj = degree(j)

41. Realistic Case Study • Physical network: transit-stub topology • Distance function: latency of links • 100 overlay nodes in 1000 node network • Randomized local search • One cost function (MaxDegree) 1 if degree(j) < MaxDegree tj = and degree(i) < MaxDegree ∞ otherwise

42. Case Study Simulations • 50 runs for simple case and 100 for realistic • Metrics • Graph Cost: sum of the node costs • Node Degree Distribution: affects resilience • Characteristic Path Length: average shortest distance • Stretch: average ratio between shortest path latency in overlay to shortest path latency in physical network

43. Simple Scenario Analysis

44. Simple Scenario Analysis • Unit-Countout is only different from optimal by constant • Unit-Nodedegree get linearly worse • Exp-Countout has a growing cost with high 

45. Realistic Scenario Analysis

46. Realistic Scenario Analysis • Varying the  impacts the degree distribution • Control overhead is proportional to number of links in graph Exponential Power-Law

47. Failure and Attack Analysis • Use K metric: ratio of connected node pairs compared to total distinct pairs in network • Failure: remove random nodes from network • Attack: remove nodes with largest degree from network

48. Simple Scenario • Star topologies are most resistant to failure and susceptible to attacks (vs. Trees) • For Unit-Nodedegree,  = 2 is the most resistant to both attacks and failures.

49. Realistic Scenario • Trade-off between performance (stretch) and resiliency • The smaller the MaxDegree is, the more resilient the network is against failures and attacks.

50. Conclusion • Examined networks created by selfish nodes using an non-cooperative game model • Showed diverse networks produced as we varied , cost functions, and the physical network • Complete graphs to trees with different properties • Degree distributions ranged from exponential to power-law distributions • Showed tradeoff between performance and resilience in the observed networks.