Applications of Game Theory Part II(b)

1. Applications of Game Theory Part II(b) John C.S. Lui Computer Science & Eng. Dept The Chinese University of Hong Kong

2. First course On the interaction between Overlay Routing and Underlay Routing Y. Liu, H. Zhang, W. Gong, D. Towsley INFOCOM 2005

3. Control Control Control Motivation: Interactions Between Application Level Network and Physical Network • physical network control • routing, congestion control,… • add an overlay • and another…… Result? • interactions? • controllers mismatch?

4. Outline • Problem Formulation • Simulation Study • Game-theoretic Study • Conclusions

5. Routing on physical network level Inter-domain: BGP, etc. Intra-domain: OSPF, MPLS, etc. determine routes for all source-destination traffic demand pairs minimize network-wide delay, cost, etc. Routing in Underlay Network C D E A B traffic demand pair: A->B traffic demand pair: A->C traffic demand pair: C->B

6. Routing in Overlay Network • An overlay network choose routes at application level to minimize its own delay or cost Overlay demand: A->B logical routes: A->C->B and A->B C A B • Overlay • gains advantage better path: delay, loss, throughput, etc • is selfish potential performance degradation to other non-overlay traffic C D E A B demand pair: A->C demand pair: C->B demand pair: A->B

7. Considering overlay and underlay together ? • How do they interact with each other? • How does selfish behavior of overlay routing • affect overall network performance? • affect non-overlay traffic performance? • affect its own performance?

8. overlay traffic demand flow allocation on physical routes: “Y” flow allocation on logical links: “X” traffic demand for underlay non-overlay traffic demand Iterative Dynamic Process • equilibrium: existence? uniqueness? • dynamic process: convergence? oscillations? • performance of overlay and underlay traffic? Interactions Between Overlay Routing and Underlay Routing Overlay Routing Optimizer To minimize overlay cost Underlay Routing Optimizer To minimize overall network cost

9. Approach by authors • Focusing interaction in a single AS • Considering two routing models for overlay and one routing model for underlay • Simulating the interaction dynamic process • Studying this process in a Game-theoretic framework

10. Routing Models • Overlay routing model • Selfish source routing • Individual user controls infinitesimal amount of traffic, to minimize its own delay • Optimal overlay routing • A central entity minimizes the total delay of all overlay traffic demands • Underlay routing model • Optimal underlay routing • A central entity minimizes the total delay of all network traffic, e.g. Traffic Engineering MPLS

11. 4 7 11 14 3 9 6 12 10 13 1 2 8 5 Simulation Study: Optimal Overlay and Optimal Underlay 14 node tier-1 POP network (Medina et.al. 2002) bimodal normal model of traffic demand 3 overlay nodes Node without overlay Node with overlay Link

12. average delay of all traffic average delay of overlay traffic underlay performance degradation overlay performance improvement percentage % percentage % iteration iteration after underlay takes turn after overlay takes turn Simulation Study ( case 1: 8% overlay traffic) Optimal Overlay and Optimal Underlay Iterative process • Underlay takes turn at step 1, 3, 5, … • Overlay takes turn at step 2, 4, 6, …

13. average delay of all traffic average delay of overlay traffic underlay performance degradation overlay performance degradation percentage % percentage % iteration iteration after underlay takes turn after overlay takes turn Simulation Study (case 2: 10% overlay traffic) Optimal Overlay and Optimal Underlay Iterative process • Underlay takes turn at step 1, 3, 5, … • Overlay takes turn at step 2, 4, 6, …

14. : Cost of “overlay” : Cost of “underlay” : Constraints of “overlay” : Constraints of “underlay” Game-theoretic Study • Two-player non-zero sum game Underlay overlay X: strategy of “overlay” traffic allocation on logical links Y: strategy of “underlay” traffic allocation on physical links

15. Game-theoretic Study • Best-reply dynamics • Nash Equilibrium

16. C A B Optimal Underlay Routing v.s. Optimal Overlay Routing • Overlay • One central entity calculates routes for all overlay demands, given current underlay routing • Assumption: it knows underlay topology and background traffic X(k) 1-X(k) Denote overlay’s routing decision with a single variable X(k): overlay’s flow on path ACB after round k

17. Overlay Routing Evolution Overlay Delay Evolution Underlay’s turn delay Overlay’s turn x(k) x* x(k)<x(k+1)<x* iteration k iteration k Best-reply Dynamics • There exists unique Nash equilibrium x*, • x* globally stable: x(k) x*, from any initial x(1) When x(1)=0, overlay performance improves

18. Best-reply Dynamics • There exists unique Nash equilibrium x*, • x* globally stable: x(k) x*, from any initial x(1) When x(1)=0.5, overlay performance degrades Overlay Routing Evolution Overlay Delay Evolution Underlay’s turn delay Overlay’s turn x(k) BAD INTERACTION! • x(k)>x(k+1)>x* x* x(k)<x(k+1)<x* Round k Round k

19. Conclusions & Open Issues • Selfish overlay routing can degrade performance of network as a whole • Interactions between blind optimizations at two levels may lead to lose-lose situation • Future work: • larger topology: analysis/experimentation • overlay routing and inter-domain routing • interactions between multiple overlays (****) • implications on design overlay routing • regulation between overlay and underlay (****)

20. Second course On the Interaction of Multiple Overlay Routings Performance 2005 Joe W.J. Jiang, D.M. Chiu, John C.S. Lui

21. Questions • These overlays tend to fully utilize available resource. • So, is there any anarchy? • How do overlay networks co-exist with each other? • What is the implication of interactions? • How to regulate selfish overlay networks via mechanism design? • Can ISPs take advantage of this?

22. Outline • Motivation • Mathematical Modeling • Overlay Routing Game • Implications of Interaction • Pricing • Conclusion

23. Adaptive routing controls on multiple layers (overlays, underlay TE --traffic engineering) over one common physical network Simultaneous feedback controls over one system Stability ? Performance ? Motivation • Overlays provide a feasibility for users to control their own routing. • Routing, possible multi-path, becomes an optimization problem. • Interaction occurs (due to same underlay) • Interaction between one overlay and underlay traffic engineering, Zhang et al, Infocom’05. • Interaction between co-existing overlays ?

24. de(le) le– aggregate traffic traversing link e Average delay (f : flow) Performance Characteristics • Objective: minimize end-to-end delay (e.g., RON) • Delay of a physical link e: • Performance Characteristics (Underlay)

25. de(le) le– aggregate traffic traversing link e Average delay (multipath routing) Performance Characteristics • Objective: minimize end-to-end delay • Delay of a physical link e: • Performance Characteristics (Underlay)

26. de(le) le– aggregate traffic traversing link e Average delay (multipath routing) Performance Characteristics • Objective: minimize end-to-end delay • Delay of a physical link e: • Performance Characteristics (Underlay)

27. System Objectives • Network Operators • Min average delay in the whole underlay network • Overlay Users • Min average delay experienced by the overlay

28. How do Overlays Interact? • Overlapping physical links. • Performance dependent on each other. • Selfish routing optimization. • Overlays are transparent to each other. • Lack of information exchange between overlays.

29. Contribution • What is the form of interaction? • Is there routing instability (oscillation), or there is an equilibrium ? • Is the routing equilibrium efficient? • What is the price of anarchy? • Fairness issues • Mechanism design: can we lead the selfish behaviors to an efficient equilibrium?

30. Mathematical Modeling • Overlay routing: An optimization problem • Decision variable: routing policy s:overlay f:flow r:path

31. Routing Matrix Delay Function (vector form) Mathematical Modeling • Overlay routing: An optimization problem • Objective: average weighted delay (matrix form)

32. Capacity Constraint • Demand constraint • (fixed transmission demand) • Non-negative Flow Constraint Overlay Routing Optimization • Convex programming

33. Algorithmic Solution • Unique optimizer • Convex programming • feasible region: convex • delay function: continuous, non-decreasing, strictlyconvex • Solution • Apply any convex programming techniques. • Marginal cost network flow (probabilistic routing ICNP’04). • This is solved in an independent, and distributed fashion by each overlay. But will independent optimization leads to system instability (route flop)?

34. Overlay Routing Game Strategic Game: Goverlay<N, (s), (≥s)> • Nash Routing Game • Player -- N all overlays • Strategy -- s feasible routing policy: feasible region of OVERLAY(s) • Preference relation -- ≥s low delay: player’s utility function is -delay(s)

35. Illustration of Interaction Aggregate traffic on physical links Delay of logical paths in overlay 1 Routing decision on logical paths in overlay 1 Overlay 1 Delay of logical paths in overlay 2 Routing decision on logical paths in overlay 2 Overlay 2 Overlay probing Aggregate overlay traffic … ∑ Routing Underlay Underlay (non-overlay) traffic … Overlay n Delay of logical paths in overlay n Routing decision on logical paths in overlay n

36. Existence of Nash Equilibrium • Definition – Nash equilibrium point (NE) • A feasible strategy profile y=(y(1),…, y(s),…, y(n))T • is a Nash equilibrium in the overlay routing game if for every overlay s∈N, delay(s)(y(1),…y(s),…y(n))≤ delay(s)(y(1),…y’(s),…y(n))for any other feasible strategy profile y’(s).

37. Existence of Nash Equilibrium • Theorem Good News: NO ROUTE FLOP !!! In the overlay routing game, there exists a Nash equilibrium if the delay function delay(s)(y(s) ; y(-s)) is continuous, non-decreasing and convex.

38. Six overlays • One flow per overlay • Congested network • Asynchronous routing update Fluid Simulation

39. Transient period Quick convergence Overlay performance

40. Overlay routing decisions

41. NSR GOR NOR The Price of Anarchy Global Performance (average delay for all flows) Efficiency Loss ? • GOR: Global Optimal Routing • NOR: Nash equilibrium for Overlay Routing Game • NSR: Nash equilibrium for Selfish Routing

42. Selfish Routing • (User) selfish routing: a single packet’s selfishness • Every single packet chooses to route via a shortest (delay) path. • A flow is at Nash equilibrium if no packet can improve its delay by changing its route.

43. Selfish Routing • Also a Nash equilibrium of a mixed strategic game • Player: flow { f } • Strategy: p  Pf • Preference: low delay • System Optimization Problem

44. Performance Comparison

45. Inspiration • Is the equilibrium point efficient (at least Pareto optimal) ? • Fairness issues of resource competition between overlays.

46. 1 unit 1 unit Example Network y1 1-y1 y2 1-y2

47. Sub-Optimality y1 y2 Non Pareto-optimal !

48. Fairness Paradox y1 y2 • a, b, c,  are non-negative parameters • Everything is symmetric except two private links – a & c

49. Fairness Paradox y1 y2 a <c Overlay 1 has a better “private” link !

50. Fairness Paradox y1 y2 Unbounded Unfairness a <c  delay1 < delay2