Network Routing Problem

1. S1 S3 S2 R1 R3 R2 Network Routing Problem • Input: • network topology, link metrics, and traffic matrix • Output: • set of routes to carry traffic B A D C E

2. Network Routing: Classical Approach • Routing as optimization problem • e.g., minimum total delay in network • focus on global network performance (social optimal) • performance of individual user not important • Centralized or distributed algorithms • e.g., link state or distance vector • Passive users • users are oblivious to routing decisions

3. Network Routing: Game-Theoretic Approach • Routing as game between users • users determine route • decision based solely on individual performance (selfish routing) • strongly dependent on other users’ decisions • Non-cooperative game (non-zero sum) • users compete for network resources • Equilibrium point of operation • Nash equilibrium point (NEP)

4. Selfish Network Routing • Advantages • no need of centralized control or global agreement on routing algorithm • individual user’s performance considered • greater adaptability • changes in user demands or changes in network conditions • Disadvantages • multiple equilibria (eq. selection problem) • convergence to equilibrium • no network-wide optimality at equilibrium • cost of “selfish routing” • user’s must have detailed knowledge of network

5. Applications of Game Theory to Network Routing • Competitive routing in multiuser communication networksA. Orda, R. Rom and N. ShimkinIEEE/ACM Transactions on Networking, 1 (5) 1993 • How bad is selfish routing?T. Roughgarden and E. TardosJournal of the ACM, 49 (2) 2002 • Selfish routing with atomic playersT. RoughgardenACM/SIAM Symp. on Discrete Algorithms (SODA) 2005

6. S1 S2 S3 S4 R1 R2 R3 R4 Simple Model: Network of Parallel Links • set of users share a set of parallel links • each user has fixed demand (data rate) • users decide how to split demand across links • minimize individual cost • link has a load dependent cost (e.g., delay) A B

7. Network of Parallel Links • set of parallel links: • set of users: • each user has a fixed demand (data rate): • user splits its demand across links • flow of user i on link l: • flow configuration of user i: • system flow configuration: • feasible configurations • satisfy nonnegative and demand constraints

8. Very mild conditions on cost function User’s Cost Function • Cost function of user i: • cost depends on flow configuration of all users • Assumptions on cost function • sum of user-link cost function: • can be infinite • convex in • when finite, continuously differentiable in • at least one user with infinite cost can change its flow configuration to have finite cost • aggregate demand must be less than aggregate link capacity

9. No user can reduce their cost by rerouting their own flow The Game • Users individually decide their flow configuration • goal is to minimize its own cost • Nash Equilibrium Point (NEP) • system flow configuration such that no user can reduce their cost by changing its flow allocation • is a NEP if for all i, the followingholds:

10. The Issues • Existence of NEP • is at least one NEP guaranteed to always exist • Uniqueness of NEP • under which conditions (if any) do we have a single NEP • Convergence to (and stability of) NEP • play dynamics that lead to a NEP • System properties at the NEP • e.g., how does users divide allocate their flows

11. N-person convex game [Rosen65] joint strategy set is convex, closed and bounded each player’s payoff function is convex in their own strategy existence of NEP proven by Katutani fixed point theorem Can also show using Kuhn-Tucker conditions necessary and sufficient for system flow configuration to be a NEP Existence of NEP

12. Uniqueness of NEP only under a type of cost functions (type-A functions) cost function has two parameters: user’s i and aggregate of all others monotonically increasing in each parameter still very general (e.g., M/M/1 delay function) Proof by contradiction using Kuhn-Tucker conditions System has a single operating point Uniqueness of NEP

13. Intuitive but assuring properties System Properties at NEP • Assume all users share same type-A cost function • but users can have different demands • Monotonicity of link usage • user with higher demand uses more of each and every link used • a user with higher demand uses more links • Higher capacity links receive more users • does not hold in general, only under yet another type of cost function (which still captures M/M/1)

14. S1 R1 A B S2 R2 Dynamical System • Simple case study • two-users sharing two parallel links • Dynamical model: Elementary Stepwise System • Users take turns in updating their flow configuration • measure load on links, adjust its flow to minimize cost • flow of user i on link l at step n

15. Convergence to NEP • Let denote unique NEP of game • Initialize system with any feasible flow configuration: f(0) • Convergence to NEP guaranteed • Framework used in proof not aplicable in general • limited to two link, two user structure

16. S1 S3 S2 R1 R3 R2 General Topology • Users decide how to split their demands over possible paths • users know network topology (directed graph) B A D C E

17. Analysis of general network in this modeling framework is much harder Existence and Uniqueness of NEP • Existence of NEP • same argument as before (N-person convex game) • No unique NEP for type-A cost functions • shown by counterexample • Uniqueness shown only under very strict conditions for cost function • not very interesting networking scenarios

18. The “Price of Anarchy” • Equilibria of non-cooperative games usually inefficient • e.g., prisoner’s dilemma • Pareto optimal usually not a NEP • Quantify inefficiency in terms of a global objective • “price of anarchy” (coordination versus competition) objective function value at NEP Price of Anarchy of a Game = optimal objective function value • if multiple NEP exists, take sup (or inf) over NEP set

19. Cost of Selfish Routing • How does total cost compare? • flow allocation at a NEP • optimal flow allocation • Total cost of flow configuration: • where is load dependent link cost function • e.g., link delay

20. Example (1/4) r1 = 0.5 S1 R1 A B • flow configuration cost • optimal flow allocation • can be realized with r2 = 0.5 S2 R2

21. Example (2/4) r1 = 0.5 S1 R1 A B • But this is not NEP… • Cost of a flow configuration to user i r2 = 0.5 S2 R2 • By rerouting traffic user 1 (or 2) can reduce its cost: lower cost!

22. Example (3/4) r1 = 0.5 S1 R1 A B • NEP given by • link 1 is a dominant strategy (link 2 never used) • Cost to user i at NEP • Total cost of NEP configuration r2 = 0.5 S2 R2 higher cost! higher cost

23. Example (4/4) r1 = 0.5 S1 R1 A B • Optimal cost: • NEP cost: • Price of Anarchy: r2 = 0.5 S2 R2 Thm:[Roughgarden/Tardos00] POA of selfish routing w/affine cost functions is at most 4/3 • for any network topology and traffic matrix!

24. Another example (non-linear cost)… r1 = 0.5 S1 R1 • NEP: both users only use link 1 • cost is 1 • Optimal: 1-ε for link 1 and ε for link 2 • ε depends on d, but is small for large d • cost ≈ 0 • Price of anarchy can be arbitrarily large • goes to infinity as d goes to infinity A B r2 = 0.5 S2 R2

25. So how bad is selfish routing? • It depends... • cost functions, network topology, traffic matrix, user demands, etc. • In reality, not so bad • achieves close to optimal cost in Internet-like environments (simulation study) • Another positive (and nice) result: Thm:[Roughgarden/Tardos00] selfish routing is no worst than the optimal routing of twice as much traffic • for any cost function, network topology and traffic matrix!

26. Title

27. S1 S3 S2 R1 R3 R2 Congestion Control Problem B • Input: • network topology, routes, link characteristics, traffic matrix • Output: • set of data rates to be used A D C E

28. Congestion Control: Classical Approach • Congestion control as optimization problem • match user’s demand to network capacity and achieve some fairness among users • focus on global network performance (social optimal) • performance of individual user not important • Centralized or distributed algorithms • e.g., TCP, max-min fairness • Passive users • users are oblivious to congestion decisions

29. Congestion Control: Game-Theoretic Approach • Congestion control as game between users • users determine their own data rates • decision based solely on individual performance • Non-cooperative game (non-zero sum) • users compete for network resources • Equilibrium point of operation • Nash equilibrium point (NEP) Key Assumption: A higher sending rate do not necessarily yields better performance for user

30. Routing Games vs Congestion Control Games • Routing games • users determine network routes • multi-path routing and traffic splitting is possible • users’ data rates are given and must be routed • Congestion games • users determine their data rate • network routes are given (single path)

31. Applications of Game Theory to Congestion Control • Making greed work in networks: a game-theoretic analysis of switch service disciplinesS. ShenkerIEEE/ACM Transactions on Networking, 3 (6) 1995 • An evolutionary game-theoretic approach to congestion controlD. Menasché, D. Figueiredo, E. de Souza e Silva Performance Evaluation, 62 (1-4) 2005

32. S1 S2 S3 S4 R1 R2 R3 R4 Simple Model: Single Bottleneck Link • set of users share a bottleneck link • users decide their data rates • maximize individual performance • user’s performance depends on link load • e.g., quality of service provided by link

33. Single Bottleneck Link • Users determine sending rate: • Link modeled as M/M/1 queue • unit capacity • packet scheduling policy • Scheduling policy induces average queue length for each user • : avg. queue length of user i • User’s utility function • strictly increasing in • strictly decreasing in • convex and derivable everywhere

34. Scheduling Policy • Determined by system operator • Allocation function • scheduling policy P induces an avg. queue length for each user given all user’s data rate • FIFO example • Must satisfy some constraints • aggregate average queue size same as M/M/1 • Allocation function can be realized by different service disciplines

35. Fair Share Allocation • Allocate service capacity fairly among user’s demand • user’s requesting less obtain higher priority • Implemented through a priority queueing algorithm • Assume:r1 <…<rN fraction of traffic gets lower priority

36. MAC: Set of Monotonic Allocation Functions • Consider a set of possible allocation functions • : increases, increases • : increases, does not decrease • Includes all typical service disciplines • FIFO, LIFO, PS, fair share allocation at for all with rk  rok

37. System designer can select service discipline that yields good equilibrium The Problem Investigated • Relationship between NEP and service disciplines (MAC functions) • Which service disciplines yield good NEP? • Properties of NEP of a given MAC • efficiency • fairness • convergence to equilibrium • user protection

38. Efficiency of NEP • Efficiency in terms of Pareto optimal • no global objective function of system outcome • Pareto optimal outcome: • no other outcome is preferred by all users Thm:[Shenker95] There is no allocation function in MAC such that every NEP is Pareto optimal • Under some additional constraints fair share is always efficient • constrained users’ utility function • symmetric rate vector

39. Uniqueness of NEP • Allocation functions can induce multiple NEP • undesirable since users cannot coordinate Thm:[Shenker95] • Fair share mechanism always has a unique NEP • Fair share is the only allocation function that always yields a unique NEP

40. Convergence to Equilibrium • Dynamics through a generalized hill climbing algorithm • users eliminate strategies that always perform worst • system converges to a reduced set of strategies • Different from best-response dynamics Thm:[Shenker95] • With the fair share mechanism, all generalized hill climbing algorithm converges to the NEP • Convergence is also fast (superlinear) and stable

41. Title

42. S1 R1 R2 R3 R4 Investigate dynamics and convergence using evolutionary game theory Application to Multimedia Traffic • Users share common bottleneck link • User’s choose data rate to be sent by source • only few data rates available • Utility given by “perceived” quality

43. Why Evolutionary Game Theory • Model how users change their strategy • Users are not perfect: stochastic dynamics, myopic, etc • Which NEP will be achieved (if more than one exists) • Efficiency of selected NEP Evolutionary Game Theory

44. Entities of Model and Interactions yields perceived quality to performance metric feeds Link model(M/M/1/k or other) QoS Model(E-model or other) Users(strategy set) choice of strategies causes impact on

45. Two-layer Markovian Model layer 1 users’ actions 3, 0 2, 1 1, 2 0, 3 link perf. layer 2 QoS of each user QoS of each user QoS of each user QoS of each user

46. States and Users Utility : number of users selecting strategy l in state : number of data rates available to users : state : utility function of strategy l in state • No constraints on users’ utility function • should be defined for every state

47. Transition Matrix • Transitions determined by QoS in each state • rate of change proportional to gain • transitions can reduce QoS (users make errors) • Markov chain is ergodic

48. Main Problem Investigated Assume: • System in steady state • Users make no mistakes States that correspond to NEP States that have non-negligible steady state probability What is the relationship?

49. Proposition 1 • under the condition that this state is contained in a quasi-absorbing set This state is also a NEP If a state has non-negligible steady state probability

50. Proposition 2 • proof via simple counter-example This state also has non-negligible steady state probability If a state is a NEP