Game Dynamics Out of Sync

1. Game DynamicsOut of Sync Michael Schapira (Yale University and UC Berkeley) Joint work withAaron D. Jaggardand Rebecca N. Wright

2. Sprint AT&T Comcast Qwest Motivation: Internet Routing Establish routes between Autonomous Systems (ASes). Currently handled by the Border Gateway Protocol (BGP).

3. Internet Routing as a Game[Levin-S-Zohar] • Internet routing is a game! • players = ASes • players’ types = preferences over routes • strategies = outgoing edges • BGP = Best-Response Dynamics • each AS constantly selects its best available route to each destination • … until a “stable state” (= PNE) is reached.

4. But… this talk • Challenge I: No synchronization ofplayers’ actions • players can best-reply simultaneously. • players can best-reply based on outdated information. • Challenge II: Are players incentivized to follow best-response dynamics? • Can a player benefit from not best-replying? [Nisan-S-Valiant-Zohar]

5. Game Dynamics and Asynchrony • Dynamic environments • Internet protocols • large-scale markets • social networks • multi-processor computer architectures • Game theory provides useful tools to analyze these interactions, but…. • … has so far primarily concentrated on synchronous environments (simultaneous, sequential).

6. Illustration Column Player 0,0 2,1 Row Player 0,0 1,2

7. Illustration Column Player 0,0 2,1 Row Player 0,0 1,2

8. But… Column Player 0,0 2,1 Row Player 0,0 1,2

9. Agenda • Model for asynchronous game dynamics • Impossibility result • Circumventing our impossibility result • Complexity of asynchronous game dynamics • Directions for future research

10. Simple Model: Nodes Interacting • n nodes 1,…,n • Node i has action spaceAi • A=A1•…•An • A-i=A1•…•Ai-1•Ai+1•…•An • Node i has reaction functionfi:A→Ai • f=(f1,…,fn)

11. Simple Model: Dynamics • Infinite sequence of discrete time steps t=1,… • Initial state a0, Schedules:{1,…} →2[n] • fair schedule • The (a0,s)-dynamics • Start at the initial state a0 • In each time step t let the nodes in s(t) react.

12. Simple Model: Convergence • Defn: an action profile a=(a1,…,an) is a stable stateif fi(a)=ai for all i. • that is, a is a fixed point of f. • Defn: The system is convergent if the (a0,s)-dynamics converges to a stable state for all choices of a0 and (fair) s.

13. Guaranteed Convergence? • Defn: f is node independent if, for each node i, fi:A-i→Ai • Thm: If f is node independent, and there exist multiple stable states, then the system is not convergent. • Can be generalized to reaction functions that are • randomized • bounded-recall • non-stationary

14. Applications • Internet protocols • Internet routing [Sami-S-Zohar] • congestion control [Godfrey-S-Zohar-Shenker] • Best-response dynamics • with consistent tie-breaking • orthogonal to the results of Hart and Mas-Colell • Diffusion of technologies in social networks • 2 technologies {A,B}. Each node wants to be consistent with the majority of its neighbours. • Circuit design

15. “Tightness” of Our Result • Example 1: (node-dependent reactions)Each fi is such that for every a=(a1,…,an) it holds that fi(a)=ai.

16. “Tightness” of Our Result • Example 1: (node dependent reactions)Each fi is such that for every a=(a1,…,an) it holds that fi(a)=ai. • Example 2: (unbounded recall) • 2 nodes, 1 and 2, each with action space {a,b}. • Node 2 wants to match node 1’s action. • Node 1 selects b if node 2 changed its action from a to b in the past, and a otherwise. • What happens at the initial state (b,a)?

17. Proving Our Result • Thm: If f is node independent, andthere exist multiple stable states, thenthe system is not convergent. • Interesting connections to fundamental results in distributed computing theory. • the Fischer-Lynch-Patterson impossibility result for consensus protocols (1983) • But, neither result is a special case of the other.

18. The Dynamics Graph State R aR:=aS except aRi:=fi(bS) bR:=bS i-transition State S knowledge transition State T action vector aS=(aS1,… aSn)knowledge vector bS=(bS1,… bSn) aT:=aS bT:=aS

19. Visualising Dynamics • The dynamics graph captures all dynamics. • The scenario where • the initial state is a0. • nodes 1 and 3 react simultaneously. • then nodes 2 and 3 react simultaneously. is captured as follows:

20. Visualising Dynamics • The dynamics graph captures all dynamics. • The scenario where • the initial state is a0. • nodes 1 and 3 react simultaneously. • then nodes 2 and 3 react simultaneously. is captured as follows: State SaS=bS=a0

21. Visualising Dynamics • The dynamics graph captures all dynamics. • The scenario where • the initial state is a0. • nodes 1 and 3 react simultaneously. • then nodes 2 and 3 react simultaneously. is captured as follows: State SaS=bS=a0 1-transition 3-transition k-transition

22. Visualising Dynamics • The dynamics graph captures all dynamics. • The scenario where • the initial state is a0. • nodes 1 and 3 react simultaneously. • then nodes 2 and 3 react simultaneously. is captured as follows: State SaS=bS=a0 1-transition 3-transition k-transition 2-transition 3-transition k-transition

23. Stability and Fairness • Defn: A state S in the dynamics graph is stable if every outgoing edge from S leads to S. • Defn: A fair pathin the dynamics graph is a path that (1) for each i, contains an i-transition; and (2) also contains a knowledge transition.

24. Attractor Regions • Defn: The attractor region of a stable state S are all states from which any (long enough) fair path reaches S.

25. Proof Sketch (Cont.) • Claim: A fair cycle in the dynamics graph implies an oscillation in our model. • Proposition: If there are multiple stable states then there are states in the dynamics graph that are not in any attractor region (“neutral states”).

26. Colouring States • Colour each attractor region in a different colour – red, blue, etc. • Colour the neutral states in purple.

27. Creating Oscillations • Key idea: We show that from every purple state there is a fair path that leads to another purple state. • The number of purple states is finite and so this implies a fair cycle.

28. Proof Sketch (Cont.) • Lemma: There cannot be two edges leading from a purple state to two non-purple states that do not have the same colour. • Intuition: We can swap the order of activations without affecting the outcome. a b a,b : different transitions a b ?

29. Proof Sketch (Cont.) • Fix a purple state p. • Let R be a “maximal” fair path from p to another purple state. R q … … p

30. Proof Sketch (Cont.) • Let a be a transition that is not on R. • Observe that a at q takes us to a non-purple state. R q … … a p

31. Proof Sketch (Cont.) • Because q is purple it must have a fair path to a non-purple non-red state. R u … … q … … a p

32. Proof Sketch (Cont.) • Now, we prove that a at u must take us to a red state --- a contradiction! R u … a … q … … a p

33. Circumventing Our Impossibility Result: Randomness • Our result holds for randomized reaction functions. • adversarially-chosen schedule • What if the schedule is randomized? • our impossibility result breaks … • … but no general possibility result either

34. Circumventing Our Impossibility Result: r-Fair Schedules • Defn: A schedule s is r-fair if each node is activated at least once within every r consecutive time steps. • Can we prove our impossibility result for schedules that are r-fair? If so, for what values of r? • We present positive and negative results.

35. Complexity Results • Thm: Determining if a system with n nodes, each with two actions, is convergent requires exponential communication (in n). • The proof requires reaction functions to be of exponential size. • Combinatorial proof: a “Snake in the Box” construction

36. Complexity Results • What if the reaction functions can be succinctly described? • Thm: Determining if a system with n nodes is convergent is PSPACE-Complete. • Hence, there is no “short” characterization of asynchronous convergence!

37. Directions for Future Research • Other notions of asynchrony • Other reaction functions • fictitious play, regret minimization • Observation: regret minimization is much more resilient to asynchrony (different framework…). • Other restrictions on schedules • random schedules • r-fair schedules • more

38. Thank You