Intrinsic Robustness of the Price of Anarchy “Smoothness”

Intrinsic Robustness of the Price of Anarchy“Smoothness” Ofir Chen for 2014 POA Seminar by Prof. Michal Feldman Paper by Tim Roughgarden ‘13

Presentation divides into 3 parts: Pure NE case, extensions to other equilibrias ,and a proof of tightness of the result for congestion games. • We’d like to bound the cost of unilateral deviation from any strategy to any other strategy. • Comparing any 2 strategies bounds the POA. • Bounding all outcomes has implications beyond games with pure-NE (PNE) – hence the intrinsic property. • smooth game reduces to a robust POA automatically. • Finally we’ll see that the achieved bound is tight for Congestion Games. Introduction

Definition: smoothness: a cost minimization game is ( if for any 2 outcomes and : Smoothness in Pure-NE

Smoothness in Pure-NE • Pure-NE key equation: • Immediate result for smooth games: : • Definition: Robust POA : smoothness NE

Definition Congestion Game: a cost minimization game that has a ground set E, strategy sets , anda non-decreasing cost function . • Example: • Non-atomic-selfish routing: -edges, are paths on the graph. • : the set of edges used by player • : the load on edge • Player- cost under strategy Claim: the game is -smooth ( Ex-1: Congestion Games

Ex-1: Congestion Games (Cont.) • Proof: • Mathematical fact (without proof) for : • For outcomes (*) (*)The load on e in is at most +1 than in under unilateral deviation and it is suffered by exactly players

Ex-2: Valid Utility Games • Definition: Valid Utility Games: • Payoff maximization, payoff functions • Ground set , strategies • Non-negative submodular function • Definitionsubmodular: • Target function: while • 2 Conclusions follow: (1) (2) relaxation on sum objective functions • Example: players are trying to win ‘red’ tokens on a board by placing their tokens – each red token goes to closest token’s player. The payoff is the number of red tokens.

Ex-2: Valid Utility Games(Cont.) • Claim: if is non-decreasing, VUGs are , with POA=0.5 • Note: redefine smoothenessin max-payoff games : • . • Proof: • Lbe the union of all players' strategies in, together with the strategies of players 1…in :], Telescopic summation s Submodularity

Other Equilibriums Coarse Correlated Equilibriums Coarse Correlated Eq Correlated Equilibriums Easy to compute = Correlated Eq Mixed NE Always exist Hard to compute = MNE = PNE Pure-NE Don’t always exist

Model: Players can play their strategies with some probability. • Definition: Mixed NE: no player can decrease its expected cost under the product independent distributions over players’ strategies • Lemma: Smoothness of distribution:for a ( , )-smooth game, when is an independent outcomes distribution: for any Mixed Strategies Games

Mixed Strategies Smooth Games • Proof : • POA: for every MNE: Smoothness (*) (*) MNE linearity (*)'s distribution doesn't matter for since is fixed.

Model: Players’ strategies are correlated, players are getting a ‘hint’. • Definition: Correlated Equilibrium (CE): no player can decrease its expected cost given a posterior ‘hint’ : • Lemma: if a game is ( , )-smooth, – Correlated outcomes distribution then for any • Proof and POA: MNE proof follows for any joint distribution! Correlated Strategies

Model: Players’ strategies are correlated (no ‘hint’). • Definition: Coarse Correlated Equilibrium (CCE): no player can decrease its expected cost • Lemma: if a game is ( , )-smooth, – Correlated outcomes distribution then for any • Proof and POA follow similarly. Coarse-Correlated Strategies

Congestion game: 6 singleton strategies: (players pick 1 resource) • Pure NE: each of the options for mutex choices. • Mixed NE: pick uniformly - the expected cost is . • Correlated eq.: pick uniformly– ‘hints’: 1 edge with 2 players and 2 edges 1 player each: expected cost is . • Coarse Correlated eq.: sets are: {0,2,4} or {1,3,5} w.p ½. Expected cost is 3/2. Ex-3: Inclusion of Equilibriums 4 1 2 3 1 2 2 hints 1 3 4 3 4

Model: a sequence is a game divided into time-intervals - each with its own distribution over outcomes: ~ • Example: Sequential Games

Model: sequential game without ‘too much’ regret in hindsight. • Definition: regret is the cost over all intervals minus the cost of the best fixed strategy in hindsight. • Definition: a no regret sequence is a sequence of distributions over outcomes where the total expected cost of each player is lesser than that of the best fixed strategy in hindsight, by at most : No Regret Sequence

Definition: Price of totalanarchy is a WC ratio between the expected cost of no-regret sequence that of an optimal outcome. PoTAUnder Smoothness: • if is optimal, define: • When the game is smooth: • Averaging over T will give us the PoTA: Price of TotalAnarchy smoothness o(1)

Congestion Games are Tight for the smoothness-bound

Definition: Tight Games: • Let denote all legal values of ( ,)-Smooth games in • Let be the games with at least 1 pure-NE • For a game Let be the POA of pure-NE • a class of games is tight if • Idea: characterize games by restricting their cost functions. Congestion Games are Tight the best upper bound provable through smoothness The WC POA of pure-NE Theorem: for a set of nondecreasing positive cost functions, the congestion games with cost functions in are tight.

: a set of congestion games with cost functions in . • all legal values of ( ,)-Smooth games in for every and for every . • Note: is the best POA achievable through smoothness. • to be the cost function of resource • Denote /to be the number of players using resource in outcomes respectively. Notations

Claim: for every setthe robust POA of is at most . • Proof: consider a game and outcomes . We'll show that is -smooth. • Hence the POA upper bound is achieved. Upper Bound For each player at most one deviation is made Definition of

Definitions: • Denote the pairs with . • Denote • Note: there’re -pairs, hence linear 2 variableequationsrepresenting half-plains • Lemma-1: for a finite and , if such that - then is the intersection point of 2 half plains . • Mathematically: there exist such that: Lower Bound

Proof: by inspecting a minimized • Note: the problem resembles LP,with non-linear target function. • The half-planes create a convex hull • increases in • The minimal is on the convex! • On the convex boundary, for every • every has its values. • increases in when Lower Bound Geometric Proof (0,0,0)

Take a down-walk on the convex hull: • When go down, otherwise – stop. • That will assure is minimized. • is on the convex on 2 intersecting half-planes: • Lemma-2: for a finite and , if there exist such that -then there exist such that • Proof: for the above Lower Bound Geometry (Cont.) is min + Q.E.D

Theorem: for every cost functions set , there’re congestion games with cost functions in with pure POA arbitrary close to • Proof: Define by following the lemmas: • - each a labeled ‘cycle’ with elements • ’s cost function: and ’s cost function: • players – each with 2 strategies and • = use consecutive elements of and consecutive elements of starting the th element of each cycle. • = useconsecutive elements of andconsecutive elements of ending with the 'th element of each cycle. • ’s size assures and are disjoint. Building the Game … … … …

Denote - the outcome of , the outcome of . • By symmetry : , and :, Building the Game (Cont.) symmetry Lemma 2 Q and P are disjoint Conclusion– the game is in a pure-NE

Repeating the Proof steps: • The lemma told us where to find candidates. • We built a game according to the lemma and found it has valid • In this game the pure-POA is , hence the supremum is higher – and we proved the lower bound. The POA of the Game Lemma 1 Conclusion– is a valid pair in

Questions?

Model: Myopic response, if the game is not in equilibrium, choose a non-optimal player and improve his cost. • The game will converge if it has a potential function satisfying: • Claim: Let be a best-response sequence of a smooth game with robust-POA , s* is optimal. • denote the deviating player at time t . • Ifsatisfies Then w.h.p.: all but of outcomes satisfies . We’ll not show the Proof here Best-Response Dynamics (BRD)

Model: a game where players are duplicated-times . • Example: routing: existing infrastructure, various amount of identical players. • Denote a superposition of 2 outcomes. We assume • Lemma: if G is smooth with duplicated players, then • Proof: Bicriteria Bounds NE smoothness superposition

Intrinsic Robustness of the Price of Anarchy “Smoothness”