A Projection Framework for Near-Potential Polynomial Games

A Projection Framework for Near-Potential Polynomial Games IEEE CDC Maui, December 13th 2012 Nikolai Matni (nmatni@caltech.edu) Control and Dynamical Systems, California Institute of Technology

Motivation – Potential Games • Informal definition: local actions have predictable global consequences. • Nice properties • Pure-strategy Nash Equilibria (NE) • Simple dynamics converge to these NE • Applications to distributed control • Marden, Arslan & Shamma 2010 • Candogan, Menache, Ozdaglar& Parrilo 2009 • Li & Marden, 2011

Motivation – Polynomial Games • Would like to consider general class of continuous games • Finite players, continuous action sets. • Why? • Goal is control: most systems of interest are analog. • Quantization leads to tradeoffs in granularity, performance and problem dimension. • Why not? • Potentially intractable to analyze (Parrilo 2006, Stein et al. 2006 for recent results). • Can lead to infinite dimensional optimization problems. • Solution? • Restrict ourselves to polynomial cost functions and use Sum Of Squares (SOS) methods.

Motivation – Near Potential Games • O. Candogan, A. Ozdalgar, P.A. Parrilo, A Projection Framework for Near-Potential Games, CDC 2010 (and subsequent work) • Basic idea: if a game is “close” to being a potential game, it behaves “almost as well.” • Projection Framework – finite dimensional case • Potential games form a subspace. • Project onto this framework to find closest potential game. • If distance from subspace is small, original game inherits many nice properties. • Goal: Extend these ideas to polynomial games.

Outline • Motivation • Potential games • Polynomial games • Near-Potential games • Preliminaries • Game Theory • Algebraic Geometry/Sum of Squares (SOS) • Projection Framework • Properties • Static • Dynamic • Example • Conclusions and Future work

Prelims – Polynomial Game • A polynomial game is given by: • A finite player set • Strategy spaces , where • Polynomial utility functions , • A polynomial game is: • Continuous if for all n, is a closed interval of the real line • Discrete if for all n, • Mixed if some strategy sets are continuous, and some are discrete. • Assume w.l.o.g.

Prelims – Potential Games • A polynomial game G is a polynomial potential game if there exists a polynomial potential function such that, for every player n, and every • Algebraic characterization (Monderer, Shapley ’96): A continuous game is a potential game iff

Prelims – Misc. Game Theory • A strategy is an approximate Nash (or ε) Equilibrium if, for all n, we have that

Prelims – SOS and p(x)≥0 • Definition: a real polynomial p(x) admits a Sum Of Squares (SOS) decomposition if • Why SOS? • Determining if p(x)≥0, is in general, NP-hard • Determining if p(x) is SOS tested through SDP • Lemma [SOS relaxation]: If there exist SOS polynomials such that then

Projection Framework – MPD & MDD • Need a notion of distance in the space of games • Candogan et al. introduced Maximum Pairwise Distance (MPD) • Use the continuity of polynomials to define Maximum Differential Difference (MDD) • Both capture how different two games are in terms of utility improvements due to unilateral deviations

Projection Framework • Task: Given a polynomial game , find a nearby potential polynomial game • Formulate as an optimization problem: • Constraint ensures we get a Potential Game • Objective function minimizes MDD. • Intractable!

Projection Framework – Convexify • Step 1: rewrite constraint in terms of algebraic characterization • Step 2: introduce slack variable γ

Projection Framework – Convexify • Step 3: apply Lemma [SOS relaxation] • This is a finite dimensional SOS program, solvable in polynomial time. It yields a polynomial potential game satisfying

Projection Framework - Extensions • Can extend this idea to mixed/discrete games • Lemma [MPD]: If , then • Continuous Relaxations: For a mixed or discrete game, set all strategy sets to [-1,1] • Apply previous SOS program and Lemma [MPD] to mixed games or discrete games with • Allows us to apply algebraic characterization, which can reduce number of constraints from O( ) to O(N)

Properties – Static • Let and be such that . Then for every ε1-equilibrium y of , z(y) is an ε-equilibrium of , where • For continuous games, D=0, z(y)=y, and local maxima of P are pure (ε=0) NE.

Properties – Dynamic • Definition: ε-better response dynamics • Round robin updates • Player updates only to improve utility by at least ε • Otherwise does not update • Suppose there exists such that Then, under ε-better response dynamics, after a finite number of iterations, dynamics will be confined to the ε-equilibria set of , for arbitrary.

Example – Distributed Power • Consider the N player game defined by • Distributed power minimization interpretation

Example – Distributed Power • Run through projection framework to find nearby potential game : satisfying

Example – Distributed Power • Potential function concave – can compute global maximum to identify .2-equilibria of G • Alternatively, can run .2-better response dynamics to converge to a .2-equilibria of G. • Quantify performance through cost function

Example – Distributed Power • Compare better-response (xbr) to centralized (optimal x*) positions • Better response comeswithin ~20% of centralized solution • Completely decentralized • Arbitrarily scalable • Requires no a prioriknowledge of basestation locations

Conclusions & Future Work • Introduce framework for analyzing polynomial games • Defined MDD and a tractable projection framework to find nearby potential games • Related static and dynamic properties of polynomial games to those of nearby potential games • Illustrated these methods on a distributed power problem • Future work • Projecting onto weighted polynomial games • Additional static properties (mixed-equilibria) • Efficiency notions (price of anarchy, price of stability, etc.)

A Projection Framework for Near-Potential Polynomial Games