430 likes | 615 Views
The Computational Complexity of Finding Nash Equilibria. Edith Elkind Intelligence, Agents, Multimedia group (IAM) School of Electronics and CS U. of Southampton. Games and Strategies. Games: strategic interactions between rational entities Solution concepts: what’s going to happen?
E N D
The Computational Complexityof Finding Nash Equilibria Edith Elkind Intelligence, Agents, Multimedia group (IAM) School of Electronics and CS U. of Southampton
Games and Strategies • Games: strategic interactions between rational entities • Solution concepts: what’s going to happen? • dominant strategies • Nash equilibrium • …. • Can it be computed? • if your computer cannot find it, the market probably cannot either
0 0 1 1 0 0 1 1 Matrix (Normal Form) Games • finite set of players {1, …, n} • each player has kactions (pure strategies): 1, …, k • payoffs of the ith player: Pi: {1, …, k}n→ R Row player: Column player:
0 0 1 1 0 0 1 1 Nash Equilibrium • Nash equilibrium: a strategy profile such that noone wants to deviate given other players’ strategies, i.e., each player’s strategy is a best response to others’ strategies: • (0, 0) and (1, 1) are both NE Row player: Column player:
H H T T H H T T Pure vs. Mixed Strategies • NE in pure strategies may not exist! • “matching pennies” • Mixed strategy: a probability distribution over actions • 50% tail, 50% head Row player: Column player:
Existence of NE • Theorem (Nash 1951): any n-player k-action game in normal form has an equilibrium in mixed strategies can we find one in poly-time?
other cool stuff n players, 2 actions Plan of the Talk 2 players, k actions
2 (r=const) players, k actions • Input representation: • 2 players: twok x k matrices • r players: rk x k x … x k matrices • poly-size for constant r • Output representation: • for 2 players all NE are in Q • but not for 3 and more players… • Checking for pure NE: easy • at most k2 strategy profiles
2 players, k actions: mixed NE • Naïve approaches: exp(k) • Simplex-like approach (Lemke-Howson algorithm): • works well in practice • exp(k) in the worst case (2004) • Is it time to give up? • maybe the problem is NP-hard?
Is Finding NE NP-hard? • Reminder: a problem P is NP-hard if you can reduce 3-SAT to it: • “yes”-instance 3-SAT→ “yes”-instance of P • “no”-instance 3-SAT→ “no”-instance of P • Problem: each instance of NASH is a “yes”-instance! • every game has a NE • Formally: if NASH is NP-hard then NP = coNP • Need: complexity theory for total search problems
g f • S is reducible to T if: • f, g easy to compute • g(T(f(x))) is in S(x) Reducibility Among Search Problems • S associates x in X with a solution set S(x) • Total search problem: for any x, S(x) is not empty S: X Y T: X’ Y’ If T is easy, so is S
Completeness Results? • Can we prove that any total search problem is reducible to r-NASH? • Not really: the class T of all total search problems is a semantic class • not known how to find complete problems for these • Want to pick a large subclass S of T s.t. • S includes some natural problems • there are problems that are complete for S • in particular, r-NASH is complete for S
END OF THE LINE • Input: Boolean circuits S (Successor), P (Predecessor): • n inputs, n outputs • S(0n) ≠ 0n, P(0n) = 0n • Output: x ≠ 0n s.t. • S(P(x)) ≠ x or P(S(x)) ≠ x Intuition: G=(V, E): • V = Sn; • E = {(x,y) | y=S(x), x=P(y)} 00000 11001 01011 01011
PPAD • PPAD: Polynomial Parity Argument, Directed version • PPAD is the class of all search problems that are reducible to END OF THE LINE search problem solution g f circuits S, T “end of the line”
r-NASH is in PPAD • Proof on Nash’s theorem: • existence of NE reduces to Brouwer’s fixpoint theorem • Brouwer’s fixpoint theorem reduces to Sperner’s lemma • Sperner’s lemma is proven by a parity argument (similar to END OF THE LINE) • Reduction of r-NASH to END OF THE LINE can be extracted from these proofs (Papadimitriou 94)
Brouwer’s Fixpoint Theorem • Brouwer’s Theorem: Any continuous mapping from the simplex to itself has a fixpoint. • Nash Brouwer proof sketch: • set of all strategy profiles → simplex • mapping: (s1, …, sn) → (s1+d1, …, sn+dn), where diis a shift in the direction of best response to (s1, …, si-1, si+1, …, sn) • NE is a point where noone wants to deviate, i.e., a fixpoint
B C A Sperner’s Lemma • Proper coloring: • vertices on BC are not blue • vertices on AC are not green • vertices on AB are not yellow • Sperner’s Lemma: there exists a trichromatic triangle • Brouwer’s theorem Sperner’s Lemma: • x is blue if the grad(F) at x points away from A, etc. • trichromatic triangle “has no direction” • repeat at increased resolution…
Reductions (Papadimitriou 1994) END OF THE LINE is PPAD-complete TRICHROMATIC TRIANGLE is PPAD-complete 3D-BROUWER is PPAD-complete r-NASH is in PPAD
r-NASH vs 3D BROUWER • Existence of NE follows from Brouwer’s fixpoint theorem • NE are special cases of Brouwer’s fixpoints • just how special? • Can any fixpoint be represented as a NE of a game? • Is there a reduction from 3D BROUWER to r-NASH?
Hardness Reductions: the Timeline • 3D-BROUWER is PPAD-complete (Papadimitriou, 1994) • 4-NASH is PPAD-complete (Daskalakis,Goldberg, Papadimitriou, Sep 2005) • 3-NASH is PPAD-complete (Daskalakis, Papadimitriou, Oct 2005, Chen, Deng, Nov 2005) • 2-NASH is PPAD-complete !!!(Chen, Deng, Dec 2005)
n players, 2 actions • representation: payoffs to each player for every action profile (vector in {0, 1}n): n2nnumbers • graphical games: • players are vertices of a graph • V’s payoff depends on actions of Win N(V)UV • n players, max degree d => n2d+1numbers t=0, u=0, v=0, w=0: 12 t=1, u=0, v=0, w=0: 31 …. t=1, u=1, v=1, w=1: -6 W W’s payoffs (16 cases): T V U
Algorithms: What Was Known • Bounded-degree trees: • Exp-time algorithm/poly-time approximation algorithm to find all NE (Kearns, Littman, Singh, UAI 2001) • ??? poly-time algorithm to find a single NE (Kearns, Littman, Singh, NIPS’2001) • Heuristics for graphs with cycles
Our Results (E., Goldberg, Goldberg’06) • Algorithm in NIPS’01 paper is incorrect (does not always output a NE) • We fix the NIPS’01 algorithm, but… • our algorithm runs in poly-time on paths • with a trick, also on cycles • There is a graph of pathwidth 2 on which our algorithm runs in exp time • true for all algorithms that use the basic approach of the UAI’01 paper
0 0 1 1 0 0 1 1 2/3 BR(R) 1/4 Warm-up: 2-player 2-action games Row player: Column player: BR(C) Suppose R plays 1 w.p. r EP(C) from playing 0: (1-r)*1 EP(C) from playing 1: r*3 1-r > 3r iff r < ¼ c Suppose C plays 1 w.p. c EP(R) from playing 0: (1-c)*2 EP(R) from playing 1: c*1 (1-c)*2 > c iff c < 2/3 1 r 1 mixed NE: r=1/4, c=2/3
U2 Algorithm for Trees (KLS’01) • Potential best response:v is a PBR to w iff when W plays w, there is a NE for TV in which V plays v. • upstream pass: construct PBRV(w) from PBRU1(v), PBRU2(v) and PBRU3(v) • downstream pass: root selects its strategy based on the children’s PBR’s; propagates to leaves W v V TV U3 U1 w
U V W Computing PBR on a Path • E0 = EP(V) from playing 0: (1-u)(1-w)*v000+(1-u)w*v001+u(1-w)*v100+uw*v101 = auw+bu+cw+d • E1 = EP(V) from playing 1: (1-u)(1-w)*v010+(1-u)w*v011+u(1-w)*v110+uw*v111= a’uw+b’u+c’w+d’ • E0 = E1 iff w = (Au+B)/(Cu+D) = f(u) v u 1 1 (v, u) → (f(u), v) .5 PBRU(v) PBRV(w) .5 1 w .1 .9 1 v
Trees: too many segments W v u t t2 u2 v1 V u1 v2 t1 v v w v1 v2 v1 v2 T U (v,t), (v,u) → (f(u,t), v) Incorrect! KLS (NIPS’01): can “trim” PBR
Solutions? • Solution 1 (for paths): algorithm of UAI’01 paper, careful analysis • the number of segments/rectangles in each PBR is O(n2) • running time O(n3) • Solution 2 (for paths): can pick a subset of each PBR consisting of O(n) segments • O(n2) running time
Extension to trees? V0 V1 V2 Vn-1 Vn U1 U2 Un-1 Un T1 T2 Tn-1 Tn
Graphical games: hardness results • NP-hard? • no: total search problem • PPAD-hard? • yes! • in fact, this is how the hardness result for 4-player games was obtained (Goldberg, Papadimitriou, Aug 2005)
r-player game G NE of G g g f f deg 3 graphical game G’ NE of G’ Equivalences: GP’05 deg d graphical game G NE of G d2-player game G’ NE of G’
g f 4 X r-player game G NE of G 9-player game G’ NE of G’ Finding NE in a 4-player game is as hard as finding NE in a r-player game for any constant r Combining Reductions: GP’05
PPAD-hardness: missing details • 3D-Brouwer is PPAD-complete (Papadimitriou, 1994) • 4-NASH is as hard as deg 3-GG (Goldberg, Papadimitriou, Aug 2005) • deg 3-GG is PPAD-complete and hence 4-NASH is PPAD-complete (Daskalakis,Goldberg, Papadimitriou, Sep 2005) • 3-NASH is PPAD-complete (Daskalakis, Papadimitriou, Oct 2005, Chen, Deng, Nov 2005) • 2-NASH is PPAD-complete !!!(Chen, Deng, Dec 2005)
NE with special properties Pure NE: • easy for constant number of players • NP-hard for general graphical games • even if max degree = 3 • NP vs. PPAD: pure NE may not exist! • poly-time on trees (KLS algorithm) • also on graphs with bounded treewidth
0 0 1 1 0 0 1 1 Welfare-Maximizing NE Row player: Column player: • Nash equilibria: • (0, 0): total payoff is 3 • (1, 1): total payoff is 4 • (1/4, 2/3): total payoff is 17/12 • not all NE are created equal…
Algorithms for Good NE • 2-player games: checking for NE with total payoff > T is NP-hard (Gilboa Zemel 89, Conitzer, Sandholm 03) • Graphical games: - for any algebraic a, deg(a) = n, there is a GG with int payoffs on a path of length O(n) in which in the best NE player 1 plays a - approximation algorithms for any e (E., Goldberg, Goldberg 07)
Approximate NE • e-Nash equilibrium: a strategy profile such that noone can gain > e by deviating • Graphical games on trees: poly-time algorithms for any e (KLS’01) • 2-player games ( utilities in [0, 1] ): • PPAD-complete for e=O(1/n) • Approximation for constant e: • 0.5 WINE’06 (Dec 2006) • 0.382 ( =1-1/f ) ACM EC’07 (June 2007) • 0.364 WINE’07 (Dec 2007) • 0.339 WINE’07 (Dec 2007)
Conclusions • Computational aspects of game-theoretic questions are crucial • Lots of cool open problems • computing NE in graphical games on trees • finding e-Nash in 2-player games for small e • A rich set of techniques • Talk to me if you want to know more…
Mixed strategies and payoffs • Payoff matrices: • the row player plays a = (a1, …, an) • the column player plays b = (b1, …, bn) • expected payoff of R when playing i: (Ri, *, b) • expected payoff of C when playing i: (C*, j, a) R11 R12 … R1n R21 R22 … R2n … Rn1 Rn2 … Rnn C11 C12 … C1n C21 C22 … C2n … Cn1 Cn2 … Cnn R: C:
2 players, k actions: support guessing • if1st player’s strategy a supported on I Nai≠ 0 iff i I 2nd player’s strategy b supported on J Nbj≠ 0 iff j J • thenI BR(b): (b, Ri, *)≥ (b, Rk, *) for all i I, k N J BR(a): (a, C*, j)≥ (a, C*, k) for all j J, k N • LP on variables a1, …, an, b1, …, bn • solutions to LP ↔ Nash equilibria • running time: 22kpoly(k) linear inequalities!
0 0 1 1 0 0 1 1 2/3 BR(R) 1/4 Reminder: 2-player 2-action games Row player: Column player: BR(C) Suppose R plays 1 w.p. r EP(C) from playing 0: (1-r)*1 EP(C) from playing 1: r*3 1-r > 3r iff r < ¼ c Suppose C plays 1 w.p. c EP(R) from playing 0: (1-c)*2 EP(R) from playing 1: c*1 (1-c)*2 > c iff c < 2/3 1 r 1 mixed NE: r=1/4, c=2/3
U V W (v, u) → (f(u), v) + “tails” Computing PBR on a path • f(u) = (au+b)/(cu+d) • a, b, c, d are determined by V’s payoffs v u 1 1 .5 PBRU(v) PBRV(w) .5 1 w .1 .9 1 v