1 / 55

On the Complexity of Search Problems George Pierrakos

On the Complexity of Search Problems George Pierrakos. Mostly based on: On the Complexity of the Parity Argument and Other Insufficient Proofs of Existence [Pap94] On total functions, existence theorems and computational complexity [MP91] How easy is local search? [JPY88]

vgentry
Download Presentation

On the Complexity of Search Problems George Pierrakos

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. On the Complexity of Search ProblemsGeorge Pierrakos Mostly based on: On the Complexity of the Parity Argument and Other Insufficient Proofs of Existence [Pap94] On total functions, existence theorems and computational complexity [MP91] How easy is local search? [JPY88] Computational Complexity [Pap92] The complexity of computing a Nash equilibrium [DGP06] The complexity of pure Nash equilibria [FPT04] slides and scribe notes from many people…

  2. Outline • Generally on Search Problems • The Class TFNP • Subclasses of TFNP part I: PPA, PPAD • Problems in PPA, PPAD • Completeness in PPAD • Subclasses of TFNP part II: PPP, PLS • PPAD-completeness of NASH & the complexity of computing equilibria in congestion games

  3. Outline • Generally on Search Problems • The Class TFNP • Subclasses of TFNP part I: PPA, PPAD • Problems in PPA, PPAD • Completeness in PPAD • Subclasses of TFNP part II: PPP, PLS • PPAD-completeness of NASH & the complexity of computing equilibria in congestion games

  4. Decision Problems vs Search (or “function”) Problems • SAT • Input: boolean CNF-formula φ • Output: “yes” or “no” • FSAT • Input: boolean CNF-formula φ • Output: satisfying assignment or “no” if none exist

  5. Are search problems harder? They are definitely not easier: a poly-time algorithm for FSAT can be easily tweaked to give a poly-time algorithm for SAT …and vice versa, FSAT “reduces” to SAT: we can figure out a satisfying assignment by running poly-time algorithm for SAT n-times

  6. Are search problems harder? • SAT is self-reducible but TSP is not… • Here is how we solve FTSP using TSP: • Use binary search to figure out the cost C of the optimum tour, using TSP algorithm • Run TSP algorithm n2 times, always with cost parameter C, each time setting the weight of some edge to C+1

  7. The Classes FP and FNP • L €NP iff there exists poly-time computable RL(x,y) s.t. X € L  y { |y| ≤ p(|x|) & RL(x,y) } • Note how RL defines the problem-language L • The corresponding search problem ΠR(L) €FNP is: given an x find any y s.t. RL(x,y) and reply “no” if none exist • FSAT € FNP… what about FTSP? • Are all FNP problems self-reducible like FSAT? [open?] • FP is the subclass of FNP where we only consider problems for which a poly-time algorithm is known

  8. Reductions and completeness • A function problem ΠR reduces to a function problem ΠS if there exist log-space computable string functions f and g, s.t. R(x,g(y))  S(f(x),y) • intuitively f reduces problem ΠR to ΠS • and g transforms a solution of ΠS to one of ΠR • Standard notion of completeness works fine…

  9. FP <?> FNP • A proof a-la-Cook shows that FSAT is FNP-complete • Hence, if FSAT € FP then FNP = FP • But we showed self-reducibility for SAT, so the theorem follows: • Theorem: FP = FNP iff P=NP • So, why care for function problems anyway??

  10. Outline • Generally on Search Problems • The Class TFNP • Subclasses of TFNP part I: PPA, PPAD • Problems in PPA, PPAD • Completeness in PPAD • Subclasses of TFNP part II: PPP, PLS • PPAD-completeness of NASH & the complexity of computing equilibria in congestion games

  11. On total “functions”: the class TFNP • What happens if the relation R is total? i.e., for each x there is at least one y s.t. R(x,y) • Define TFNP to be the subclass of FNP where the relation R is total • TFNP contains problems that always have a solution, e.g. factoring, fix-point theorems, graph-theoretic problems, … • How do we know a solution exists? By an “inefficient proof of existence”, i.e. non-(efficiently)-constructive proof • The idea is to categorize the problems in TFNP based on the type of inefficient argument that guarantees their solution

  12. Basic stuff about TFNP • FP TFNP FNP • TFNP = F(NP coNP) • NP = problems with “yes” certificate y s.t. R1(x,y) • coNP = problems with “no” certificate z s.t. R2(x,y) • for TFNP F(NP coNP) take R = R1 U R2 • for F(NP coNP) TFNP take R1 = R and R2 = ø • There is an FNP-complete problem in TFNP iff NP = coNP • : If NP = coNP then trivially FNP = TFNP • : If the FNP-complete problem ΠR is in TFNP then:FSAT reduces to ΠR via f and g, hence any unsatisfiable formula φ has a “no” certificate y, s.t. R(f(φ),y) (y exists since ΠR is in TFNP) and g(y)=“no” • TFNP is a semantic complexity class  no complete problems! • note how telling whether a relation is total is undecidable (and not even RE!!)

  13. Outline • Generally on Search Problems • The Class TFNP • Subclasses of TFNP part I: PPA, PPAD • Problems in PPA, PPAD • Completeness in PPAD • Subclasses of TFNP part II: PPP, PLS • PPAD-completeness of NASH & the complexity of computing equilibria in congestion games

  14. Subclasses of TFNP: PPA, PPAD • PPA stands for polynomial parity argument: • Every graph has an even number of odd-degree nodes • Every graph of degree two or less has an even number of leaves • Both arguments turn out to be equally generic • PPAD stands for polynomial parity argument directed  • Parity argument in directed graphs, where we now seek for a sink or a source

  15. PPAANOTHER HAMILTON CYCLE • Thm: any graph with odd degrees has an even number of HC through edge xy • Proof Idea: • take a HC • remove edge (1,2) & take a HP • fix endpoint 1 and start “rotating” from the other end • each HP has two “valid” neighbors (d=3) except for those paths with endpoints 1,2 • Evoke the parity argument: at least two (different) such HPs must exist: add edge (1,2) et voila! • time: worst case exponential…

  16. PPAD2D-SPERNER • Given a triangulation of a triangle (castle) s.t. • each original vertex obtains its own color • no vertex on edge ij contains color 3-i-j there exists a trichromatic triangle • Proof Idea: • add external 0-1 edges • traverse triangles (“rooms”) using 0-1 edges (“doors”) • If a “room” has no “door” out then it is trichromatic • Note that • the process cannot exit the castle and cannot fold upon itself • our tour leaves 0 nodes to the right, hence direction, hence PPAD…

  17. Definition of PPA • Let A be a problem and M the (associated) poly-time TM • Let x be an input of A • Let Cx= Σp(|x|) be the configuration space of x, i.e. the set of all strings of length at most p(|x|) • On input c € Cx machine M outputs in time O(p(n)) a set of at most two configurations M(x,c) • M(x,c) may well be empty if c is “rubbish” • c and c’ are neighbors ([c,c’] € G(x)) iff c € M(x,c) and c’ € M(x,c’) • G(x) is symmetric with degree at most 2 • It is the implicit search graph of the problem • Let M(x, 0…0) = {1…1} and 0…0 € M(x, 1…1), hence 0…0 is the standard leaf • PPA is the class of problems defined as follows: “Given x, find a leaf of G(x) other than the standard leaf 0…0”

  18. Definition of PPA (an example) Take A to be ANOTHER HAMILTON CYCLE. We show that A € PPA • x is the input of A: • a graph and a Hamilton Cycle • Cxis the configuration space of x: • Cxcontains encodings of “valid” Hamilton Paths (poly-space) and other irrelevant stuff • On input c € Cx machine M outputs in time O(p(n)) a set M(x,c) of: • the at most two neighbors of a valid Hamilton Path, or possibly nothing • c and c’ are neighbors ([c,c’] € G(x)) iff c € M(x,c) and c’ € M(x,c’) • remember the graph of neighboring “valid” Hamilton Paths • the standard leaf is the initial Hamilton Path (from removal of (1,2))

  19. Definition of PPAD • Same as PPA but… • M(x,c) = In(x,c) U Out (x,c)is an ordered pair of configurations with|In(x,c)| ≤ 1 & |Out (x,c)| ≤ 1 • G(x) is now directed:[c, c’] € G(x) iff c € In(x,c’) and c’ € Out(x,c) • We search for a node (other than the standard leaf) with |In(x,c)| + |Out (x,c)| = 1,i.e. any source or sink other than the standard one

  20. Definition of PPAD (an example) Take A to be 2D-SPERNER. We show that A € PPAD • x is the input of A: • a triangle, a triangulation and a valid color assignment • CAUTION: restricted input  “promise” problems: we usually need some sort of syntactic guarantee of the validity of the input • Cxis the configuration space of x: • Cxcontains encodings of all sub-triangles in the triangulation • M(x,c) = In(x,c) U Out (x,c) where: • In(x,c) is the castle room where we came from • Out(x,c) is the castle room where we are heading to • c and c’ are neighbors ([c,c’] € G(x)) iff c € M(x,c) and c’ € M(x,c’) • remember our tour through the triangular castle • the standard leaf is the node lying outside the triangle

  21. Couple of Theorems • PPA as defined above makes use of the “even leaves” argument • Why not modify the definition as follows (PPA’): • …on input c € Cx machine M outputs in time O(p(n)) a set M(x,c) s.t. |M(x,c)| = poly(n)… • …“Given x, find an odd-degree node of G(x) other than the standard leaf 0…0”… and make use of the (more general?) parity argument? • Quite surprisingly: PPA’ = PPA

  22. Theorem 1: PPA’ = PPA(The Chessplayer Algorithm) • Obviously PPA PPA’ • For the inverse: • For each node c of degree k ≥ 1, we create new nodes (c,i), where i = 0,…,k div 2 • each node (c,i) is connected to the two nodes (c’,j) s.t. the lexicographically 2i, 2i+1 edges out of c are the 2j, 2j+1 edges into c’ • nonstandard leaves of the new graph correspond to nonstandard odd-degree nodes of the former

  23. Two more propositions • Proposition 1: FP PPAD PPA FNP • Proposition 2: • Consider the more natural definition of PPA, where the leaf we search for is the one connected to the standard leaf (PPA’’) • Then PPA’’ = PPAD’’ = FPSPACE • just show FPSPACE PPA’’ • any problem in FPSPACE can be solved by a reversible TM, i.e. one where each configuration has at most one predecessor and one successor [Be84] • define M(x,c) to return the (at most) two configurations that are the predecessor and successor of c on input x • hence, the nonstandard leaf on the same component as the standard leaf is the halting configuration with the desired output

  24. PPADBROUWER FIX POINT • Brouwer’s fix-point theorem: let S be any n-dimensional simplex and let φ: SSbe a continuous function. Then φ has a fix point, i.e. an x* s.t. φ(x*) = x* • Proof for the special case of a triangle, using Sperner lemma • Let T1, …, Ti,… be a sequence of finer triangulations • Define labels L1, …, Li,… for the triangulations as depicted: Li is a valid labeling for Ti • Consider the sequence of centers c1, …, ci,… of the resulting trichromatic triangles t1, …, ti,…(guaranteed by Sperner’s Lemma) • This is an infinite sequence in a closed convex space, hence it has a subsequence converging at x*: a point which is arbitrarily closed to the centroid of a trichromatic triangle • x* is a fix point of φ: if not, then we can find a small triangle containing x* but not φ(x*). Because of the continuity of φ, we can show that this cannot be labeled with 3 colors • Also need to take care of continuity for representation in TM

  25. PPAD NASH & LINEAR COMPL/RITY • Theorem[Nash51]: every game has a mixed Nash equilibrium • Original proof of Nash uses Kakutani’s fix point theorem; KAKUTANI is in PPAD by a reduction from BROWER • Nash’s theorem can be proved using Brower’s fix-point theorem too • In [Pap94] they show that NASH is in PPAD by a reduction from LINEAR COMPLENTARITY

  26. PPAD NASH & LINEAR COMPL/RITY • LINEAR COMPLEMENTARITY • We are given an nxn matrix A and an n-vector b • We are seeking vectors x,y ≥ 0 s.t. • Ax + b = y • x·y = 0, i.e. at least one of xi, yi is 0 • Solved by Lemke’s algorithm: • some times it does not even converge • if it converges it might well be (worst case) exponential • In general, the problem is NP-complete (?) • For the case of P-matrices A (matrices satisfying det(AS) > 0 for all non-empty subsets S of {1,…,n}) the problem has a unique solution and Lemke’s algorithm converges to it. • Is checking for the P-property for general matrices in P? Open… • PPAD formulation of L-COMPL: Given A and b, find a solution x, y to the linear complementarity problem or a subset S s.t. det(AS) ≤ 0

  27. PPAD NASH & LINEAR COMPL/RITY • Example: from bimatrix games to a linear complementarity problem • Let • are the probability and payoff of player 1 for row i • are the probability and payoff of player 2 for column j • is the payoff of player 1 • is the payoff of player 2

  28. PPAD NASH & LINEAR COMPL/RITY • Approach 1: • implies that the player has equal expected payoff from both his choices (else he would pick the one maximizing his profit with probability 1 and drop the other) • Hence we can write down the following linear system: • To compute all equilibria we need to solve a linear system (in poly-time) for all different supports of players 1 and 2 in time 2n2mP(n,m)(support = subset of strategies played with p>0)

  29. PPAD NASH & LINEAR COMPL/RITY • Approach 2: reduce to L-COMPL

  30. PPAD NASH & LINEAR COMPL/RITY Use Lemke’s algorithm: “double pivoting”

  31. PPAANOTHER HAMILTON PATH • Given a graph G and a HP in it, find either another HP in G or in its complement Gc • Construct a bipartite graph, whose odd-degree nodes are HP of G and Gc • “left” nodes: all HP of the complete graph on the nodes of G • “right” nodes: all subgraphs of Gc -including the empty subgraph • Edge from h (hpath) on left side to s (subgraph) on the right side iff s is a proper subgraph of h • Claim: the only odd-degree nodes are the HP of G and Gc on the left side • If node on the left is not a HP of G or Gc then it contains a non-empty subset of edges of Gc and will be adjacent to all proper subsets of this set: these are 2k • A HP of G on the left will only be adjacent to the empty subgraph • A HP of Gc on the left will be adjacent to its 2n-1-1 proper subgraphs (excluding itself) and will also have odd degree • Any subgraph (node on the right) will be connected to all its possible connections to a HP (usually 0): for graphs with more than 5 nodes this number is even (can be proved via an easy calculation…)

  32. Other problems • PPA: • HAMILTON DECOMPOSITION • TSP NONADJACENCY • CHEVALLEY MOD 2 • TOTALLY EVEN SUBGRAPH • CUBIC SUBGRAPH • PPAD: • k-D SPERNER • KAKUTANI • COMPETITIVE & EXCHANGE EQUILIBRIUM • TUCKER • BORSUK-ULAM, DISCRETE HAM SANDWICH, NECKLAGE SPLITTING…

  33. Outline • Generally on Search Problems • The Class TFNP • Subclasses of TFNP part I: PPA, PPAD • Problems in PPA, PPAD • Completeness in PPAD • Subclasses of TFNP part II: PPP, PLS • PPAD-completeness of NASH & the complexity of computing equilibria in congestion games

  34. The Class PPP • PPP stands for Polynomial Pigeonhole Principle • Definition: • A problem A is in PPP if it is defined in terms of a poly-time TM M • Given input x define the set of “solutions” S(x) = Σp(|x|)and • For any y € S(x), M(x,y) € S(x) • Desired output: • y € S(x), s.t. M(x,y) = 0…0 or • y, y’ € S(x) s.t. M(x,y) = M(x,y’) • One of the two is guaranteed to exist by pigeonhole principle

  35. A PPP-complete problem:PIGEONHOLE CIRCUIT • Input: Boolean circuit C with n inputs and n output • Output: either find an input y s.t. C(y) = 0n or find two inputs y, y’ s.t. C(y) = C(y’) • Obviously PIGEONHOLE CIRCUIT € PPP: • M is the poly-time TM that simulates circuit C on input y • Input x is the description of the circuit C • S(x) = {0,1}n and for any y € S(x), M(x,y) € S(x) • And any problem in PPP can be evaluated by such a circuit: just tweak the proof of CIRCUIT-VALUE P-completeness

  36. PIGEONHOLE CIRCUITfor MONOTONE circuits? • Monotone circuits are the circuits that do not contain “no” gates • Property of monotone circuits:swapping xi from false to true cannot change any output bit from true to false • Proof: trivial by induction on the depth of the tree • MONOTONE PIGEONHOLE CIRCUIT € FP • Proof: • We need to consider only the input sequences of the form 0i1n-i, i=0,…,n, i.e. only n input sequences • Start with 1n and start swapping bits to 0. The only way to have n distinct outputs is if: output starts at 0n and each time one output bit swaps to 1 • Hence, either output = 0n for some input, or two inputs have same output

  37. PPPEQUAL SUMS • Input: n positive integers with sum less than 2n-1 • Output: two subsets with the same sum • EQUAL SUMS in PPP: • Two such subsets trivially exist since there are more subsets than possible sums • The formal definitions utilizes a TM M that given the input x (set of all sets) and the description of a set y, outputs in polynomial time the sum of the set’s elements. Since the sum of all elements is less than 2n-1 we cannot have M(x,y) = 0…0 • Is EQUAL SUMS PPP-complete?

  38. Two more propositions • One-way permutation π is a poly-time algorithm mapping Σn to itself, such that for all x≠y, π(x) ≠ π(y), s.t. it is hard to recover x from π(x). • Prop1: If PPP=FP then one-way permutations do not exist • Proof: • suppose PIGEONHOLE CIRCUIT is in FP • We wish to invert an alleged one-way permutation π, i.e. given y, find x s.t. y=π(x) • Turn the algorithm for π into a circuit (CIRCUIT VAL is P-complete) and add bitwise XOR gates between output y and x; call this circuit C • Apply the poly-time algorithm for PIGEONHOLE CIRCUIT to C. Since π is a permutation, the algorithm cannot return two inputs giving the same output, hence it returns y, s.t. C(y) = 0, i.e. π(x) = y

  39. Two more propositions • Prop2: PPAD PPP • Proof: • Reformulate any problem in PPAD with graph G as a problem in PPP by use of π(x) defined as • π(x) = x if x is a sink • π(x) = G2(x) if x is a nonstandard source • π(x) = G(x) otherwise • Trivially, from any confluence of values of π we can recover a nonstandard source or a sink of G

  40. The class PLS • PLS originally defined in [Johnson, et. al. ‘88]: “find some local optimum in a reasonable search space”. • Basic ingredient: a problem with a search space, i.e. a set of feasible solutions which has a neighborhood structure • Every problem L (e.g. TSP) in PLS has a set of instances DL (e.g. encodings of adjacency matrix) • Every instance x in DL has a finite set of solutions FL(x) (e.g. tours) considered wlog polynomial in size • For each solution s in FL(x) we have • A poly-time cost function c(x,s) that given an instance x and a solution s, it computes its cost (e.g. length of a tour) • A poly-time neighbor function N(x,s) FL(x) that given an instance x and a solution s, returns its neighboring solutions (e.g. a tour that differs from the original one in at most two edges)

  41. The class PLS • Remaining constraints are expressed in terms of 3 polynomial time algorithms AL, BL, CL • AL, given an instance x in DL produces a particular standard solution of L, AL(x) € FL(x) (e.g. a standard tour for TSP) • BL, given an instance x in DL and a string s determines whether s is a solution (s € FL(x)?) and if so, it computes its cost c(x,s) • CL, given an instance x € DL and a string s € FL(x) has two possible outcomes, depending on s: • If there is any solution s’ € N(x,s) s.t. c(x,s’) < c(x,s) it outputs s’ • Otherwise it outputs “no”, which implies that s is locally optimal

  42. The class PLS • The sense of local searching is obvious: starting at some arbitrary solution start searching locally for a better solution and move towards it. • Standard local search algorithm: Given x, use AL to produce a starting solution s= AL(x) Repeat until locally optimal solution is found: Apply algorithm CL to x and s If CL yields a better cost neighbor s’ of then set s=s’ • Since the solution space is finite the above algorithm always halts • If FL(x) is polynomially bounded then the above algorithm is polynomial • However, things can turn out pretty bad: • Finding a local optimum reachable from a specific state can be proven PSPACE-complete • There are instances with states exponentially far from any local optimum • Correct question to ask: “Given x, find some locally optimum solution s”

  43. The class PLS • PLS-reduction: we must map the local optima of one problem to those of another, thus pairtaining the neighboring structure. Hence problem L reduces to K if there exist poly-time functions f and g s.t. • f maps instances x of L to instances f(x) of K • g maps (solutions of f(x), x) pairs to solutions of x • for all instances x of L, if s is a local optimum for instance f(x) of K, then g(s,x) is a local optimum for x • Basic PLS-complete problem: • weighted CIRCUIT – SAT under input bitflips • some relatives of TSP, MAXCUT and: • POS-NAE-3SAT: [Shaeffer&Yannakakis-91] Given an instance of NAE-3SAT with weights on its clauses and with positive literals only, find a truth assignment, whose total weight (i.e. the weight of the unsatisfied and totally satisfied clauses) cannot be improved by flipping a variable

  44. Outline • Generally on Search Problems • The Class TFNP • Subclasses of TFNP part I: PPA, PPAD • Problems in PPA, PPAD • Completeness in PPAD • Subclasses of TFNP part II: PPP, PLS • PPAD-completeness of NASH & the complexity of computing equilibria in congestion games

  45. Congestion GamesDefinitions • Idea: generalization of Selfish Task Allocation • Intuition: • Route traffic through a network • Resource allocation • We have • a set of resources E (network edges…?) • a set of players N • the resource delays de • For each player i • an action set Ai 2E • a weight (traffic demand) wi

  46. Congestion GamesDefinitions • let A=(a1,…, an), ai€ Ai, be a strategy profile • denote by • Pe(A) the set of players picking resource e in profile A • ne(A) the number of players picking resource e in profile A, ne(A)=|Pe(A)| • define the load of resource e to be L(e) = Σiin P(e)wi • for each strategy profile A, each player incurs a total delay/cost ci(A) = Σein aiL(e)

  47. Congestion GamesEquilibria and Potential Functions • A Pure Nash Equilibrium (PNE) is a strategy profile A=(a1,…, an), s.t. for all i € N and for all ai’ € Ai c(a1,…, ai,…, an) ≤ c(a1,…, ai’,…, an) • Notice the local optimality flavor of a PNE • Exact Potential for a c.g. is a function Φ:Ε→R s.t. for all i € N, ai’ € Ai and A ci(A) - ci(A-i, ai’) = Φ(A) - Φ(A-i, ai’) • It is easy to verify that every game that admits an exact potential possesses a PNE

  48. Types of Congestion Games • Symmetric vs AsymmetricPlayers have the same action set and the same weights, i.e. they are indistinguishable • Network vs General Congestion Games • the resources are graph edges • each player has a source and a sink vertex (si,ti) • the action sets Ai are si-ti paths • Single vs multi-commodity (network) games • the (si,ti) pairs are common for all players

  49. Congestion GamesExistence of PNE • Theorem [Rosenthal ’73]:Every unweighted congestion game is an exact potential game • The inverse holds [Mondere&Shapley]: Every finite exact potential game is isomorphic to an unweighted c.g. • So it is only for c.g. that we can find an exact potential • Does this mean that the potential technique (for guaranteeing existence of a local minimum) can be used only for c.g.?

  50. Congestion GamesPotential Functions • We need to be less strict about the potential function. • Consider the party affiliation game (HAPPYNET): • n players (nodes in graph) • Symmetric preference relations between players-nodes • 2 actions per player: {-1,1} • State S is a mapping V{-1,1} (players pick sides) • payoff of i: sgn[S(i) ΣjS(j)wij] (if the payoff is positive then we say “node i is happy”) • The problem: “Find a state in which all nodes are happy” • intuition: everyone wants to be with his friends • application: convergence of Hopfield NN

More Related