On the Complexity of Search Problems George Pierrakos

On the Complexity of Search ProblemsGeorge Pierrakos Mostly based on: On the Complexity of the Parity Argument and Other Insufficient Proofs of Existence [Pap94] On total functions, existence theorems and computational complexity [MP91] How easy is local search? [JPY88] Computational Complexity [Pap92] The complexity of computing a Nash equilibrium [DGP06] The complexity of pure Nash equilibria [FPT04] slides and scribe notes from many people…

Outline • Generally on Search Problems • The Class TFNP • Subclasses of TFNP part I: PPA, PPAD • Problems in PPA, PPAD • Completeness in PPAD • Subclasses of TFNP part II: PPP, PLS • PPAD-completeness of NASH & the complexity of computing equilibria in congestion games

Decision Problems vs Search (or “function”) Problems • SAT • Input: boolean CNF-formula φ • Output: “yes” or “no” • FSAT • Input: boolean CNF-formula φ • Output: satisfying assignment or “no” if none exist

Are search problems harder? They are definitely not easier: a poly-time algorithm for FSAT can be easily tweaked to give a poly-time algorithm for SAT …and vice versa, FSAT “reduces” to SAT: we can figure out a satisfying assignment by running poly-time algorithm for SAT n-times

Are search problems harder? • SAT is self-reducible but TSP is not… • Here is how we solve FTSP using TSP: • Use binary search to figure out the cost C of the optimum tour, using TSP algorithm • Run TSP algorithm n2 times, always with cost parameter C, each time setting the weight of some edge to C+1

The Classes FP and FNP • L €NP iff there exists poly-time computable RL(x,y) s.t. X € L  y { |y| ≤ p(|x|) & RL(x,y) } • Note how RL defines the problem-language L • The corresponding search problem ΠR(L) €FNP is: given an x find any y s.t. RL(x,y) and reply “no” if none exist • FSAT € FNP… what about FTSP? • Are all FNP problems self-reducible like FSAT? [open?] • FP is the subclass of FNP where we only consider problems for which a poly-time algorithm is known

Reductions and completeness • A function problem ΠR reduces to a function problem ΠS if there exist log-space computable string functions f and g, s.t. R(x,g(y))  S(f(x),y) • intuitively f reduces problem ΠR to ΠS • and g transforms a solution of ΠS to one of ΠR • Standard notion of completeness works fine…

FP <?> FNP • A proof a-la-Cook shows that FSAT is FNP-complete • Hence, if FSAT € FP then FNP = FP • But we showed self-reducibility for SAT, so the theorem follows: • Theorem: FP = FNP iff P=NP • So, why care for function problems anyway??

On total “functions”: the class TFNP • What happens if the relation R is total? i.e., for each x there is at least one y s.t. R(x,y) • Define TFNP to be the subclass of FNP where the relation R is total • TFNP contains problems that always have a solution, e.g. factoring, fix-point theorems, graph-theoretic problems, … • How do we know a solution exists? By an “inefficient proof of existence”, i.e. non-(efficiently)-constructive proof • The idea is to categorize the problems in TFNP based on the type of inefficient argument that guarantees their solution

Basic stuff about TFNP • FP TFNP FNP • TFNP = F(NP coNP) • NP = problems with “yes” certificate y s.t. R1(x,y) • coNP = problems with “no” certificate z s.t. R2(x,y) • for TFNP F(NP coNP) take R = R1 U R2 • for F(NP coNP) TFNP take R1 = R and R2 = ø • There is an FNP-complete problem in TFNP iff NP = coNP • : If NP = coNP then trivially FNP = TFNP • : If the FNP-complete problem ΠR is in TFNP then:FSAT reduces to ΠR via f and g, hence any unsatisfiable formula φ has a “no” certificate y, s.t. R(f(φ),y) (y exists since ΠR is in TFNP) and g(y)=“no” • TFNP is a semantic complexity class  no complete problems! • note how telling whether a relation is total is undecidable (and not even RE!!)

Subclasses of TFNP: PPA, PPAD • PPA stands for polynomial parity argument: • Every graph has an even number of odd-degree nodes • Every graph of degree two or less has an even number of leaves • Both arguments turn out to be equally generic • PPAD stands for polynomial parity argument directed  • Parity argument in directed graphs, where we now seek for a sink or a source

PPAANOTHER HAMILTON CYCLE • Thm: any graph with odd degrees has an even number of HC through edge xy • Proof Idea: • take a HC • remove edge (1,2) & take a HP • fix endpoint 1 and start “rotating” from the other end • each HP has two “valid” neighbors (d=3) except for those paths with endpoints 1,2 • Evoke the parity argument: at least two (different) such HPs must exist: add edge (1,2) et voila! • time: worst case exponential…

PPAD2D-SPERNER • Given a triangulation of a triangle (castle) s.t. • each original vertex obtains its own color • no vertex on edge ij contains color 3-i-j there exists a trichromatic triangle • Proof Idea: • add external 0-1 edges • traverse triangles (“rooms”) using 0-1 edges (“doors”) • If a “room” has no “door” out then it is trichromatic • Note that • the process cannot exit the castle and cannot fold upon itself • our tour leaves 0 nodes to the right, hence direction, hence PPAD…

Definition of PPA • Let A be a problem and M the (associated) poly-time TM • Let x be an input of A • Let Cx= Σp(|x|) be the configuration space of x, i.e. the set of all strings of length at most p(|x|) • On input c € Cx machine M outputs in time O(p(n)) a set of at most two configurations M(x,c) • M(x,c) may well be empty if c is “rubbish” • c and c’ are neighbors ([c,c’] € G(x)) iff c € M(x,c) and c’ € M(x,c’) • G(x) is symmetric with degree at most 2 • It is the implicit search graph of the problem • Let M(x, 0…0) = {1…1} and 0…0 € M(x, 1…1), hence 0…0 is the standard leaf • PPA is the class of problems defined as follows: “Given x, find a leaf of G(x) other than the standard leaf 0…0”

Definition of PPA (an example) Take A to be ANOTHER HAMILTON CYCLE. We show that A € PPA • x is the input of A: • a graph and a Hamilton Cycle • Cxis the configuration space of x: • Cxcontains encodings of “valid” Hamilton Paths (poly-space) and other irrelevant stuff • On input c € Cx machine M outputs in time O(p(n)) a set M(x,c) of: • the at most two neighbors of a valid Hamilton Path, or possibly nothing • c and c’ are neighbors ([c,c’] € G(x)) iff c € M(x,c) and c’ € M(x,c’) • remember the graph of neighboring “valid” Hamilton Paths • the standard leaf is the initial Hamilton Path (from removal of (1,2))

Definition of PPAD (an example) Take A to be 2D-SPERNER. We show that A € PPAD • x is the input of A: • a triangle, a triangulation and a valid color assignment • CAUTION: restricted input  “promise” problems: we usually need some sort of syntactic guarantee of the validity of the input • Cxis the configuration space of x: • Cxcontains encodings of all sub-triangles in the triangulation • M(x,c) = In(x,c) U Out (x,c) where: • In(x,c) is the castle room where we came from • Out(x,c) is the castle room where we are heading to • c and c’ are neighbors ([c,c’] € G(x)) iff c € M(x,c) and c’ € M(x,c’) • remember our tour through the triangular castle • the standard leaf is the node lying outside the triangle

Couple of Theorems • PPA as defined above makes use of the “even leaves” argument • Why not modify the definition as follows (PPA’): • …on input c € Cx machine M outputs in time O(p(n)) a set M(x,c) s.t. |M(x,c)| = poly(n)… • …“Given x, find an odd-degree node of G(x) other than the standard leaf 0…0”… and make use of the (more general?) parity argument? • Quite surprisingly: PPA’ = PPA

Theorem 1: PPA’ = PPA(The Chessplayer Algorithm) • Obviously PPA PPA’ • For the inverse: • For each node c of degree k ≥ 1, we create new nodes (c,i), where i = 0,…,k div 2 • each node (c,i) is connected to the two nodes (c’,j) s.t. the lexicographically 2i, 2i+1 edges out of c are the 2j, 2j+1 edges into c’ • nonstandard leaves of the new graph correspond to nonstandard odd-degree nodes of the former

Two more propositions • Proposition 1: FP PPAD PPA FNP • Proposition 2: • Consider the more natural definition of PPA, where the leaf we search for is the one connected to the standard leaf (PPA’’) • Then PPA’’ = PPAD’’ = FPSPACE • just show FPSPACE PPA’’ • any problem in FPSPACE can be solved by a reversible TM, i.e. one where each configuration has at most one predecessor and one successor [Be84] • define M(x,c) to return the (at most) two configurations that are the predecessor and successor of c on input x • hence, the nonstandard leaf on the same component as the standard leaf is the halting configuration with the desired output

PPADBROUWER FIX POINT • Brouwer’s fix-point theorem: let S be any n-dimensional simplex and let φ: SSbe a continuous function. Then φ has a fix point, i.e. an x* s.t. φ(x*) = x* • Proof for the special case of a triangle, using Sperner lemma • Let T1, …, Ti,… be a sequence of finer triangulations • Define labels L1, …, Li,… for the triangulations as depicted: Li is a valid labeling for Ti • Consider the sequence of centers c1, …, ci,… of the resulting trichromatic triangles t1, …, ti,…(guaranteed by Sperner’s Lemma) • This is an infinite sequence in a closed convex space, hence it has a subsequence converging at x*: a point which is arbitrarily closed to the centroid of a trichromatic triangle • x* is a fix point of φ: if not, then we can find a small triangle containing x* but not φ(x*). Because of the continuity of φ, we can show that this cannot be labeled with 3 colors • Also need to take care of continuity for representation in TM

PPAD NASH & LINEAR COMPL/RITY • Theorem[Nash51]: every game has a mixed Nash equilibrium • Original proof of Nash uses Kakutani’s fix point theorem; KAKUTANI is in PPAD by a reduction from BROWER • Nash’s theorem can be proved using Brower’s fix-point theorem too • In [Pap94] they show that NASH is in PPAD by a reduction from LINEAR COMPLENTARITY

PPAD NASH & LINEAR COMPL/RITY • LINEAR COMPLEMENTARITY • We are given an nxn matrix A and an n-vector b • We are seeking vectors x,y ≥ 0 s.t. • Ax + b = y • x·y = 0, i.e. at least one of xi, yi is 0 • Solved by Lemke’s algorithm: • some times it does not even converge • if it converges it might well be (worst case) exponential • In general, the problem is NP-complete (?) • For the case of P-matrices A (matrices satisfying det(AS) > 0 for all non-empty subsets S of {1,…,n}) the problem has a unique solution and Lemke’s algorithm converges to it. • Is checking for the P-property for general matrices in P? Open… • PPAD formulation of L-COMPL: Given A and b, find a solution x, y to the linear complementarity problem or a subset S s.t. det(AS) ≤ 0

PPAD NASH & LINEAR COMPL/RITY • Example: from bimatrix games to a linear complementarity problem • Let • are the probability and payoff of player 1 for row i • are the probability and payoff of player 2 for column j • is the payoff of player 1 • is the payoff of player 2

PPAD NASH & LINEAR COMPL/RITY • Approach 1: • implies that the player has equal expected payoff from both his choices (else he would pick the one maximizing his profit with probability 1 and drop the other) • Hence we can write down the following linear system: • To compute all equilibria we need to solve a linear system (in poly-time) for all different supports of players 1 and 2 in time 2n2mP(n,m)(support = subset of strategies played with p>0)

PPAD NASH & LINEAR COMPL/RITY • Approach 2: reduce to L-COMPL

PPAD NASH & LINEAR COMPL/RITY Use Lemke’s algorithm: “double pivoting”

PPAANOTHER HAMILTON PATH • Given a graph G and a HP in it, find either another HP in G or in its complement Gc • Construct a bipartite graph, whose odd-degree nodes are HP of G and Gc • “left” nodes: all HP of the complete graph on the nodes of G • “right” nodes: all subgraphs of Gc -including the empty subgraph • Edge from h (hpath) on left side to s (subgraph) on the right side iff s is a proper subgraph of h • Claim: the only odd-degree nodes are the HP of G and Gc on the left side • If node on the left is not a HP of G or Gc then it contains a non-empty subset of edges of Gc and will be adjacent to all proper subsets of this set: these are 2k • A HP of G on the left will only be adjacent to the empty subgraph • A HP of Gc on the left will be adjacent to its 2n-1-1 proper subgraphs (excluding itself) and will also have odd degree • Any subgraph (node on the right) will be connected to all its possible connections to a HP (usually 0): for graphs with more than 5 nodes this number is even (can be proved via an easy calculation…)

Other problems • PPA: • HAMILTON DECOMPOSITION • TSP NONADJACENCY • CHEVALLEY MOD 2 • TOTALLY EVEN SUBGRAPH • CUBIC SUBGRAPH • PPAD: • k-D SPERNER • KAKUTANI • COMPETITIVE & EXCHANGE EQUILIBRIUM • TUCKER • BORSUK-ULAM, DISCRETE HAM SANDWICH, NECKLAGE SPLITTING…

The Class PPP • PPP stands for Polynomial Pigeonhole Principle • Definition: • A problem A is in PPP if it is defined in terms of a poly-time TM M • Given input x define the set of “solutions” S(x) = Σp(|x|)and • For any y € S(x), M(x,y) € S(x) • Desired output: • y € S(x), s.t. M(x,y) = 0…0 or • y, y’ € S(x) s.t. M(x,y) = M(x,y’) • One of the two is guaranteed to exist by pigeonhole principle

A PPP-complete problem:PIGEONHOLE CIRCUIT • Input: Boolean circuit C with n inputs and n output • Output: either find an input y s.t. C(y) = 0n or find two inputs y, y’ s.t. C(y) = C(y’) • Obviously PIGEONHOLE CIRCUIT € PPP: • M is the poly-time TM that simulates circuit C on input y • Input x is the description of the circuit C • S(x) = {0,1}n and for any y € S(x), M(x,y) € S(x) • And any problem in PPP can be evaluated by such a circuit: just tweak the proof of CIRCUIT-VALUE P-completeness

PIGEONHOLE CIRCUITfor MONOTONE circuits? • Monotone circuits are the circuits that do not contain “no” gates • Property of monotone circuits:swapping xi from false to true cannot change any output bit from true to false • Proof: trivial by induction on the depth of the tree • MONOTONE PIGEONHOLE CIRCUIT € FP • Proof: • We need to consider only the input sequences of the form 0i1n-i, i=0,…,n, i.e. only n input sequences • Start with 1n and start swapping bits to 0. The only way to have n distinct outputs is if: output starts at 0n and each time one output bit swaps to 1 • Hence, either output = 0n for some input, or two inputs have same output

PPPEQUAL SUMS • Input: n positive integers with sum less than 2n-1 • Output: two subsets with the same sum • EQUAL SUMS in PPP: • Two such subsets trivially exist since there are more subsets than possible sums • The formal definitions utilizes a TM M that given the input x (set of all sets) and the description of a set y, outputs in polynomial time the sum of the set’s elements. Since the sum of all elements is less than 2n-1 we cannot have M(x,y) = 0…0 • Is EQUAL SUMS PPP-complete?

Two more propositions • One-way permutation π is a poly-time algorithm mapping Σn to itself, such that for all x≠y, π(x) ≠ π(y), s.t. it is hard to recover x from π(x). • Prop1: If PPP=FP then one-way permutations do not exist • Proof: • suppose PIGEONHOLE CIRCUIT is in FP • We wish to invert an alleged one-way permutation π, i.e. given y, find x s.t. y=π(x) • Turn the algorithm for π into a circuit (CIRCUIT VAL is P-complete) and add bitwise XOR gates between output y and x; call this circuit C • Apply the poly-time algorithm for PIGEONHOLE CIRCUIT to C. Since π is a permutation, the algorithm cannot return two inputs giving the same output, hence it returns y, s.t. C(y) = 0, i.e. π(x) = y

Two more propositions • Prop2: PPAD PPP • Proof: • Reformulate any problem in PPAD with graph G as a problem in PPP by use of π(x) defined as • π(x) = x if x is a sink • π(x) = G2(x) if x is a nonstandard source • π(x) = G(x) otherwise • Trivially, from any confluence of values of π we can recover a nonstandard source or a sink of G

The class PLS • PLS originally defined in [Johnson, et. al. ‘88]: “find some local optimum in a reasonable search space”. • Basic ingredient: a problem with a search space, i.e. a set of feasible solutions which has a neighborhood structure • Every problem L (e.g. TSP) in PLS has a set of instances DL (e.g. encodings of adjacency matrix) • Every instance x in DL has a finite set of solutions FL(x) (e.g. tours) considered wlog polynomial in size • For each solution s in FL(x) we have • A poly-time cost function c(x,s) that given an instance x and a solution s, it computes its cost (e.g. length of a tour) • A poly-time neighbor function N(x,s) FL(x) that given an instance x and a solution s, returns its neighboring solutions (e.g. a tour that differs from the original one in at most two edges)

The class PLS • Remaining constraints are expressed in terms of 3 polynomial time algorithms AL, BL, CL • AL, given an instance x in DL produces a particular standard solution of L, AL(x) € FL(x) (e.g. a standard tour for TSP) • BL, given an instance x in DL and a string s determines whether s is a solution (s € FL(x)?) and if so, it computes its cost c(x,s) • CL, given an instance x € DL and a string s € FL(x) has two possible outcomes, depending on s: • If there is any solution s’ € N(x,s) s.t. c(x,s’) < c(x,s) it outputs s’ • Otherwise it outputs “no”, which implies that s is locally optimal

The class PLS • The sense of local searching is obvious: starting at some arbitrary solution start searching locally for a better solution and move towards it. • Standard local search algorithm: Given x, use AL to produce a starting solution s= AL(x) Repeat until locally optimal solution is found: Apply algorithm CL to x and s If CL yields a better cost neighbor s’ of then set s=s’ • Since the solution space is finite the above algorithm always halts • If FL(x) is polynomially bounded then the above algorithm is polynomial • However, things can turn out pretty bad: • Finding a local optimum reachable from a specific state can be proven PSPACE-complete • There are instances with states exponentially far from any local optimum • Correct question to ask: “Given x, find some locally optimum solution s”

The class PLS • PLS-reduction: we must map the local optima of one problem to those of another, thus pairtaining the neighboring structure. Hence problem L reduces to K if there exist poly-time functions f and g s.t. • f maps instances x of L to instances f(x) of K • g maps (solutions of f(x), x) pairs to solutions of x • for all instances x of L, if s is a local optimum for instance f(x) of K, then g(s,x) is a local optimum for x • Basic PLS-complete problem: • weighted CIRCUIT – SAT under input bitflips • some relatives of TSP, MAXCUT and: • POS-NAE-3SAT: [Shaeffer&Yannakakis-91] Given an instance of NAE-3SAT with weights on its clauses and with positive literals only, find a truth assignment, whose total weight (i.e. the weight of the unsatisfied and totally satisfied clauses) cannot be improved by flipping a variable

Congestion GamesDefinitions • Idea: generalization of Selfish Task Allocation • Intuition: • Route traffic through a network • Resource allocation • We have • a set of resources E (network edges…?) • a set of players N • the resource delays de • For each player i • an action set Ai 2E • a weight (traffic demand) wi

Congestion GamesDefinitions • let A=(a1,…, an), ai€ Ai, be a strategy profile • denote by • Pe(A) the set of players picking resource e in profile A • ne(A) the number of players picking resource e in profile A, ne(A)=|Pe(A)| • define the load of resource e to be L(e) = Σiin P(e)wi • for each strategy profile A, each player incurs a total delay/cost ci(A) = Σein aiL(e)

Congestion GamesEquilibria and Potential Functions • A Pure Nash Equilibrium (PNE) is a strategy profile A=(a1,…, an), s.t. for all i € N and for all ai’ € Ai c(a1,…, ai,…, an) ≤ c(a1,…, ai’,…, an) • Notice the local optimality flavor of a PNE • Exact Potential for a c.g. is a function Φ:Ε→R s.t. for all i € N, ai’ € Ai and A ci(A) - ci(A-i, ai’) = Φ(A) - Φ(A-i, ai’) • It is easy to verify that every game that admits an exact potential possesses a PNE

Types of Congestion Games • Symmetric vs AsymmetricPlayers have the same action set and the same weights, i.e. they are indistinguishable • Network vs General Congestion Games • the resources are graph edges • each player has a source and a sink vertex (si,ti) • the action sets Ai are si-ti paths • Single vs multi-commodity (network) games • the (si,ti) pairs are common for all players

Congestion GamesExistence of PNE • Theorem [Rosenthal ’73]:Every unweighted congestion game is an exact potential game • The inverse holds [Mondere&Shapley]: Every finite exact potential game is isomorphic to an unweighted c.g. • So it is only for c.g. that we can find an exact potential • Does this mean that the potential technique (for guaranteeing existence of a local minimum) can be used only for c.g.?

Congestion GamesPotential Functions • We need to be less strict about the potential function. • Consider the party affiliation game (HAPPYNET): • n players (nodes in graph) • Symmetric preference relations between players-nodes • 2 actions per player: {-1,1} • State S is a mapping V{-1,1} (players pick sides) • payoff of i: sgn[S(i) ΣjS(j)wij] (if the payoff is positive then we say “node i is happy”) • The problem: “Find a state in which all nodes are happy” • intuition: everyone wants to be with his friends • application: convergence of Hopfield NN

On the Complexity of Search Problems George Pierrakos

On the Complexity of Search Problems George Pierrakos

Presentation Transcript

On the Complexity of Scheduling

On the Parallel Complexity of Hierarchical Clustering and CC -Complete Problems

Classification of Search Problems

Search Problems

On the Complexity of Join Predicates

Complexity of Approximating Constraint Satisfaction Problems

Resolving the Complexity of Some Data Privacy Problems

Search Problems

The Nature of Problems – Complexity

Combinatorial Problems III: The Next Level of Complexity

On the Complexity of Allocation Problems with Probabilistic Players

on the complexity of orthogonal compaction

Geometrical Complexity of Classification Problems

Geometrical Complexity of Classification Problems

On the Complexity of TSP with Neighborhoods and Related problems

Modeling the complexity of pain – possibilities and problems

Search Problems

On the Complexity of Search Problems George Pierrakos

On the Complexity of Asynchronous Gossip

The Complexity of Lattice Problems

Search Problems