Intractable Problems

1. Intractable Problems Juan Carlos Guzmán CS 6413 Theory of Computation Southern Polytechnic State University

2. Problems Solvable by Computers • Decidable Problems • Problems solvable by Turing Machines (and computers) • How costly is the solution (xL)? • Time • # of steps of the Turing Machine • Space • # of cells used • These measurements are parameterized by the length of x

3. Some Real Functions

4. Time Complexity • A Turing Machine M is of time complexity T(n) if whenever M is given an input w of length n, M halts after making at most T(n) moves, regardless of whether or not M accepts • We will be interested in polynomial functions T(n)

5. Classes P and NP • Class P collects all problems that are solvable in polynomial time by a deterministic Turing Machine • Class NP is the analogous class for non-deterministic Turing Machines • Note that PNP • Is P=NP ? NP P

6. Sample Problems in P • Sorting • Bubblesort • Quicksort • Kruskal’s Algorithm • Minimum weight spanning tree in a graph • Encoding in a TM • Integers coded in binary, plus separators , ( ) • m, # of nodes • W, maximum edge weight • (i,j,w) for each edge from node i to node j with weight w

7. Sample Problems in NP • Everything in P • The Traveling Salesman Problem (TSP) • Does the graph have a hamiltonian circuit of weight W or less? • Encoding in TM • Graph, as before • Threshold • A hamiltonian circuit is one that visits every node of the graph only once!

8. Traveling Salesman Problem • The number of different possible hamiltonian circuit configurations grows as O(m!) • Larger than 2cm for any c • It will cost O(m!) to find a possible solution • Checking that a configuration is a possible solution is easy – O(m4) • DTM – seems not to be polynomial • NTM – O(m4), where m is the number of nodes

9. It is in NP Yes! Is it in P? We have no idea We are unable to exhibit a polynomial time algorithm on a TM We cannot prove that such algorithm does not exist Traveling Salesman Problem NP P

10. P=NP PNP Which Venn Diagram is Accurate? NP P (b) P=NP · TSP (a) · TSP NP P (c) · TSP

11. Polynomial-Time Reduction • Remember the reductions of the previous Chapter? • We are currently studying decidable problems • How about the transformation T? • We will now require that transformations be of polynomial-time complexity (by a DTM) Machine M’ for P1 Machine M For P2 Trans formation P1 instance P2 instance Yes No

12. Transformations, Again Machine M’ for P1 Machine M For P2 Accept Accept Trans formation w x Reject Reject

13. NP-Complete Problems • It is not known whether P = NP • In case P NP • Wouldn’t it be nice to find problems that would lie in NP – P? • Those are the NP-complete problems • An NP-complete problem L is a problem that satisfies the following • LNP • For every problem L’NP, there is a polynomial-time reduction from L’ to L • TSP is NP-complete

14. NP-Complete Problems • A typical NP-complete problem is one on a combinatorial space for which the best algorithm known falls in the generate-and-test category • Generate a possible solution • Test if it works • Solutions could be thought of being generated by a NTM • The test is of polynomial complexity

15. NP-Complete Problems – Theorems • If • P1 is NP-complete • P2 is in NP • There is a polynomial-time reduction from P1 to P2 • Then • P2 is NP-complete

16. NP-Complete Problems – Theorems • If some NP-complete problem L is in P, then P = NP

17. Which Is the First NP-Complete Problem? • To prove that a problem is in class NP-complete, we need to reduce it, in polynomial-time, to another that we already know is NP-complete • So, how do we get the first one? • The first one is the satisfiability problem

18. Boolean Expressions • A boolean expression is defined as • A boolean-valued variable (1=true, 0=false) • Two boolean expressions connected by a binary operator: either  (and) or  (or) • A boolean expression prefixed with the unary operator  • A parenthesized boolean expression

19. Boolean Expressions • x  y  z • x1 (x10 x11) • (x  z) (x  z) x = true y = true z = false x1 = true x10 = false x11 = false x = false z = false

20. Problem satisfiability of boolean expressions Is there a model (variable assignment) that makes the expression evaluate to true? Input A boolean expression Output “Yes”, iff there is model for the expression Reduction From NTM that implements any NP-complete problem Satisfiability of CNF • The expression (x1 x2  x3)  (x1 x2  x3) is satisfiable: • x1 = false • x2 = true • x3 = false

21. SAT is NP-Complete • SAT is NP • Any other NP problem is reducible to SAT in polynomial-time • Use a restricted TM • Never writes a blank symbol • Never goes past the initial leftmost character • The proof needs to show the transformation • It needs to show that an instance accepted in any other NP-complete problem is equivalent of a (yet to be designed) boolean expression being feasible

22. How to Reduce Any Other NP-Complete Problem to SAT • Any NP problem is solved by a nondeterministic Turing Machine in polynomial time • Design a boolean expression that is satisfiable if and only if the Turing Machine is able to accept the instance of the problem • Each instance will be reduced to a (different) boolean expression

23. Reducing to SAT • Features on the NTM of complexity p(n) • Cannot move to the left of the input • Cannot write back a blank symbol • Only the first p(n) symbols of the NTM will be controlled (after all, the machine will never be beyond those elements)

24. Reducing to SAT • How to design the boolean expression • Must be true only in the case that the Turing Machine accepts the instance of the problem (its input)

25. … B X1 X2 … Xi … Xn B … q Instantaneous Descriptions • An instantaneous description is a sequence X1X2…Xi-1qXi…Xn, where • Xk in Γ • q in Q • All Xi’s are nonblank, except when q is at one of the extremes, when there may be a sequence of blanks between q and the string • The elements not represented in the tape are all blank • In this description, q is scanning Xi, so, if we were more sophisticated, we could write

26. A Turing Machine Acceptance • The initial ID (ID0) must correspond to q0wB…B • The final ID (IDp(n)) must land in a final state • Each ID logically entails the next ID • If a tape character is “far” from the one being scanned, then it will not be changed from one ID to the next • Otherwise, the following are implied by the transition function • the new character of the tape • The new scanned character

27. Simplifications • In order to make the construction of this humongous expression more systematic, the following will be adopted • Only the first p(n)+1 ID’s will be considered • The machine must stop by then • only the first p(n) symbols will be represented in the tape • All these symbols will be represented in the ID’s • Although the machine actually may stop before time p(n), we will assume that machines repeat their ID’s once stopped (i.e., IDp=IDp+1)

28. Sequence of ID’s XijQΓ Proposition “Xij has value a” is represented by variable yija

29. Initial Configuration Specs • The state is q0, the initial state • The initial part of the tape contains w = a1 … an • All other tape cells are blank • X0,0q0  • X0,1a1  … X0,n = an  • X0,n+1B  … X0,p(n) = B

30. Final Configuration Specs • Xp(n),0F  …  Xp(n),p(n)F

31. Machine Movement • Need to capture the notion of i ├i+1 • Very complex, as for each element in the ID, it needs to consider the case whether it is a state symbol or a tape symbol • Both cases are possible, but only one will be true at a time • To handle state symbols, the whole NTM’s transition relation will be encoded!

32. Machine Movement • Need to capture the notion of i ├i+1 • For Xi,j is not close to a state symbol, then Xi+1,j = Xi,j • Bi,jXi,j-1 Γ  Xi,jΓ  Xi,j+1 Γ  Xi+1,j=Xi,j i … Xi,j-1 Xi,j Xi,j+1 … i+1 … Xi,j …

33. Machine Movement • For any possible movement to the left • (p,C,left)  δ(q,A) • Ai,jXi,j-1 Γ  Xi,j=q  Xi,j+1 = A  Xi+1,j-1 =p  Xi+1,j=Xi,j-1  Xi+1,j+1 =C i … Xi,j-1 q A … i+1 … p Xi,j-1 C …

34. Machine Movement • For any possible movement to the right • (p,C,right)  δ(q,A) • Ai,jXi,j-1 Γ  Xi,j=q  Xi,j+1 = A  Xi+1,j-1 = Xi,j-1  Xi+1,j=C Xi+1,j+1 =p i … Xi,j-1 q A … i+1 … Xi,j-1 C p …

35. Boolean Expression • The desired boolean expression is • EM,w = S  N  F

36. Representation in Boolean Variables

37. CSAT Problem • Normal Forms • Boolean expressions can be transformed as either • Conjunctive Normal Form • An and of or’s (product of sums), or a (b  c)  (b d ) • Disjunctive Normal Form • An or of and’s (sum of products) (b c )  (a b  d ) • In either case, the literals are variables or their negation

38. Problem satisfiability of boolean expressions in conjunctive normal form Input A boolean expression in CNF Output “Yes”, iff there is model for the expression Reduction From SAT Satisfiability of CNF The expression (x1+x2+x3)(x2+x3+x4)(x3+x4+x1) is satisfied whenever x2 and x3 are true It is also satisfied when x1,x2 andx3 are false

39. Any boolean expression can be written in CNF Use algebraic laws to simplify the expressions De Morgan (E1E2)  (E1)(E2) (E1E2)  (E1)(E2) Double negation ((E))  E The transformed expression is satisfiable iff the original one is Reduction from SAT to CSAT

40. Problem satisfiability of boolean expressions in conjunctive normal form where each clause consists of exactly the sum of three literals Input A boolean expression in 3-CNF Output “Yes”, iff there is model for the expression Reduction From CSAT Satisfiability of 3-CNF The expression (x1+x2+x3)(x2+x3+x4)(x3+x4+x1) is satisfied whenever x2 and x3 are true It is also satisfied when x1,x2 andx3 are false

41. Any expression in CNF can be written in 3-CNF Any clause of more than three literals can be replaced by several of exactly 3 literals (x1+x2+…+xm) Is replaced by (x1+x2+y1) (x3+y1+y2) (x4+y2+y3) … (xm-2+ym-4+ym-3) (xm-1+xm+ym-3) The transformed expression is satisfiable iff the original one is Reduction from CSAT to 3SAT

42. Problem A subset of the nodes of a graph such that neither node is connected to any other in the subset Input A graph G and lower bound k Output “Yes”, iff G has an independent set of, at least, k nodes Reduction From 3SAT Independent Sets 1 2 3 4

43. A 3-CNF (a1+b1+c1) … (an+bn+cn) is satisfiable iff each clause has a variable that evaluates to true Build a graph with all occurrences of literals as nodes such that An independent set (of size n) must include a literal taken from each clause The independent set cannot include a literal and its complement An independent set (of size n) exists iff their literals form a model for the expression Reduction from 3SAT to IS (x1+x2+x3)(x2+x3+x4)(x3+x4+x1) x1 x2 x3 x2 x3 x4 x3 x4 x1

44. Problem A subset of the nodes of a graph such that all edges are connected to at least one node in the set Input A graph G and upper bound k Output “Yes”, iff G has a set of, at most, k edges that covers the graph Reduction From IS Node Cover 1 2 3 4

45. The graph to the left has an independent set k1=2 nodes iff the one to the right has a cover set of k2= 4-2 = 2 A subset of the nodes is independent iff its complement covers the graph Reduction from IS to NC 1 2 3 4 1 2 3 4

46. Problem A directed cycle in G that passes through each node only Input A directed graph G Output “Yes”, iff G has a directed Hamilton circuit Reduction From 3SAT Directed Hamilton-Circuit 1 2 3 4

47. Problem An undirected cycle in G that passes through each node only Input An undirected graph G Output “Yes”, iff G has a Hamilton circuit Reduction From DHC Undirected Hamilton-Circuit 1 2 3 4

48. The graph to the left has a directed Hamilton circuit iff the one to the right has an undirected Hamilton circuit Reduction DHC to HC 1(1) 1(2) 2(0) 2(1) 1(0) 2(2) 1 2 3(2) 4(0) 3(1) 3(0) 4(2) 4(1) 3 4

49. Problem A Hamilton circuit with weights associated to edges Input An undirected graph G with integer weights, and an upper limit k Output “Yes”, iff G has a Hamilton circuit whose sum of weights of edges os less than k Reduction From HC TSP answers “yes” for k  63 Traveling Salesman 12 15 1 2 10 20 3 4 18

50. The graph to the left has a Hamilton circuit iff the one to the right solves TSP for k=5 Reduction from HC to TSP 1 1 1 2 1 1 3 4 1 1 2 3 4