NP-Complete

1. NP-Complete • What is an algorithm • What is a proof • What is a hard problem • NP-Complete problems -- practical problems that are strongly suspected to be computationally hard. Prabhas Chongstitvatana

2. Computability theory : When can this be computed? Can this be computed at all? Complexity theory : How cheaply can this be computed? How hard is this to compute? Inconsistencies 1900 Georg Cantor, invented set theory. Prabhas Chongstitvatana

3. August, 1900, The Second International Congress of Mathematicians in Paris. Hilbert, formalist school of mathematic, manipulated symbols by fixed rules without thinking of any possible meaning the symbols could have. Strict formalism, by not letting semantic interpretations influence how or when rules are applied, supposedly guarantees that whenever we prove something we are sure it is correct. Prabhas Chongstitvatana

4. Formal system A formal system is a finite set of assumed truths, called axioms, and a finite set of rule, called inference rules. We can use the rules on the axioms to infer new results called theorems. A proof of a theorem is a demonstration of a sequence of inferred theorems starting only with the axioms and using only the inference rules and ending with the theorem. Prabhas Chongstitvatana

5. One of Hilbert’s problem was to establish that arithmetic was consistent, meaning that, under the accepted rules of arithmetic, it is not possible to generate a contradiction. 1931 Kurt Godel (24), showed that if any system as "powerful" as arithmetic is consistent, then it cannot be complete -- meaning that some true statements are unprovable in the system! Prabhas Chongstitvatana

6. Today we don't know whether arithmetic is consistent; all we can say is that if it is, then there are true things we cannot prove in it. This is similar to the uncertainty principle in quantum mechanics, Heisenberg 1927. Hilbert's continuum problem : Cantor, there is more than one infinity. Using the proof called diagonalization, Cantor showed that there were more real numbers than there were integers. But no one knew if there were an infinity between the number of integers and the number of reals. Prabhas Chongstitvatana

7. r0 = 0.d00d01d02. . . • r1 = 0.d10d11d12. . . • r2 = 0.d20d21d22. . . • . . . • rn = 0.dn0dn1dn2. . .dnn • . . . • diagonal d00d11d22dnn. . . change each digit into the "opposite". This will be a new number not in the original listing. | N | < | R | • Are there any thing between | N | < | S | < | R | ? Prabhas Chongstitvatana

8. 1938 Godel showed that it couldn't be proved false from Cantor's axioms of set theory. 1963 Cohen showed that it couldn't be proved true from Cantor's axioms. The continuum hypothesis is independent of the usual axioms of set theory. If set theory is consistent to begin with, then we can assume the continuum hypothesis true or false and still have a consistent set theory! Prabhas Chongstitvatana

9. The formalist school was resisted by a group of mathematicians called constructivists (intuitionists) who wanted to toss infinite sets and rebuild mathematics from the bottom up starting from the integers and applying only finite operations to them. Prabhas Chongstitvatana

10. Model of computation Another Hilbert's problem was to find a "finitary way" to solve any diophantine equation. A diophantine equation is a polynomial equation in n unknowns that can have only integer solutions. Hilbert wanted an "algorithm" (not yet invented the name) 1970 Matijasevic (22) showed, using fibonacci numbers, that this problem is computationally unsolvable. Prabhas Chongstitvatana

11. Turing machine, Alan Turing (24) 1936 • Herbrand functions, Jacques Herbrand • Godel functions, Kurt Godel • Church calculus, Alonzo Church • Kleen functions, Stephen Kleen • Post systems, Emil Post • Markov algorithms, Andrei Markov • same computational ability Prabhas Chongstitvatana

12. Hilbert's problem : the decision problem -- asked whether there is an algorithm deciding the truth of falsify of any mathematical statement. Church and Turing, 1936, no such algorithm exists. The result is provable from Godel's work on incompleteness in 1931. What it meant for something to be computable. Prabhas Chongstitvatana

13. Traveling salesman problem • Knapsack problem • Graph coloring problem • Are some problems harder than the other? • Which ones? • How much harder are they? • 1965 Edmonds, Cobham, definition of a feasible problem : A problem is feasible if it has a solution whose cost is at most polynomial. Prabhas Chongstitvatana

14. Why polynomial • Polynomials are closed under composition and addition • All sequential digital computers are polynomially related • In general, a polynomial algorithm will do a feasible amount of work. Prabhas Chongstitvatana

15. 1947 Dantzig, algorithm for optimize linear programs, simplex. Simplex worst cost is exponential in the number of variables. 1979 Hacijan (kah-chi-yan) proved that an algorithm, ellipsoid algorithm, is polynomial. (multidimensional binary search). 1984 Karmakar, projective scaling, improved ellipsoid to run well in practice. Karmakar's algorithm is polynomial. Although Simplex is exponential, the linear programming problem is polynomial. Prabhas Chongstitvatana

16. NP-completeness polynomial time algorithm O( n^k ) = tractable superpolynomial time algorithm = intractable Prabhas Chongstitvatana

17. Class of problems NP-complete, status unknown No polynomial time algorithm has yet been discovered for any NP-complete problem. P != NP question (posed 1971) Believe NP-complete problems are intractable Prabhas Chongstitvatana

18. If any single NP-complete problem can be solved in polynomial time, then every NP-complete problem has a polynomial time algorithm. problem -> NP-complete -> approximate algorithm Prabhas Chongstitvatana

19. Define complexity P (formal language) complexity NP : decision problem where solutions can be verified in polynomial time. (more work on this slide) Prabhas Chongstitvatana

20. Abstract problems Binary relations of a set of problem instances to a set of solutions. Theory of NP-completeness restricts to decision problem : yes/no answer. Example Shortest Path Give a graph G = (V,E) and u, v in V, non negative integer k, does a path exist in G between u and v whose length is at least k? Prabhas Chongstitvatana

21. Cast optimization problems to decision problems by imposing a bound on the value to be optimized. Prabhas Chongstitvatana

22. Encoding An encoding of a set S of abstract object is a mapping e from S to the set of binary string. A problem whose instance set is the set of binary strings is called a concrete problem. An algorithm "solves" a concrete problem in O(T(n)) if when it is provided a problem instance i of length n = | i | the algorithm can produce the solution in at most O(T(n)). Prabhas Chongstitvatana

23. Complexity class P; the set of concrete decision problem that are solvable in polynomial time. Encoding map abstract problem Q to concrete problem. If solution to an abstract problem i inI is Q(i) in {0,1}, then the solution to the concrete problem instance e(i) in {0,1}* is also Q(i). Prabhas Chongstitvatana

24. We would like the definition to be independent of any particular encoding. An algorithm with the only input k is running in time Zeta(k) if k is unary ( a string of 1's ) its running time is O(n) if k is binary n = ceil( lg k) its running time is Zeta(2^n) Prabhas Chongstitvatana

25. A function f : {0,1}* -> {0,1}* is polynomial time computable if there exists a polynomial time algorithm A that, given input x in {0,1}*, produces as output f(x). For some set I of problem instances, two encoding e1 and e2 are polynomially related if there exist two polynomial time computable functions f12 and f21 such that for any i in I, we have f12(e1(i)) = e2(i) and f21(e2(i)) = e1(i). Prabhas Chongstitvatana

26. That is, encoding e2(i) can be computed from the encoding e1(i) by a polynomial time algorithm, and vice versa. Lemma 36.1 Let Q be an abstract decision problem on an instance set I, and let e1 and e2 be polynomially related encodings on I. Then, e1(Q) is-in P if and only if e2(Q) is-in P. Prabhas Chongstitvatana

27. “Standard” encoding we shall generally assume that problem instances are encoded in any reasonable, concise fashion. Encoding of an integer is polynomially related to its binary representation. Encoding of a finite set is polynomially related to its encoding as a list of elements. Prabhas Chongstitvatana

28. Formal language framework An algorithm Aaccepts a string x in {0,1}* if, given input x, the algorithm outputs A(x) = 1. The language accepted by an algorithm A is the set L = { x in {0,1}* | A(x) = 1 }. An algorithm rejects a string x if A(x) = 0. Prabhas Chongstitvatana

29. A language L is decided by an algorithm A if every binary string is either accepted or rejected by the algorithm. A language L is accepted in polynomial time by an algorithm A if for any length-n string x in L, the algorithm accepts x in time O(n^k) for some constant k. A language L is decided in polynomial time by an algorithm A if for any length-n x in {0,1}*, the algorithm decidesx in time O(n^k) for some constant k. Prabhas Chongstitvatana

30. Complexity class P P = {L subset {0,1}* | there exists an algorithm A that decides L in polynomial time}. Theorem 36.2 P = {L | L is accepted by a polynomial time algorithm}. Prabhas Chongstitvatana

31. Polynomial time verification Hamiltonian cycles Does a graph G have a Hamiltonian cycle? HAM-CYCLE = { <G> | G is hamiltonian grah } How an algorithm decide the language HAM-CYCLE? Prabhas Chongstitvatana

32. Given a problem instance <G>, lists all permutations of the vertices of G and then checks each permutation. With "reasonable" encoding of a graph as its adjacency matrix, the number m of vertices in the graph is Ohmega(sqrt(n)), n = | <G> | is the length of the encoding of G. The running time is Ohmega(m!) = Ohmega(sqrt(n)!) = Ohmega( 2 ^ sqrt n). Prabhas Chongstitvatana

33. Another easier problem : Given a graph G, claims to be hamiltonian with the prove as a list of vertices. It is easy to verify the proof by checking if the given list of vertices is a permutation of the vertices of V and whether each of the consecutive edges along the cycle actually exists in the graph. O(n^2). Thus a proof that a hamiltonian cycle exists in a graph can be verified in polynomial time. Prabhas Chongstitvatana

34. Define a verification algorithm , A , a two-argument algorithm, an input string x and a binary string y called a certificate. A verifies an input x if there exists a certificate y such that A(x,y) = 1. The language verified by a verification algorithm A is L = { x in {0,1}* | there exists y in {0,1}* such that A(x,y) = 1 }. Prabhas Chongstitvatana

35. An algorithm Averifies a language L if for any string x in L, there is a certificate y that A can use to prove that x is-in L. Prabhas Chongstitvatana