CHAPTER 3 The Theory of NP-completeness

1. CHAPTER 3The Theory of NP-completeness S. J. Shyu

2. An Informal Discussion of the Theory of NP-completeness • NP-cpmpleteness 的理論之所以重要乃在於它指出了一群「困難」的問題（difficult problems, 其 lower bound 似乎有指數函數的 order）亦即 NP-complete 的理論，指出了一群似乎找不到 polynomail time algorithm 的問題。

3. NP-complete 的理論首先定出 NP （Non-deterministic Polynomial）problems；其中有難題也有容易的。

4. 到目前為止，所有 NP-complete 的問題在 the worst case 下都找不出任何 polynomial algorithm 來解！亦即任一 NP-complete 的問題在 the worst case 時目前最好的 algorithm 都得花 exponential 的時間才能解決。 • 注意：這兒強調的是 the worst case！也就是說有的 NP-complete 問題可能在 the average case 下，可在 polynomial 的時間解決。

5. NP-complete 理論宣稱：若某個 NP-complete 的問題可在 Polynomial time 內解掉，則所有 NP problem 皆可在 polynomial time 內解決。（或說 NP＝P。） • 既然『所有 NP problem 皆可在 P 時間內解掉』似乎不太可能，因此任一 NP-complete 問題可在 P 時間內解決的可能性亦不大。

6. 不過 NP-complete 的理論並沒有提到 NP-complete 的問題一定找不到 polynomial algorithm！它僅說明 NP-complete 的問題似乎相當不可能找到 polynomial algorithm；其關鍵在於到目前為止，尚未有人提出某個NP-complete 問題的 low bound （一旦這個 low bound 出現，NP-complete 的問題就不可能是 P 的問題，即 NP≠P 了！）。 • 換個角度思考，這個理論提醒了我們：面對 NP-complete 問題時，別去找 Polynomial time 的解法（而是找近似解，或針對其困難尋找應用！）

7. The Theory of NP-Completeness • P: the class of problems which can be solved by a deterministic polynomial algorithm. • NP : the class of decision problems which can be solved by a non-deterministic polynomial algorithm.

8. The Theory of NP-Completeness (cont.) • NP-hard:the class of problems to which every NP problem reduces. • NP-complete (NPC): the class of problems which are NP-hard and belong to NP.

9. Some concepts of NPC • Definition of reduction: Problem A reduces to problem B (A  B) iff A can be solved by a deterministic polynomial time algorithm using a deterministic algorithm that solves B in polynomial time. • Up to now, none of the NPC problems can be solved by a deterministic polynomial time algorithm in the worst case.

10. Some concepts of NPC (cont.) • It does notseem to have any polynomial time algorithm to solve the NPC problems. • The theory of NP-completeness always considers the worst case. • The lower bound of any NPC problem seems to be in the order of an exponential function.

11. Some concepts of NPC (cont.) • Not all NP problems are difficult. (e.g. the MST problem is an NP problem.) • If A, B  NPC, then A  B and B  A. • Theory of NP-completeness If any NPC problem can be solved in polynomial time, then all NP problems can be solved in polynomial time. (NP = P)

12. Decision Problems • The solution is simply “Yes” or “No”. • Optimization problems are more difficult. • e.g. the traveling salesperson problem • Optimization version: Find the shortest tour • Decision version: Is there a tour whose total length is less than or equal to a constant c ?

13. The satisfiability problem • The satisfiability problem • A logical formula : x1 v x2 v x3 & -x1 & -x2 the assignment : x1 ← F , x2 ← F , x3 ← T will make the above formula true . (-x1, -x2 , x3) represents x1 ← F , x2 ← F , x3 ← T

14. The satisfiability problem(cont.) • If there is at least one assignment which satisfies a formula, then we say that this formula is satisfiable; otherwise, it is unsatisfiable. • An unsatisfiable formula : x1 v x2 & x1 v -x2 & -x1 v x2 & -x1 v -x2

15. The satisfiability problem(cont.) • Definition of the satisfiability problem: Given a Boolean formula, determine whether this formula is satisfiable or not. • A literal : xi or -xi • A clause : x1 v x2 v -x3Ci • A formula : conjunctive normal form C1& C2 & … & Cm

16. The Resolution Principle • Resolution Principle C1 : x1 v x2 C2 : -x1 v x3 C3 : x2 v x3 • From C1 &C2, we can obtain C3, and C3 can be added into the formula. • We have a new formula: C1 &C2 &C3.

17. The Resolution Principle (cont.) • Another example of resolution principle C1 : -x1 v -x2 v x3 C2 : x1 v x4 C3 : -x2 v x3 v x4 • If no new clauses can be deduced, then it is satisfiable. -x1 v -x2 v x3 (1) x1 (2) x2 (3) (1) & (2) -x2 v x3 (4) (4) & (3) x3 (5) (1) & (3) -x1 v x3 (6) Thus this set of clauses is satisfiable.

18. The Resolution Principle (cont.) • If an empty clause is deduced, then it is unsatisfiable. - x1 v -x2 v x3 (1) x1 v -x2 (2) x2 (3) - x3 (4)  deduce   (1) & (2) -x2 v x3 (5) (4) & (5) -x2 (6) (6) & (3) □ (7) Not satisfiable.

19. The Resolution Principle (cont.) -x1-x2 x3 (1) x1 x4 (2) x2  -x1 . (3) A semantic tree. Satisfiable

20. The Resolution Principle (cont.) x1 (1) -x1 (2) A semantic tree. Unsatisfiable

21. The Resolution Principle (cont.) • In a semantic tree, each path from the root to a leaf node represents a class of assignments. • If each leaf node is attached with a clause, then it is unsatisfiable.

22. Nondeterministic Algorithms • A nondeterminstic algorithm consists of phase 1: guessingand phase 2: checking • If the checking phase of a nondeterministic algorithm is of polynomial time-complexity, then this algorithm is called an NP (nondeterministic polynomial) algorithm.

23. Nondeterministic Algorithms (cont.) • NP problems : (must be decision problems) • e.g. searching, • MST sorting satisfiability problem (SAT) traveling salesperson problem (TSP)

24. Decision Problems • Decision version of sorting: Given a1, a2,…, an and c, is there a permutation of ais ( a1, a2 , … ,an ) such that∣a2–a1∣+∣a3–a2∣+ … +∣an–an-1∣＜ c ?

25. Decision Problems (cont.) • Not all decision problems are NP problems. • E.g. halting problem : • Given a program with a certain input data, will the program terminate or not? • NP-hard • Undecidable

26. Guessing and Checking [Horowitz 1998] • Choice(S) : arbitrarily chooses one of the elements in set S • Failure : an unsuccessful completion • Success : a successful completion • Nonderministic searching algorithm: j ← choice(1 : n) /* guessing */ if A(j) = x then success /* checking */ else failure

27. Guessing and Checking (cont.) • A nondeterministic algorithm terminates unsuccessfully iff there exist no set of choices leading to a success completion. • The time required for choice(1: n) is O(1).

28. Cook’s theorem NP = P iff the satisfiability problem is a P problem. • SAT is NP-complete. • It was the first NP-complete problem proved. • Every NP problem reduces to SAT.

29. Transforming searching to SAT • Does there exist a number in { x(1), x(2), …, x(n) }, which is equal to 7? • Assume n = 2. nondeterministic algorithm: i = choice(1,2) if x(i)=7 then SUCCESS else FAILURE

30. Transforming searching to SAT (cont.) i=1 v i=2 & i=1 → i≠2 & i=2 → i≠1 & x(1)=7 & i=1 → SUCCESS & x(2)=7 & i=2 → SUCCESS & x(1)≠7 & i=1 → FAILURE & x(2)≠7 & i=2 → FAILURE & FAILURE → -SUCCESS & SUCCESS (Guarantees a successful termination) & x(1)=7 (Input Data) & x(2)≠7

31. Transforming searching to SAT (cont.) • CNF (conjunctive normal form) : i=1 v i=2 (1) i≠1 v i≠2 (2) x(1)≠7 v i≠1 v SUCCESS (3) x(2)≠7 v i≠2 v SUCCESS (4) x(1)=7 v i≠1 v FAILURE (5) x(2)=7 v i≠2 v FAILURE (6) -FAILURE v –SUCCESS (7) SUCCESS (8) x(1)=7 (9) x(2)≠7 (10)

32. Transforming searching to SAT (cont.) • Satisfiable with the following assignment : i=1 satisfying (1) i≠2 satisfying (2), (4) and (6) SUCCESS satisfying (3), (4) and (8) -FAILURE satisfying (7) x(1)=7 satisfying (5) and (9) x(2)≠7 satisfying (4) and (10)

33. The semantic tree

34. Searching for 7, but x(1)7, x(2)7 • CNF (conjunctive normal form) : i=1 v i=2 (1) i1 v i2 (2) x(1)7 v i1 v SUCCESS (3) x(2)7 v i2 v SUCCESS (4) x(1)=7 v i1 v FAILURE (5) x(2)=7 v i2 v FAILURE (6) SUCCESS (7) -SUCCESS v -FAILURE (8) x(1)  7 (9) x(2)  7 (10)

35. Searching for 7, but x(1)7, x(2)7 (cont.) • Apply resolution principle : (9) & (5) i1 v FAILURE (11) (10) & (6) i2 v FAILURE (12) (7) & (8) -FAILURE (13) (13) & (11) i1 (14) (13) & (12) i2 (15) (14) & (1) i=2 (16) (15) & (16) □ (17) We get an empty clause  unsatisfiable  7 does not exit in x(1) or x(2).

36. Searching for 7, where x(1)=7, x(2)=7 • CNF: i=1 v i=2 (1) i1 v i2 (2) x(1)7 v i1 v SUCCESS (3) x(2)7 v i2 v SUCCESS (4) x(1)=7 v i1 v FAILURE (5) x(2)=7 v i2 v FAILURE (6) SUCCESS (7) -SUCCESS v -FAILURE (8) x(1)=7 (9) x(2)=7 (10)

37. The semantic tree It implies that both assignments (i=1, i=2) satisfy the clauses.

38. The Node Cover Problem • Def: Given a graph G=(V, E), S is the node cover of G if S  V and for ever edge (u, v)  E, (u, v) is incident to a node in S. • Decision problem :  S  S  K  node cover : {1, 3} {5, 2, 4}

39. Transforming the Node Cover Problem to SAT BEGIN i1 choice({1, 2, …, n}) i2 choice({1, 2, …, n} – {i1})  ik choice({1, 2, …, n} – {i1, i2, …, ik-1}). For j=1 to m do BEGIN if ej is not incident to one of Vit(1tk) then FAILURE END SUCCESS

40. A Graph i1 = 1 i1 = 2 i1 = 3 (1) i1 ≠ 1 v1e1FAILURE (2) i1 ≠ 1 v1e2FAILURE (3) i1 ≠ 2 v2e1FAILURE (4) i1 ≠ 2 v2e2FAILURE (5) i1 ≠ 3 v3e1FAILURE (6) i1 ≠ 3 v3e2FAILURE (7) v1 e1 (8) v2e1 (9) v2e2 (10) v3e2 (11) SUCCESS (12) -SUCCESS  -FAILURE . (13) Transforming the Node Cover Problem to SAT (cont.)

41. Proofs to Cook’s Theorem NP = P iff the satisfiability problem is a P problem. • => • If NP=P, since SATNP (SAT has an NP algorithm), thus SATP. • <= • Consider ANP. A has an NP algorithm including guessing and checking. We could transform steps of guessing and checking into a boolean formula such that ASAT. • Then A can be solved by the P algorithm solving SAT. Therefore, NP=P.

42. Definition of NP-completeness A problem A is NP-complete if • (1) ANP • (2) every NP problem reduces to A。

43. SAT is NP-complete (1) SAT is an NP problem. (2) SAT is NP-hard: • Every NP problem can be transformed in polynomial time to SAT such that SAT is satisfiable if and only if the answer for the original NP problem is “YES”. • That is, every NP problem  SAT . • By (1) and (2), SAT is NP-complete.

44. The Transitivity of Reduction • If A1  A2 and A2  A3 , then A1  A3 . • （by definition）

45. Proof of NP-Completeness • To show that any problem A is NP-complete: (I)Prove that A is an NP problem. (II)Prove that  B  NPC, B  A.  A  NPC. • Why ?

47. SAT 3SAT 3DM VC PARTITION HC CLIQUE Six Basic Problems (Garey and Johnson 1979)

48. 3-satisfiability problem (3-SAT) • Def: Each clause contains exactly three literals. • (I)3-SAT is an NP problem (obviously) • (II)SAT  3-SAT Proof: (1) One literal L1 in a clause in SAT : In 3-SAT : L1 v y1 v y2 L1 v -y1 v y2 L1 v y1 v -y2 L1 v -y1 v -y2

49. 3-satisfiability problem (3-SAT) (cont.) (2)Two literals L1, L2 in a clause in SAT : In 3-SAT : L1 v L2 v y1 L1 v L2 v -y1 (3)Three literals in a clause : remain unchanged.

50. 3-satisfiability problem (3-SAT) (cont.) (4) More than 3 literals L1, L2, …, Lk in a clause : in 3-SAT : L1 v L2 v y1 L3 v -y1 v y2  Lk-2 v -yk-4 v yk-3 Lk-1 v Lk v -yk-3