1 / 68

The Theory of NP-Completeness

The Theory of NP-Completeness. Chapter 8. 8-1 Concepts. Two categories of problems: Polynomial time algorithm : searching O(log n) 、 polynomial evaluation O(n) 、 sorting O(n log n) 、 string editing O(mn) Exponential time algorithm : TSP O(n 2 2 n ) 與 0/1 knapsack O(2 n/2 )

amoriarty
Download Presentation

The Theory of NP-Completeness

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Theory of NP-Completeness Chapter 8

  2. 8-1 Concepts • Two categories of problems: • Polynomial time algorithm: searching O(log n)、polynomial evaluation O(n)、sorting O(n log n)、string editing O(mn) • Exponential time algorithm: TSP O(n2 2n)與 0/1 knapsack O(2n/2) • 第二類的問題其實也無法證明它們一定沒有 polynomial time algorithm，但是就是找不出來。 • NP-complete 的理論就是要討論哪些問題是屬於前面所說的第二類？這些問題有什麼樣的特性等等。 • 在討論這些沒有 polynomial time algorithm 的問題時，我們再將這些問題分成兩類，一是 NP-complete、另一個是 NP-hard。 • 一個 NP-complete的問題有 polynomial time 的演算法  所有 NP-complete 的問題都有 polynomial time algorithm。兩個互為充份必要條件。 • 一個 NP-hard的問題如果有 polynomial time algorithm 則所有 NP-complete 的問題也都有 polynomial time algorithm。

  3. 8-1 Concepts • 目前所講的演算法都有一個前提假設，就是它的每個運算(operation)的結果都是獨一(確定)的。這種性質的演算法可以在實際的電腦上執行，稱為 deterministic algorithms. • 在討論計算理論時，可以將這種限制拿掉，也就是可以假設一個運算的結果不唯一，可能是某 n個結果中的一個，而且一定會是正確的那一個。這樣子的演算法稱為 non-deterministic algorithm。這種演算法無法在實際的電腦上執行。

  4. 8-1 Concepts A non-deterministic algorithm terminates unsuccessfully if and only if there exists no set of choices leading to a success signal。只有在不可能有正確答案的情形下， non-deterministic algorithm 才無法成功地完成工作。 • 一個可以執行 non-deterministic algorithm 的機器稱為 non-deterministic machine. (That machine does not exit in the world) • 我們定義下列三個函數來說明這種演算法。 • Choice(S): 從 S中選一個正確的元素出來。 • Failure(): 沒有成功地完成工作。 • Success(): 成功地完成工作 • 這三個函數的執行時間都是 O(1)。

  5. 8-1 Concepts • 要如何來看待 non-deterministic algorithm 呢？ • 可以用平行處理的角度來看，也就是當作同時有很多很多機器(依據所有可能資料集合)一起作同一個問題，每個機器用不同的 choice 的結果來作，如果有一個成功的話，其他的就不用再作；如果有一個失敗、就自己停下來即可。 • 另外，也可以當作實際上可能有一種方法可以選擇一個正確的答案出來，只是我們還不曉得而已（上帝知道），所以答案可能會存在。

  6. 8-1 Concepts • NP: the class of decision problems which can be solved by a non-deterministic polynomial algorithm. • P: the class of decision problems which can be solved by a deterministic polynomial algorithm. • NP-complete: The problems which belong to NP. • Until now, no deterministic polynomial algorithm can solve it. • Such as, the satisfiability problem (the first NPC problem) • NP-hard: • the class of problems to which every NP problem reduces. • An optimization problem if its correspondingdecisionproblem is NP-complete.

  7. 8-1 Concept of reduction • Def: Problem A reduces to problem B (A  B) iff A can be solved by a deterministic polynomial time algorithm using a deterministic algorithm that solves B in polynomial time.

  8. 8-1 Some concepts • Up to now, none of the NPC problems can be solved by a deterministic polynomial time algorithm in the worst case. • It does not seem to have any polynomial time algorithm to solve the NPC problems. • The lower bound of any NPC problem seems to be in the order of an exponential function. • The theory of NP-completeness always considers the worst case.

  9. 8-1 Some concepts • Not all NP problems are difficult. (e.g. the MST problem is an NP problem.) • If A, B  NPC, then A  B and B  A. • Theory of NP-completeness If any NPC problem can be solved in polynomial time, then all NP problems can be solved in polynomial time. (NP = P)

  10. 8-2 Decision problems • Any problem for which the answer is either zero or one is called a decisionproblem. An algorithm for a decision problem is termed a decisionalgorithm. • Any problem that involves the identification of an optimal (either minimum or maximum) value of a given cost function is known as an optimizationproblem. An optimizationalgorithm is used to solve an optimization problem. • The solution is simply “Yes” or “No”. • Optimization problem : more difficult than Decision problem • E.g. the traveling salesperson problem • Optimization version: Find the shortest tour • Decision version: Is there a tour whose total length is less than or equal to a constant C ?

  11. 8-2 0/1 Knapsack Problem • 共有 n個東西，每個東西的價值為 p1, p2, …, pn, 每個東西的重量為 w1, w2, …, wn, 一個袋子的重量限制為 m，重量與價值都至少為 0。 • 其 optimization 的版本是：所能夠裝的東西之最大價值是多少？ • decision 的版本為：是否能裝超過某個價值 r？ • decision 版本的問題若沒有 polynomial time algorithm，則 optimization 版本的問題也沒有 polynomial time 演算法。

  12. 8-2 Solving an optimization problem by a decision algorithm : • Solving TSP optimization problem by decision algorithm : • Give c1 and test Give c2 and test  Give cn and test • Decide ci’s by binary search the range 1~K, where K may be selected as the sum of all edges or by some heuristics (approximation algorithm to be described in Chap.9)

  13. 8-3 The satisfiability problem • The satisfiability problem • The logical formula : x1 v x2 v x3 & - x1 & - x2 the assignment : x1 ← F , x2 ← F , x3 ← T will make the above formula true (-x1, -x2 , x3) represents x1 ← F , x2 ← F , x3 ← T

  14. 8-3 The satisfiability problem • If there is at least one assignment which satisfies a formula, then we say that this formula is satisfiable; otherwise, it is unsatisfiable. • An unsatisfiable formula : x1 v x2 & x1 v -x2 & -x1 v x2 & -x1 v -x2

  15. 8-3 The satisfiability problem • Def : Given a Boolean formula, determine whether this formula is satisfiable or not. • A literal : xi or -xi • A clause : x1 v x2 v -x3ci • A formula : conjunctive normal form (CNF) c1& c2 & … & cm

  16. The resolution principle • Resolution principle C1 : x1 v x2 C2 : -x1 v x3  C3 :x2 v x3 • From C1 &C2, we can obtain C3, and C3 can be added into the formula. • The formula becomes: C1 &C2 &C3

  17. 8-3 The satisfiability problem • Resolution principle c1 : -x1 v -x2 v x3 c2 : x1 v x4  c3 :-x2 v x3 v x4 (resolvent) • If no new clauses can be deduced satisfiable -x1 v -x2 v x3 (1) x1 (2) x2 (3) (1) & (2) -x2 v x3 (4) (4) & (3) x3 (5) (1) & (3) -x1 v x3 (6)

  18. 8-3 The satisfiability problem • If an empty clause is deduced  unsatisfiable - x1 v -x2 v x3 (1) x1 v -x2 (2) x2 (3) - x3 (4)  deduce (1) & (2) -x2 v x3 (5) (4) & (5) -x2 (6) (6) & (3) □ (7)

  19. 8-3 Semantic tree • In a semantic tree, each path from the root to a leaf node represents a class of assignments. • If each leaf node is attached with a clause unsatisfiable.

  20. 8-4 Nondeterministic algorithms • Guessing • Checking • If the checking stage of a nondeterministic algorithm is of polynomial time-complexity, then this algorithm is called an NP (nondeterministic polynomial) algorithm. • NP problems : (must be decision problems) • E.g. searching, MST (Final examination, NP algorithm) sorting satisfiability problem (SAT) traveling salesperson problem (TSP)

  21. 8-4 Decision version of sorting • Given a1, a2,…, an and c, is there a permutation of ais ( a1, a2 , … ,an ) such that∣a2–a1∣+∣a3–a2∣+ … +∣an–an-1∣＜ C ? • Not all decision problems are NP problems • E.g. halting problem : • Given a program with a certain input data, will the program terminate or not? • NP-hard • Undecidable • Because it can’t be solved by three functions: Choice, failure, success

  22. 8-4 Nondeterministic algorithms • Choice(S) : arbitrarily chooses one of the elements in set S • Failure : an unsuccessful completion • Success : a successful completion

  23. 8-4 Nondeterministic searching : j ← choice(1 : n) /* guess if A(j) = x then success /* check else failure • A nondeterministic algorithm terminates unsuccessfully iff there exist no set of choices leading to a success signal. • The time required for choice(1 : n) is O(1). • A deterministic interpretation of a non-deterministic algorithm can be made by allowing unbounded parallelism in computation.

  24. 8-4 Nondeterministic sorting • B ← 0 for i = 1 to n do Guessing j ← choice(1 : n) if B[j] ≠ 0 then failure B[j] = A[i] for i = 1 to n-1 do Checking if B[i] > B[i+1] then failure success • Note: • Nondeterministic algorithm: O(n) • Deterministic algorithm: O(nlogn) Generating a list Testing the list

  25. Nondeterministic SAT • Guessing for i = 1 to n do xi← choice( true, false ) if E(x1, x2, … ,xn) is true Checking then success else failure

  26. 8-5 Cook’s theorem NP = P iff the satisfiability problem is a P problem. • SAT is NP-complete. • It is the first NP-complete problem. • Every NP problem reduces to SAT. Stephen Arthur Cook Transforming searching to SAT

  27. 8-5 Transforming searching to SAT • Does there exist a number in { x(1), x(2), … x(n) }, which is equal to 7? • Assume n = 2, x(1)=7, x(2)≠7

  28. 8-5 Transforming searching to SAT i=1 v i=2 & i=1 → i≠2 & i=2 → i≠1 & x(1)=7 & i=1 → SUCCESS & x(2)=7 & i=2 → SUCCESS & x(1)≠7 & i=1 → FAILURE & x(2)≠7 & i=2 → FAILURE & FAILURE → -SUCCESS & SUCCESS (Guarantees a successful termination) & x(1)=7 (Input Data) & x(2)≠7

  29. 8-5 Transforming searching to SAT • CNF (conjunctive normal form) : i=1 v i=2 (1) i≠1 v i≠2 (2) x(1)≠7 v i≠1 v SUCCESS (3) x(2)≠7 v i≠2 v SUCCESS (4) x(1)=7 v i≠1 v FAILURE (5) x(2)=7 v i≠2 v FAILURE (6) -FAILURE v -SUCCESS (7) SUCCESS (8) x(1)=7 (9) x(2)≠7 (10)

  30. 8-5 Transforming searching to SAT • Satisfiable at the following assignment : i=1 satisfying (1) i≠2 satisfying (2), (4) and (6) SUCCESS satisfying (3), (4) and (8) -FAILURE satisfying (7) x(1)=7 satisfying (5) and (9) x(2)≠7 satisfying (4) and (10) Note: ex. i=1 is a literal and i≠1 is its negative

  31. 8-5 The semantic tree

  32. 8-5 Searchingin CNF with inputs • Searching for 7, but x(1)7, x(2)7 CNF :

  33. 8-5 Searchingin CNF with inputs • Apply resolution principle :

  34. 8-5 Searchingin CNF with inputs

  35. 8-5 The semantic tree It implies that both assignments (i=1, i=2) satisfy the clauses.

  36. 8-6 NP-Complete • Def: Problem A reduces to problem B (A  B) iff A can be solved by a deterministic polynomial time algorithm using a deterministic algorithm that solves B in polynomial time.

  37. 8-6 SAT is NP-complete • Every NP problem can be solved by an NP algorithm • Every NP algorithm can be transformed in polynomial time to an SAT problem • [Horowitz and Shani 1998] • Such that the SAT problem is satisfiable iff the answer for the original NP problem is “yes” • That is, every NP problem  SAT • SAT is NP-complete

  38. 8-6 NP-Complete • A problem L is NP-hard if and only if satisfiability reduces to L ( SAT problemL). • A problem L is NP-complete if and only if L is NP-hard and L NP. • To show that a problem L2 is NP-hard, it is adequate to show L1L2, where L1 is some problem already known to be NP-hard. • To show that an NP-hard decision problem is NP-complete, we have just to exhibit a polynomial time non-deterministic algorithm for it.

  39. 8-7 Proof of NP-Completeness • To show that A is NP-complete (I)Prove that A is an NP problem (Find a non-deterministic polynomial algorithm to solve it, such as sorting or searching) (See Sections 3-4 ~ 3-5, Sorting and searching problems) (II)Prove that  B  NPC, B  A •  A  NPC • Two problems L1 and L2 are said to be polynomially equivalent iff L1L2 and L2L1.

  40. 8-7 3-satisfiability problem (3-SAT) • Each clause contains exactly three literals. (I)3-SAT is an NP problem (obviously See 3-24 slide) (II)SAT  3-SAT (Transformation rules, Prove its correctness) • One literal L1 in a clause in SAT : in 3-SAT : L1 v y1 v y2 L1 v -y1 v y2 L1 v y1 v -y2 L1 v -y1 v -y2

  41. 8-7 3-satisfiability problem (3-SAT) • Two literals L1, L2 in a clause in SAT : in 3-SAT : L1 v L2 v y1 L1 v L2 v -y1 • Three literals in a clause : remain unchanged

  42. 8-7 3-satisfiability problem (3-SAT) • More than 3 literals L1, L2, …, Lk in a clause : in 3-SAT : L1 v L2 v y1 L3 v -y1 v y2  Lk-2 v -yk-4 v yk-3 Lk-1 v Lk v -yk-3

  43. 8-7 3-satisfiability problem (3-SAT) • Transform to an instance in 3-SAT : x1 v x2 v y1 x1 v x2 v -y1 -x3 v y2 v y3 -x3 v -y2 v y3 -x3 v y2 v -y3 -x3 v -y2 v -y3 x1 v -x2 v y4 x3 v -y4 v y5 -x4 v x5 v -y5 • E.g. an instance in SAT : x1 v x2 -x3 x1 v -x2 v x3 v -x4 v x5 SAT transform 3-SA S S

  44. 8-7 3-satisfiability problem (3-SAT) • Proof : S is satisfiable  S is satisfiable “”  3 literals in S (trivial) consider  4 literals S : L1 v L2 v … v Lk S: L1 v L2 v y1 L3 v -y1 v y2 L4 v -y2 v y3  Lk-2 v -yk-4 v yk-3 Lk-1 v Lk v -yk-3

  45. 8-7 3-satisfiability problem (3-SAT) • S is satisfiable at least Li = T in this clause of S Assume : Lj = F  j  i assign : yi-1 = F • yj = T  j  i-1 yj = F  j  i-1 ( Li v -yi-2 v yi-1 ) S is satisfiable.

  46. 8-7 3-satisfiability problem (3-SAT) • “” If S is satisfiable, then assignment satisfying S can not contain yi’s only  at least Li must be true. (We can also apply the resolution principle). •  3-SAT is NP-complete 

  47. 8-7 Caution ! • If a problem is NP-complete, its special cases may or may not be NP-complete. • The 0/1 knapsack problem is NP-complete • The knapsack problem is not NP-complete

  48. 8-7 The knapsack problem • The greedy algorithm: Step 1: Sort pi/wi into non-increasing order. Step 2: Put the objects into the knapsack according to the sorted sequence as possible as we can.

  49. 8-7 The knapsack problem • E.g. n = 3, M = 20, (p1, p2, p3) = (25, 24, 15) (w1, w2, w3) = (18, 15, 10) Sol: p1/w1 = 25/18 = 1.32 p2/w2 = 24/15 = 1.6 p3/w3 = 15/10 = 1.5 Optimal solution: x1 = 0, x2 = 1, x3 = 1/2

  50. Note: NP-Complete problem • [0/1 Knapsack].Consider the knapsack problem discussed in the previous section. We add the requirement that xi = 1 or xi = 0, 1≦i ≦n; i.e., an object is either included or not included into the knapsack. We wish to solve the problem One greedy strategy is to consider the objects in order of nonincreasing density pi/wi and add the object into the knapsack if it fits.

More Related