1 / 67

Three basic concepts: Decision Problems: output yes/no

Three basic concepts: Decision Problems: output yes/no Nondeterministic algorithm: certificate additional input Polynomial transformation: one problem input to another type problem input in poly-time. 1. DECISION PROBLEMS. Decision Problems: output is “True/False,” “Yes/No”

magdalenae
Download Presentation

Three basic concepts: Decision Problems: output yes/no

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Three basic concepts: Decision Problems: output yes/no Nondeterministic algorithm: certificate additional input Polynomial transformation: one problem input to another type problem input in poly-time (C) Debasis Mitra

  2. 1. DECISION PROBLEMS Decision Problems: output is “True/False,” “Yes/No” Example (sorting-decision problem): Input: a list of numbers L Output, an answer: is L sorted? Complexity of Decision Problem? Sorting problem P: lower bound Ω(n logn), but Sorting-decision problem PD : O(n) (C) Debasis Mitra

  3. ADVANTAGE OF DECISION PROBLEMS Every problem P has a corresponding related decision problem PD, Has a lesser or equal complexity Logical output to work with makes it formal Hence, complexity theory (or, any computer science formal theory) uses only decision problems But, the results are valid for all types of problems (C) Debasis Mitra

  4. Example (0-1 Knapsack-decision problem): Input: a list of objects with (weight, profit), a knapsack upper bound by weight M, and a profit lower bound t Output: Does there exist a ks_profit ≥ t by choosing some objects with ks_wt ≤ M? Complexity? Which one is lower – original 01KS or 01KSD? 01KSD algorithm: Backtrack just like before for 01KS but may terminate as soon as a valid ks_profit ≥ tfound, need not go over all branches. EXAMPLE DECISION PROBLEM (C) Debasis Mitra

  5. USE OF DECISION PROBLEMS Algorithm A(PD ) for a decision problem PD may be used to solve original problem P, and vice versa Example of sorting, using sorting-decision algorithm: Sort (input L): For each permutationp of input list L if AlgoSortingD(p) == True then return p; SortingD (input L): If any comparison within the algorithm Sorting(L) leads to an actual exchange of elements, then return “False” (L was not sorted) Similarly Alg(01KSD) may be used to solve 0-1KS: Hint: binary search over ks_profit bound variable t Reverse is easy: ?? (C) Debasis Mitra

  6. MORE CONCEPTS:::2. NON-DETERMINISTIC ALGORITHMS (C) Debasis Mitra

  7. NON-DETERMINISIM Non-deterministic version of a problem: Presume the solution exists: “certificate” Problem is simplified: Check only the certificate, Is the certificate correct answer? – return “True/False” If an algorithm can check certificate in polynomial time => The problem is Non-deterministically Polynomial, in NP-class (C) Debasis Mitra

  8. NON-DETERMINISTIC PROBLEMS:EXAMPLE SortingD is an ND problem: Input: (1) a list of numbers L, and (2) a certificate c (another list) Output: Is c a sort of L? Algorithm? … Can you find a polynomial-time algorithm? If your answer is yes: SortingD problem is in NP-class Is SortingD also a P-class problem? (C) Debasis Mitra

  9. NON-DETERMINISIM IS NOT A MISTERY Non-deterministic algorithm: Given the certificate, it traverses only one path from root to a leaf: bounded by depth of the tree Deterministic algorithm: Traverses all nodes of the tree in worst case [ - - - ] O1: No O1: Yes (w=0, p=0) OptP=33 (w=5, p=3) OptP=0 O2: Yes B =33, B == OptP (w=5+3, p=33) OptP=0 (w=3, p=30) OptP=33 OptP is still 33 B =65 > OptP Pruned because OptP cannot be improved O3: Yes O3: No B =56, but OptP now =56 B > OptP X (w=3, p=30) OptP=33, not updated (w=3+8, ---) pruned (w=8, p=33) OptP=33, updated (w=8, p=56) OptP=56, updated (w=8+8, ---) pruned Deterministic Machine versus Non-deterministic Machine (C) Debasis Mitra

  10. CLASSIFICATION OF PROBLEMS BY COMPLEXITY (C) Debasis Mitra

  11. COMPLEXITY OF PROBLEMS Polynomial algorithms are “faster” than Exponential algorithms, or more accurately P-algorithms has slower growth rate than that of E-algorithms, with respect to input size n O(nk) < O(an), a>1 and k≥0 are constants (C) Debasis Mitra

  12. COMPLEXITY THEORY Each Problem has a lower bound, for Any algorithm What is the lower bound for Comparison Sort? Problems may be classified by such bounds An important, but “unknown” classification would be: Polynomial-class versus “Exponential-class” Unfortunately: Life is much more complicated! There is no E-class of problems (with exponential lower bounds) (C) Debasis Mitra

  13. P-CLASS PROBLEMS AND BEYOND … P-class of problems: which have asymptotic Polynomial-time algorithms O(nk), for k≥0 There is no “E-class” of problems: No problem S may be designated as in E-class just because no polynomial algorithm for S has been found yet! Even if you have an exponential algorithm O(an), for a>1, that does not mean a polynomial algorithm will also not exist! We are talking about provable lower bound Ω (C) Debasis Mitra

  14. P-CLASS PROBLEMS AND BEYOND … Example: No polynomial algorithm for Multi-processor Last Finish Time (MPLFT) problem is found yet. That does not mean in future a polynomial alg will not be found! Although, it is not yet in P-class, it is not E-class either No exponential lower bound has been found yet as well That is all you can say about MPLFT However, ….continued, we use some new concepts …. (C) Debasis Mitra

  15. P VERSUS NP Say, problem X is in P-class (read: has a polynomial algorithm) Is X in NP-class as well? Read : has non-deterministic Polynomial algorithm also? Read: given a certificate, can you check it in polynomial-time? (C) Debasis Mitra

  16. P VERSUS NP Which one is faster? Polynomial Alg, or Non-det polynomial alg? ≡ What is more? Number of nodes in the tree? or the height of the tree? [ - - - ] (C) Debasis Mitra

  17. P VERSUS NPP  NP True NP-class P-class (C) Debasis Mitra

  18. P VERSUS NP Say, X is in NP-class Read: has a non-det polynomial alg Is X in P-class as well? Read: does it have a regular polynomial alg? [ - - - ] (C) Debasis Mitra

  19. P VERSUS NPNP  P True? P-class NP-class? (C) Debasis Mitra

  20. P VERSUS NP P  NP is True. But is NP  P also true? Meaning:NP ≡ P? A million dollar question: http://www.claymath.org/millennium-problems Jury is out there: NP ≡ P (i.e., NP  P and P  NP), XOR,NP ≠ P? (C) Debasis Mitra

  21. P VERSUS NPHow would you prove: NP  P not True?Or, that the truth is P  NP, a proper subset NP-class X P-class (C) Debasis Mitra

  22. P VERSUS NP LEADS TO COMPLEXITY HIERARCHY Jury is out there: Prove, NP  P, and so, NP ≡ P? Or, show P ≠ NP? Did not happen yet! But, can we find a “hardest” problem Hin NP class? And, then maybe look for a Poly-algorithm for that H? Or, find exponential lower bound for H? Looking for “hardest” needs some concepts on Polynomial Transformation (another new concept!!) (C) Debasis Mitra

  23. ANOTHER NEW CONCEPT!3. POLYNOMIAL PROBLEM-TRANSFORMATION (C) Debasis Mitra

  24. POLYNOMIAL PROBLEM-TRANSFORMATION How do you formally express a problem? Problem: (Input, Output) unambiguously expressed Problem instance: Specific values for Input, Remember: Output is T/F in decision problems (C) Debasis Mitra

  25. POLYNOMIAL PROBLEM-TRANSFORMATION Problem X can be transformed to another problem Y Problem Transformation: an algorithm TXY (input of X) := (input of Y) Such that, corresponding output of X and Y are same (T/F) TXY is an algorithm, like any other ones we have seen WAIT TO SEE EXAMPLE! TXY Y X (C) Debasis Mitra

  26. POLYNOMIAL PROBLEM-TRANSFORMATION Problem Transformation: an algorithm TXY TXY(input of X) => (input of Y) Such that, output T/F for (input of X) = output for TXY(input X) If there exists a TXY, and its time-complexity is Polynomial, then X is polynomially transformed to Y TXY Y X (C) Debasis Mitra

  27. SIGNIFICANCE OF POLYNOMIAL TRANSFORMATION If there exists a TXY, and time-complexity of it is Polynomial, then X is polynomially transformable to Y Suppose, Y has a polynomial algorithm Ay but, NO polynomial algorithm exists for X, YET! However, now you can solve X by Ay( TXY(X) )  T/F What is the time-complexity of this dual algorithm? TXY Y: apply poly alg AY on Y X: no poly alg yet X is True X is False (C) Debasis Mitra

  28. A CATCH IN POLYNOMIAL TRANSFORMATION AY(TXY(X))  T/F What is the time-complexity of this dual algorithm? Note: |(Youtput(TXY(X))| is a polynomial function of |X|, where | | indicates problem size Ay(Y) takes polynomial-time with respect to |Y| But, |Y| is a polynomial of |X|, as said above Hence: Ay(Y) above takes polynomial of polynomial-time over |X|, Or, polynomial-time with respect to |X| (C) Debasis Mitra

  29. SIGNIFICANCE OF POLYNOMIAL TRANSFORMATION Ay(TXY(X))  T/F Time-complexity of this dual algorithm is polynomial So, X is now in P-class Because Y was in P-class and polynomial transformation TXY is found So, X has an indirect polynomial algorithm now Note, this is not true in the opposite direction (C) Debasis Mitra

  30. SIGNIFICANCE OF POLYNOMIAL TRANSFORMATION Ay(TXY(X)) => T/F If Ay is polynomial, then the time-complexity of this dual algorithm is also polynomial IfY is in P-class, so is X Y is at least as “hard” as X Note, opposite direction is not necessarily true: If, source problem X is already in P-class (has poly alg), Then, polynomial transformation TXY does not say anything about the classification of target problem Y (C) Debasis Mitra

  31. COOK-LEVIN THEOREM All NP-class problems may be “expressed” generically as a machine: a Non-deterministic Turing Machine Cook and Levin 1971 https://en.wikipedia.org/wiki/Cook%E2%80%93Levin_theorem There exists a polynomial-transformation from this NDTM to the SAT problem: by an algorithm (Karp ‘72) (C) Debasis Mitra

  32. C-L-K’S THEOREM NP-class SAT problem Poly-trans P-class (C) Debasis Mitra

  33. C-L-K’S THEOREM Find a polynomial algorithm for SAT, and win a million dollar from Clay Institute! Then, Cook et al.’s polynomial transformation can be used to solve any NP-class problem in polynomial-time using “your polynomial-SAT” algorithm, or P=NP Alternatively, SAT is a good candidate for finding an exponential lower bound Then, P ≠NP will be proved (C) Debasis Mitra

  34. BOOLEAN SATISFIABILITY PROBLEMinPROPOSITIONAL LOGIC(SAT) (C) Debasis Mitra

  35. “SAT” PROBLEM Input: (U, V) U: a set of n Boolean variables {v1, v2, … vn}(value: T, F) V: a conjunctive normal formula (CNF) over U Conjunctive normal formula is conjunction of disjunctions: { (. v . v .) ^(. v . v . v .) ^ …} Output: Does there exist a satisfiable assignment to make C True? ≡ can you find values of U such that V becomes True? V: expressed as a set of m Clauses {C1, C2, … Cm} Ci: is a list (disjunction) of literals (a variable, or its negation) What is the complexity of finding above output? (C) Debasis Mitra

  36. EXAMPLE: SAT PROBLEM Input: (U, V) U: {a, b, c, d} V: {{a, b, c}, {a, d}, {~b, ~d}}, ~ indicates NOT in CNF: (a V b V c) ˄ (a V d) ˄ (~b V ~d) Output: Does there exist a satisfiable assignment to make V True? Answer: Yes A=T, b=F, c=?, d=? , question-mark ? means “don’t care” or T and F both correct What is the complexity of answering the above question? (C) Debasis Mitra

  37. SAT is in WHICH-CLASS? Input: (U, V) U: {a, b, c, d} V: {{a, b, c}, {a, d}, {~b, ~d}} a certificate: (a=F, b=T, c=T, d=T) Output: Is the certificate a satisfiable assignment to make V True? Answer: No C1 = T, C2 = T, C3 = F V = False What is the complexity of answering the above question? (C) Debasis Mitra

  38. SAT is in NP-CLASS Input: (U, V) U: {a, b, c, d} V: {{a, b, c}, {a, d}, {~b, ~d}} a certificate: (a=F, b=T, c=T, d=T) Output: Is the certificate a satisfiable assignment to make V True? Answer: No C1 = T, C2 = T, C3 = F: makes V = False Complexity: O(nm) So, SAT is in NP-class But not in P-class, “yet”: no one found a polynomial alg (C) Debasis Mitra

  39. C-L-K’S THEOREM: SAT IS NP-HARD Polynomial Transformation from NDTM to SAT SAT is NP-hard Either find a polynomial algorithm for SAT: then P=NP Or, find exponential lower bound for SAT: then P≠NP (C) Debasis Mitra

  40. C-L-K’S THEOREM: SAT IS NP-COMPLETE Polynomial Transformation from NDTM to SAT SAT is NP-hard NP-hard: poly-trans from NDTM NP-complete: NP-hard & NP-class SAT is NP-class => SAT is NP-complete (C) Debasis Mitra

  41. C-L-K’S THEOREM NP-class Poly-trans * SAT P-class Significance: IF there exists a poly-alg for SAT, THEN P=NP or, all NP-class problems WILL have poly-alg (C) Debasis Mitra

  42. C-L-K’S MORE RESULT: 3-SAT IS NP-HARD Polynomial Transformation from SAT to 3-SAT Significance: IF there exists a polynomial alg for 3-SAT, Then, SAT would have an indirect poly-alg, So, P=NP would be true, or, all NP-class problems WOULD have poly-algs Polynomial Transforms are Transitive A  B, and B  C, implies AC Chain of poly-transformation creates NP-hard class (C) Debasis Mitra

  43. C-L-K THEOREM LEADS TO NP-HARD CLASS NP-hard NP-class SAT 3-SAT Poly-trans 3-DM P-class (C) Debasis Mitra

  44. C-L-K THEOREM LEADS TO NP-COMPLETE CLASS NP  NP-hard = NP-complete NP-class NP-complete SAT 3-SAT Poly-trans 3-DM P-class (C) Debasis Mitra

  45. C-L-K’S MORE RESULTS: 3-SAT IS NP-HARD Chain of poly-transformation creates NP-hard class Any poly-alg found for a problem in NP-hard will solve P=NP question (for “yes”) Overlap of NP-hard and NP-class makes NP-complete NP-class problem has an exponential alg. O(an), and So, is any NP-complete problem Branching factor of decision tree is bounded by a constant a (C) Debasis Mitra

  46. C-L-K THEOREM LEADS TO NP-COMPLETE CLASS NP-hard NP-class NP-complete P-class (C) Debasis Mitra

  47. WHAT IS 3-SAT? SAT with limit on clause-size ≤ 3 Input: (U, V) U: {a, b, c, d} V: {{a, b, c}, {a, d}, {~b, ~d}} This is 3-SAT ok We will use clause size = 3, rather than ≤ 3 (C) Debasis Mitra

  48. 3-SAT IS NP-HARD Theorem: There exists a poly-trans from SAT to 3-SAT TXY: Given ANY given SAT input, you can create a size-fixed set of clauses, by introducing new variables and clauses, … such that, T/F of the <input SAT> is preserved! And you can do that in polynomial time with respect to SAT’s input size However, 2-SAT is in P-class: David-Putnam poly algorithm You cannot find a polynomial-trans from SAT to 2-SAT! (C) Debasis Mitra

  49. 3-SAT IS NP-HARD Classify SAT clauses into 4 groups: 1-clauses, 2-clauses, 3-clauses, and p-clauses with p>3 literals in these clauses We will (1) show transformations (to the corresponding 3-clauses) for each of these 4 types of clauses (2) prove that the transformations are correct, i.e., truth preserving; and then (3) we will discuss the polynomial-time nature of the aggregate-transformation at the end (C) Debasis Mitra

  50. 3-SAT IS NP-HARD For each clause do WHY? u is a literal Remember: 3 clauses are created, and 2 NEW variables added in target 3-SAT for EACH 1-clause in the source SAT (C) Debasis Mitra

More Related