1 / 61

Chapter 2: The Theory of NP-completeness

Chapter 2: The Theory of NP-completeness. Problems vs. Languages Encoding schemes P, NP. Problem vs. Language. As computer scientists, we are typically concerned with solving problems, not recognizing languages (unless your last name is Stansifer).

allayna
Download Presentation

Chapter 2: The Theory of NP-completeness

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Chapter 2: The Theoryof NP-completeness • Problems vs. Languages • Encoding schemes • P, NP

  2. Problem vs. Language • As computer scientists, we are typically concerned with solving problems, not recognizing languages (unless your last name is Stansifer). • Ultimately we want to make precise claims about problems: •   P,   NP or   NP-complete • 2-SAT can be solved by a deterministic polynomial time algorithm • TSP can be solved by a non-deterministic polynomial time algorithm • Such claims raise several questions: • What is an algorithm? • What do deterministic and non-deterministic mean? • What does it mean to “solve” a problem?” • What are P, NP, NP-complete? • How is time measured, and what is input length? • In order to answer such questions precisely, we will us a formal model of computation, i.e., the Turing Machine. • Note that some other model could be used, but the TM is easy to use in this context.

  3. The Language Level • Symbol – An atomic unit, such as a digit, character, lower-case letter, etc. Sometimes a word. • Alphabet – A finite set of symbols, usually denoted by Σ. Σ = {0, 1} Σ = {0, a, 4} Σ = {a, b, c, d} • String – A finite length sequence of symbols, presumably from some alphabet. w = 0110 y = 0aa x = aabcaa z = 111 special string: ε concatenation: wz = 0110111 length: |w| = 4 |ε| = 0 |x| = 6 reversal: yR = aa0

  4. The Language Level • Some special sets of strings: Σ* All strings of symbols from Σ (Kleene closure) Σ+ Σ* - {ε} (positive closure) • Example: Σ = {0, 1} Σ* = {ε, 0, 1, 00, 01, 10, 11, 000, 001,…} Σ+ = {0, 1, 00, 01, 10, 11, 000, 001,…} • A (formal) language is: 1) A (finite or infinite) set of strings from some alphabet. 2) Any subset L of Σ* • Some special languages: {} The empty set/language, containing no strings {ε} A language containing one string, the empty string.

  5. The Language Level • Language Examples: Σ = {0, 1} L = {x | x Σ* and x contains an even number of 0’s} Σ = {0, 1, 2,…, 9, .} L = {x | x Σ* and x forms a finite length real number} = {0, 1.5, 9.326,…} Σ = {a, b, c,…, z, A, B,…, Z} L = {x | x Σ* and x is a Java reserved word} = {while, for, if,…} Σ = {Java reserved words} U { (, ), ., :, ;,…} U {Legal Java identifiers} L = {x | x is in Σ* and x is a syntactically correct Java program} Σ = {English words} L = {x | x is in Σ* and x is a syntactically correct English sentence}

  6. Problem vs. Language • An analogy between languages and problems: • L partitions Σ* into two sets: L and Σ* - L • Ypartitions D into two sets: Yand D - Y • Additionally: • DTM vs. NDTM • deterministic vs. nondeterministic algorithms • How we use these analogies: • The formal language level will be used to define our terms, i.e., algorithm, deterministic, non-deterministic mean, solve, problem, P, NP, NP-complete, time, input length. • These terms will then be applied in the context of problems. • Encoding schemes will form the “bridge” between the formal and informal levels that will allow this.

  7. Encoding Schemes • Purpose: • Bridges between the problem level and the more formal language level, thereby enabling the application of formal terms, definitions and concepts to be applied to problems. • What is an encoding scheme? • Suppose you wrote a program that inputs a graph G = (V, E) • You need to come up with an input (or file) format • That format is an encoding scheme => see page 10 • Notes: • A particular problem has many different encoding schemes • Length is now precise - the number of input symbols • A problem and an associated encoding scheme define a language

  8. Encoding Schemes, cont. • Let  be a problem, e an encoding scheme that uses symbols from Σ. Then  partitions Σ*into 3 subsets: • { x | x  Σ* and x  D} • { x | x  Σ*, x  D and x  Y} • { x | x  Σ*, x  D and x  Y (x  D - Y)} • The second of the above is the language associated with and e: L[, e] = {x  Σ* | Σ is the alphabet used by e, and x is the encoding under e of an instance I  Y} (page 20) • How does this allow application of formal terms, concepts and definitions at the problem level? • See page 20, second paragraph • Terms such as “NP-complete” are defined formally in terms of languages • Saying that a problem  is NP-complete is a somewhat informal way of saying that the language L[, e] is NP-complete, for some encoding scheme e.

  9. Encoding Schemes, cont. • A problem can have many encoding schemes • The previous page suggests our results for a problem are encoding dependent • But is this true? • Could a property hold for a problem under one encoding scheme, but not another? • Yes! • Example #1: • Input a graph G = (V, E) and for each pair of vertices output an indication of whether or not the two vertices are adjacent. • Adjacency matrix: O(n) (n = input length) • Vertex-edge list: O(n2) (no edges) • Note that the encoding scheme does make a difference

  10. Encoding Schemes, cont. • Example #2: • Input a positive integer n and output all binary numbers between 0 and n-1 • Input n in binary: O(m2m) (m = input length) • Input n in unary: O(log(m)2log(m)) = O(mlogm) • Note that the encoding scheme makes a dramatic difference in running time, i.e., polynomial vs. exponential • The second encoding scheme is unnecessarily padding the input, and is therefore said to be unreasonable • Reasonable encoding schemes are: • Concise (relative to how a computer stores things) • Decodable (in polynomial time, pages 10, 21) • We do not want the fact that a problem is NP-complete or solvable in polynomial time to depend on a specific encoding scheme • Hence, we restrict ourselves to reasonable encoding schemes • Comes naturally, unreasonable encoding schemes will look suspicious

  11. Encoding Schemes and Input Length. • The running time of an algorithm, or the computational complexity of a problem is typically expressed as a function of the length of a given instance. • O(n2) • Ω (n3) • In order to talk about the computational complexity of a problem or the running time of an algorithm, we need to associate a length with each instance. • Every problem will have an associated length function: LENGTH: D Z+ which must be encoding-scheme independent, i.e., “polynomial related” to the input lengths we would obtain from a reasonable encoding scheme (see page 19 for a definition). • This ensures that the length function is also reasonable and that complexity results for a problem are consistent with those for an associated encoded language.

  12. Input Length Function • Recall the CLIQUE problem: CLIQUE INSTANCE: A Graph G = (V, E) and a positive integer J <= |V|. QUESTION: Does G contain a clique of size J or more? • What would be a reasonable length function? • # of vertices • # of edges • # of vertices + # of edges • Which would be polynomial related to a reasonable encoding scheme? • Note that the first and last differ quite substantially, although not exponentially.

  13. Deterministic, One-Tape TM …….. …….. • Two-way, infinite tape, broken into cells, each containing one symbol. • Two-way, read/write tape head. • Finite control, i.e., a program, containing the position of the read head, current symbol being scanned, and the current state. • An input string is placed on the tape, padded to the left and right infinitely with blanks, read/write head is positioned at the left end of input string. • In one move, depending on the current state and the current symbol being scanned, the TM 1) changes state, 2) prints a symbol over the cell being scanned, and 3) moves its’ tape head one cell left or right. Finite Control

  14. Deterministic, One-Tape TM • A DTM is a seven-tuple: M = (Q, Σ, Γ, δ, q0, B, F) Q A finite set of states Γ A finite tape alphabet B A distinguished blank symbol, which is in Γ Σ A finite input alphabet, which is a subset of Γ– {B} q0 The initial/starting state, q0 is in Q qn, qy Halt states, which are in Q δ A next-move function, which is a mapping (i.e., may be undefined) from (Q - {qn, qy}) x Γ –> Q x Γ x {-1,+1} Intuitively, δ(q,s) specifies the next state, symbol to be written, and the direction of tape head movement by M after reading symbol s while in state q.

  15. DTM Definitions • Let M be a DTM. • M acceptsx  Σ* IFF M halts in qy when give x as input. • The language recognized by M is: LM = { x | x  Σ* and M accepts x} • M is an algorithm IFF it always halts • M solves a decision problem under encoding scheme e if M always halts and LM = L[,e]

  16. DTM Definitions, cont. • The time used by M on x  Σ*is the number of steps that occur up until a halt state is entered. • In what sense does this represent “time?” • Suppose M always halts. Then the time complexity function TM : Z+ Z+ is denoted by TM(n) • See page 26 • M is a polynomial time DTM program if there exists a polynomial p such that: TM(n) <= p(n) for all n  Z+

  17. Definition of P • Definition: P = { L : There exists a polynomial-time DTM M such that L = LM } • If L[, e]  P then we say that  belongs to P under e. • What are the implications for  if L[, e]  P? • The problem can be solved in polynomial time

  18. Non-DeterministicOne-Tape TM (Version #1) • A NDTM is a seven-tuple: M = (Q, Σ, Γ, δ, q0, B, F) Q A finite set of states Γ A finite tape alphabet B A distinguished blank symbol, which is in Γ Σ A finite input alphabet, which is a subset of Γ– {B} q0 The initial/starting state, q0 is in Q qn, qy Halt states, which are in Q δ A next-move function, which is a mapping (i.e., may be undefined) from (Q - {qn, qy}) x Γ –> finite subsets of Q x Γ x {-1,+1} Intuitively, δ(q,s) specifies zero or more options, each of which includes the next state, symbol to be written, and the direction of tape head movement by M after reading symbol s while in state q.

  19. NDTM Definitions • Let M be a NDTM. • M acceptsx  Σ* IFF there exists some sequence of transitions that leads M to (halt in) qy. • The language recognized by M is: LM = { x | x  Σ* and M accepts x} • M solves a decision problem  under encoding scheme e if LM = L[, e] • Why is there no requirement, as there was with a DTM, for the NDTM to always halt?

  20. NDTM Definitions • Why is there no requirement, as there is with a DTM, for a NDTM to always halt? • See slide 1.15 • Why is there no definition for the notion of a non-deterministic algorithm? • Although not stated explicitly, every NDTM is a non-deterministic algorithm • But under that definition, a non-deterministic algorithm for some inputs might only go into infinite loops…why do we tolerate this? • Basically, for convenience, generality and simplicity • Some authors do require all computations to terminate • Others require all non-accepting computations to enter an infinite loop

  21. DTMs vs. NDTMs • A DTM is both a language defining device and an intuitive model of computation. • A computer program can solve a problem in polynomial time IFF and DTM can. • Anything computable by a DTM is computable by a computer program. • A NDTM is a language defining device, but not so much a good intuitive model of computation, i.e., it is not clear on the surface that anything “doable” by a NDTM is “doable” by a computer program. • In fact, the first is NOT clear at all! • How a NDTM “computes” is not even clear, at least in a real-world sense. => For out purposes, it is not as important to think of a NDTM as a “practical” computing device, so much as a language defining device.

  22. NDTM Definitions, cont. • The time used by M to accept x  LMis: min{ i | M accepts x in i transitions} • Then the time complexity function TM : Z+ Z+ is denoted by TM(n) • See page 31 • M is a polynomial time NDTM program if there exists a polynomial p such that: TM(n) <= p(n) for all n >= 1.

  23. NDTM Definitions, cont. • Note that the time complexity function only depends on accepting computations. • In fact, if no inputs of a particular size n are accepted, the complexity function is 1, even if some computations go into an infinite loop. • Why? Aren’t we concerned about the time required for non-accepted strings? • The time complexity function could be modified to reflect all inputs • We could even require every path in the NDTM to terminate (some authors do this) • Neither of these changes would ultimately matter (I.e., we could convert a NDTM with the first type of polynomial complexity bound to a NDTM with the second type of polynomial complexity bound and visa versa) • The books model of a NDTM allows infinite computations for convenience - its easy to describe their version of a NDTM, and showing problems are in NP. • To accommodate this, the time complexity function only reflects accepting computations • Note that we could have defined P based only on accepting computations. • Altough equivalent, it would be less intuitive (some authors do this)

  24. Definition of NP • Definition: NP = { L : There exists a polynomial-time NDTM program M such that L = LM } • If L[, e]  NP then we say that  belongs to NP under e. • How would one show that a language is in NP? • Give a polynomial time NDTM for it (wow!) • What are the implications for  if L[, e]  NP? • How useful or important are problems in NP? • Not as clear in this case…

  25. DTMs vs. NDTMs Lemma: Every polynomial-time DTM is a polynomial-time NDTM. Proof: • Observe that every DTM is a NDTM • If DTM runs in TM(n) deterministic time, then it runs in TM(n) non-deterministic time, by definition (the reader can verify that the latter is less than the former, i.e., TNDM(n) <= TdM(n). • Therefore, if a DTM runs in deterministic polynomial time, then it runs in non-deterministic polynomial time. Theorem: P  NP Proof: • Suppose L  P • By definition, there exists a polynomial-time DTM M such that L = LM • By the lemma, M is also a polynomial-time NDTM such that L = LM • Hence, L  NP

  26. DTMs vs. NDTMs Theorem: If L  NP then there exists a polynomial p and a DTM M such that L = LM and TM(n) <= O(2p(n)). Proof: • Let M be a NDTM where L = LM • Let q(n) be a polynomial-time bound on M • Let r be the maximum number of options provided by δ, i.e: r = max{ |S| : q  Q, s  Γ, and δ(q,s) = S } Proof: If x  LM and |x| = n, then • There is a “road map” of length <= q(n) for getting to qy • There are rq(n) road maps of length <= q(n) • A (3-tape) DTM M’ generates and tries each road map • M’ operates in (approximately) O(q(n)rq(n)) time, which is O(2p(n)) for some p(n).

  27. DTMs vs. NDTMs • Question: • Could this DTM simulation of a NDTM be improved to run in polynomial time? • Only if P = NP • Observations: • The preceding proof differs slightly from that given in the book, due to the different NDTM model • Non-deterministic algorithms are starting to look like they might be useful for realistic problems, i.e., those using deterministic exponential time algorithms. • Note the “exhaustive search” nature of the simulation • For many problem in NP, exhaustive search is the only non-deterministic algorithm • Additional question: • Why didn’t we just define P as problems solvable by polynomial time DTMS, and NP as problems solvable by exponential time DTMS, and then just observe that P is a subset of NP….hmmm….requires more thought…

  28. Non-DeterministicOne-Tape TM (Version #2) • A NDTM is a seven-tuple: M = (Q, Σ, Γ, δ, q0, B, F) Q A finite set of states Γ A finite tape alphabet B A distinguished blank symbol, which is in Γ Σ A finite input alphabet, which is a subset of Γ– {B} q1 The initial/starting state, q0 is in Q qn, qy Halt states, which are in Q δ A next-move function, which is a mapping (i.e., may be undefined) from (Q - {qn, qy}) x Γ –> finite subsets of Q x Γ x {-1,+1} Exactly as before, however,…(this will make clear how NP problems are important)

  29. write head read/write head Guessing Module Finite Control NDTM (Version #2) -3 -2 -1 0 1 2 3 …….. …….. • Just like a DTM, but with an additional non-deterministic guessing module • Initial configuration: • input string x is placed in squares 1 through |x| • write-only head is scanning square -1, read-write head is scanning square 1 • Computation on x takes place in two separate, distinct stages • Guessing stage: • at each step, the guessing module directs the write-only head to either write a symbol from Γ and move one square to the left, or to stop • guessing module decides when (or if!) to stop • Checking stage: • if the guessing module stops, then control is transferred to q0 the start state of the finite control • the finite control, which is deterministic, then computes using the same rules as a DTM • The finite control either halts and accepts, halts and rejects, or loops infinitely

  30. NDTM (Version #2) Notes: • The guessing module can guess any string from Γ* • Hmmm…so what exactly does a guessing module look like? • The guessing module allows for an infinite # of computations • This explains the need for the time complexity function to only consider accepting computations. • We could put a bound on the guess to eliminate this, but its easier to describe the NDTM without it, and it doesn’t matter anyway (class NP stays the same). • The guessed string is usually used in the second stage • For a given input string x, there are an infinite number of computations of a NDTM

  31. NDTM (Version #2) • M accepts x  Σ*if there exists a guess x  Γ*that brings M to qy • Note that this is basically the same as our our definition of string acceptance with Version #1 • The time to accept x  Σ* … same as before (see slide 1.21) • All definitions and theorems proved about the first verson of NDTMs still hold: • LM, TM(n), NP • P  NP (verify this one) • If L  NP then there exists a polynomial p and a DTM such that L = LM and TM(n) <= O(2p(n))

  32. NDTM (Version #2) • Are the two different versions of NDTMs equivalent? • There exists a NDTM version #1 that accepts L IFF there exist a NDTM version #2 that accepts L • Prove this as an exercise • But are the models equivalent with respect to time complexity? • Do the two models define the same class NP? • Yes! • Why do Garey and Johnson use Version #2? • Convenience • At the problem level, it is easer to show a problem (or language) is in NP • How does one show a problem (or language) is in P? • How does one show a problem (or language) is in NP?

  33. Proving a Problem NP-Complete(overview) • To show a problem  is NP-complete we must show: •   NP • ’  , for all ’  NP • To show   NP we need to give a NDTM, or rather, a non-deterministic algorithm for , that operates in polynomial time. • For most decision problems, Garey and Johnson’s NDTM model makes this easy.

  34. Proving a Problem is in NP Another way to look at model #2: •   NP IFF there exits a deterministic polynomial time algorithm A such that: • If I  Y then there exists a “structure” S such that (I,S) is accepted by A • If I  Y then for any “structure” S, it will always be the case that (I,S) is rejected by A

  35. Example #1: TSP • Recall TSP: TRAVELING SALESMAN INSTANCE: Set C of m cities, distance d(ci, cj)  Z+ for each pair of cities ci, cj C positive integer B. QUESTION: Is there a tour of C having length B or less, I.e., a permutation <c(1) , c(2),…, c(m)> of C such that: • A Non-Deterministic Polynomial Time Algorithm for TSP: • Guess (non-deterministically, in polynomial-time) a permutation of the cities. • Verify (deterministically, in polynomial-time) that the permutation is a valid tour of the cities having total length less than B (output “yes,” output “no” otherwise). • From this it follows that TSP  NP

  36. Example #2: Clique • Recall Clique: CLIQUE INSTANCE: A Graph G = (V, E) and a positive integer J <= |V|. QUESTION: Does G contain a clique of size J or more? • A Non-Deterministic Polynomial Time Algorithm for Clique: • Guess (non-deterministically, in polynomial-time) a set of vertices from G. • Verify (deterministically, in polynomial-time) that the set is a clique of size J or more. • From this it follows that Clique  NP

  37. Example #3: Graph K-Colorability • Recall Graph K-Colorability: GRAPH K-COLORABILITY INSTANCE: A Graph G = (V, E) and a positive integer K <= |V|. QUESTION: Is the graph GK-colorable? • A Non-Deterministic Algorithm Polynomial Time for Graph K-colorability: • Guess (non-deterministically, in polynomial-time) a coloring of G. • Verify (deterministically, in polynomial-time) that the coloring is a valid K-coloring of G. • From this it follows that Graph K-colorability  NP

  38. Example #4: SAT • Recall Satisfiability: SATISFIABILITY INSTANCE: Set U of variables and a collection C of clauses over U. QUESTION: Is there a satisfying truth assignment for C? • A Non-Deterministic Polynomial Time Algorithm for Satisfiability: • Guess (non-deterministically, in polynomial-time) a truth assignment for the variables in U. • Verify (deterministically, in polynomial-time) that the truth assignment satisfies C. • From this it follows that Satisfiability  NP

  39. Example #5: Non-Tautology NON-TAUTOLOGY INSTANCE: Boolean expression E (, , , ). QUESTION: Is E not a tautology? • A Non-Deterministic Polynomial Time Algorithm for Non-Tautology: • Guess (non-deterministically, in polynomial-time) a truth assignment for E. • Verify (deterministically, in polynomial-time) that the truth assignment does not satisfy E. • From this it follows that Non-Tautology  NP

  40. Example #6: Tautology TAUTOLOGY INSTANCE: Boolean expression E (, , , ). QUESTION: Is E a tautology? • A Non-Deterministic Polynomial Time Algorithm for Tautology: • Guess ? • Verify ? • Is Tautology  NP? • We do not know!

  41. Tentative View of the World of NP • If P = NP then everything in NP can be solved in deterministic polynomial time. • If P≠ NP then: • P  NP • Everything in P can be solved in deterministic polynomial time • Everything in NP - P is intractable • Hence, given a problem  we would like to know if   P or   NP - P • Unfortunately, we do not know if P = NP or P≠ NP • It does appear that P≠ NP since there are many problem in NP for which nobody has been able to find deterministic polynomial time algorithms

  42. Tentative View of the World of NP • For some problems  we can show that   P • Unfortunately, we cannot not show that   NP - P, at least not without resolving the P =? NP question, which appears to be very difficult • Hence, we will restrict ourselves to showing that for some problems, if P≠ NP then   NP - P • Although the theory of NP-completeness assumes that P≠ NP, if it turned out that P = NP, then parts of the theory (i.e., transformations and reductions) can still be used to prove results concerning the relative complexity of problems.

  43. Polynomial Transformation • A polynomial transformation from a language L1  Σ1*and to a language L2  Σ2* is a function f: Σ1*  Σ2* that statisfies the following conditions: • There is a polynomial time DTM program that computes f • For all x Σ1*, x L1*iff and only if f(x) L2* • If there is a polynomial transformation from L1 to L2 then we write L1  L2, read “L1 transforms to L2“

  44. Polynomial Transformation • The significance of polynomial transformations follows from Lemma 2.1 Lemma 2.1: If L1  L2 and L2  P then L1  P Proof: Let M be a polynomial time DTM that decides L2 , and let M’ be a polynomial time DTM that computes f. Then M’ and M can be composed to give a polynomial time DTM that decides . • Equivalently, if L1  L2 and L1  P then L2  P

  45. Polynomial Transformation • Informally, if L1  L2 then it is frequently said that: • L1 Is no harder than L2 • L2Is no easier than L1 • L2Is at least as hard as L1 • L1 Is as easy as L2 • Note that “hard” and “easy” here are only with respect to polynomial v.s. exponential time • We could have L1 decidable in 2ntime, L2 decidable in time, but where L1 is not.

  46. Polynomial Transformation • At the problem level we can regard a polynomial transformation from the decision problem 1  Σ1*and to a decision problem 2  Σ2* is a function f : D1 D2that statisfies the following conditions: • f is computable by a polynomial time algorithm • For all I  D1, I  Y1 if and only if f ( I)  Y2 • We will deal with transformations only at the problem level

  47. Polynomial Transformation • Another helpful factoid concerning transformations: Lemma 2.2: If L1  L2 and L2  L3 then L1  L3 • In other words, polynomial transformations are transitive.

  48. Definition of NP-Complete • A decision problem  is NP-complete if: •   NP, and • For all other decision problems ’  NP, ’   • For many problems, proving the first requirement is easy. • On the surface, the second requirement would appear very difficult to prove, especially given that there are an infinite number of problems in NP. • As will be shown, the second requirement will be established using the following lemma: Lemma 2.3: If L1and L2 belong to NP, L1 is NP-complete, andL1  L2 then L2 is NP-complete.

  49. Definition of NP-Complete, cont. • Let  be an NP-complete problem • Observation: •   P If and only if P = NP (only if) • If an NP-complete problem  can be solved in polynomial time, then every problem in NP can be solved in polynomial time (should be clear, by definition). • If   P then P = NP • If any problem in NP is intractable, then so are all NP-complete problems. • If P≠ NP then   P (i.e.,   NP-P), for all NP-complete problems 

  50. Definition of NP-Complete, cont. • Let  be an NP-complete problem • Observation: •   P If and only if P = NP (if) • If every problem in NP can be solved in polynomial time, then clearly every NP-complete problem  can be also, since every NP-complete problem, by definition, is in NP. • If P = NP then   P, for all NP-complete problems  • If any NP-complete problem  in NP is intractable, then clearly P ≠ NP. • If   P (i.e.,   NP-P) then P ≠ NP, for any NP-complete problem 

More Related