Chapter 2: The Theory of NP-completeness

1. Chapter 2: The Theoryof NP-completeness • Problems vs. Languages • Encoding schemes • P, NP

2. Deterministic Turing Machine (DTM) …….. …….. • Two-way, infinite tape, broken into cells, each containing one symbol. • Two-way, read/write tape head. • Finite control - a program, containing the position of the read head, current symbol being scanned, and the current state. • An input string is placed on the tape, padded to the left and right infinitely with blanks, read/write head is positioned at the left end of input string. • In one move, depending on the current state and symbol being scanned, the TM: • changes state • prints a symbol over the cell being scanned, and • moves its’ tape head one cell left or right. • Many modifications possible. Finite Control

3. Formal Definition of a DTM • A DTM is a seven-tuple: M = (Q, Σ, Γ, δ, q0, B, F) Q A finite set of states Γ A finite tape alphabet B A distinguished blank symbol, which is in Γ Σ A finite input alphabet, which is a subset of Γ– {B} q0 The initial/starting state, q0 is in Q F A set of final/accepting states, which is a subset of Q δ A next-move function, which is a mapping (i.e., may be undefined) from Q x Γ –> Q x Γ x {L,R} • Intuitively, δ(q,s) specifies the next state, symbol to be written, and the direction of tape head movement by M after reading symbol s while in state q.

4. Example #1: {w | w is in {0,1}* and w ends with a 0} 0 00 10 10110 Not ε Q = {q0, q1, q2} Γ = {0, 1, B} Σ = {0, 1} F = {q2} δ: 0 1 B q0 (q0, 0, R) (q0, 1, R) (q1, B, L) q1 (q2, 0, R) - - q2 - - - • q0 is the “scan right” state • q1 is the verify 0 state

5. Example #1: {w | w is in {0,1}* and w ends with a 0} 0 1 B q0 (q0, 0, R) (q0, 1, R) (q1, B, L) q1 (q2, 0, R) - - q2 - - - • Sample Computation: (on 0010) q00010 |— 0q0010 |— 00q010 |— 001q00 |— 0010q0 |— 001q10 |— 0010q2

6. Example #2: {0n1n | n >= 1} 0 1 X Y B q0 (q1, X, R)- - (q3, Y, R) - q1 (q1, 0, R) (q2, Y, L) - (q1, Y, R) - q2 (q2, 0, L) - (q0, X, R) (q2, Y, L) - q3 - - - (q3, Y, R) (q4, B, R) q4 - - - - - • Sample Computation: (on 0011) q00011 |— Xq1011 |— X0q111 |— Xq20Y1 |— q2X0Y1 |— Xq00Y1 |— XXq1Y1 |— XXYq11 |— XXq2YY |— Xq2XYY |— XXq0YY |— XXYq3Y |— XXYYq3 |— XXYYBq4

7. Example #2: {0n1n | n >= 1} 0 1 X Y B q0 (q1, X, R)- - (q3, Y, R) - q1 (q1, 0, R) (q2, Y, L) - (q1, Y, R) - q2 (q2, 0, L) - (q0, X, R) (q2, Y, L) - q3 - - - (q3, Y, R) (q4, B, R) q4 - - - - - • The TM basically matches up 0’s and 1’s • q0 is the “check off a zero” state • q1 is the “scan right, find and check off a matching 1” state • q2 is the “scan left” state • q3 is the “verify no more ones” state • q4 is the final state • Other Examples: 000111 00 11 001 011 010 110

8. What kinds of things can TMs do? • Accept languages (as above) • Simulate DFAs, NFAs and PDAs • Compute functions and operators (+, *, etc) • Have subroutines, pass parameters • Simulate arbitrary computer programs • Exercises (some are from homework #6): Construct a DTM for each of the following. • {w | w is in {0,1}* and w ends in 00} • {w | w is in {0,1}* and w contains at least 2 0’s} • {w | w is in {0,1}* and w contains at least one 0 and one 1} • {w | w is in {0,1}* and w contains at least one occurrence of the substring 00} • A TM that adds 1 to a given binary number • Suppose the input to a Turing machine consists of two n-bit binary numbers separated by a # character. Give a deterministic Turing machine that will determine if the first binary number is larger than the second. Note that the turning machine should output 1 to the right of the second number if the answer is yes, and a 0 if the answer is no.

9. Be Careful! • Note that, just like a computer program, a TM can contain an infinite loop: 0 1 B q0 (q0, 0, R) (q0, 1, R) (q0, B, R) • This is somewhat equivalent to: while (true) ; • As with more general programs, more complicated TM infinite loops can occur.

10. Problem vs. Language • As computer scientists, we are typically concerned with solving problems, not recognizing languages (unless your last name is Stansifer). • However, ultimately we want to make precise claims about problems: •   P,   NP or   NP-complete • 2-SAT can be solved by a deterministic polynomial time algorithm • TSP can be solved by a non-deterministic polynomial time algorithm

11. Problem vs. Language • Such claims raise several questions: • What is an algorithm? • What do deterministic and non-deterministic mean? • What does it mean to “solve” a problem?” • What are P, NP, NP-complete? • How is time measured, and what is input length? • In order to answer such questions precisely, we need to precisely define our terms, i.e., we will us a formal model of computation, i.e., the Turing Machine.

12. The Language Level • Symbol – An atomic unit, such as a digit, character, lower-case letter, etc. Sometimes a word. • Alphabet – A finite set of symbols, usually denoted by Σ. Σ = {0, 1} Σ = {0, a, 4} Σ = {a, b, c, d} • String – A finite length sequence of symbols, presumably from some alphabet. w = 0110 y = 0aa x = aabcaa z = 111 special string: ε

13. The Language Level • String operations: w = 0110 y = 0aa x = aabcaa z = 111 concatenation: wz = 0110111 length: |w| = 4 |ε| = 0 |x| = 6 reversal: yR = aa0

14. The Language Level • Some special sets of strings: Σ* All strings of symbols from Σ (Kleene closure) Σ+ Σ* - {ε} (positive closure) • Example: Σ = {0, 1} Σ* = {ε, 0, 1, 00, 01, 10, 11, 000, 001,…} Σ+ = {0, 1, 00, 01, 10, 11, 000, 001,…}

15. The Language Level • A (formal) language is: 1) A (finite or infinite) set of strings from some alphabet. 2) Any subset L of Σ* • Some special languages: {} The empty set/language, containing no strings {ε} A language containing one string, the empty string. Σ* and Σ+

16. The Language Level • Language Examples: Σ = {0, 1} L = {x | x ∊ Σ* and x contains an even number of 0’s} Σ = {0, 1, 2,…, 9, .} L = {x | x ∊ Σ* and x forms a finite length real number} = {0, 1.5, 9.326,…} Σ = {a, b, c,…, z, A, B,…, Z} L = {x | x ∊ Σ* and x is a Java reserved word} = {while, for, if,…}

17. The Language Level • Language Examples: Σ = {Java reserved words} U { (, ), ., :, ;,…} U {Legal Java identifiers} L = {x | x ∊ Σ* and x is a syntactically correct Java program} Σ = {English words} L = {x | x ∊ Σ* and x is a syntactically correct English sentence}

18. Problem vs. Language • An analogy between languages and problems: • L partitions Σ* into two sets: L and Σ* - L • Ypartitions D into two sets: Yand D - Y • Additionally: • DTM & NDTM • Deterministic & nondeterministic algorithms • How we use these analogies: • The formal language level will be used to define various terms and concepts, i.e., algorithm, deterministic, non-deterministic, solve, problem, P, NP, NP-complete, etc. • Encoding schemes will form the “bridge” between the formal and informal levels that will allow this. • This will allow us to use these terms in the context of problems.

19. Encoding Schemes • What is an encoding scheme (informally)? • Suppose you wrote a program that inputs a graph G = (V, E) • You need to come up with an input (or file) format • That format is (basically) an encoding scheme

20. Encoding Schemes • Suppose we are dealing with a problem in which each instance is a graph: G = (V,E) V = {V1, V2, V3, V4} E = {{V1, V2}, {V2, V3}} • Encoding schemes: length • vertex list & edge list V[1],V[2],V[3],V[4],(V[1],V[2]),(V[2],V[3]) 36 • neighbor list (V[2])(V[1],V[3])(V[2])() 24 • adjacency matrix 0100/1010/0010/0000 19 • Notes: • A particular problem has many different encoding schemes • Length is now precise - the number of input symbols • A problem and an associated encoding scheme define a language

21. Encoding Schemes, cont. • Let  be a problem, e an encoding scheme that uses symbols from Σ. Then  partitions Σ* into 3 subsets: • { x | x  Σ* and x  D} • { x | x  Σ*, x  D and x  Y} • { x | x  Σ*, x  D and x  Y(x  D - Y)} • The second of the above is the language associated with  and e: L[, e] = {x  Σ* | Σ is the alphabet used by e, and x is the encoding under e of an instance I  Y} (page 20)

22. Encoding Schemes, cont. • How does this allow application of formal terms, concepts and definitions at the problem level? • See page 20, second paragraph • Terms such as “NP-complete” are defined formally in terms of languages • Saying that a problem  is NP-complete is a somewhat informal way of saying that the language L[, e] is NP-complete, for some encoding scheme e.

23. Encoding Schemes, cont. • A problem can have many encoding schemes • The previous page suggests our results for a problem are encoding dependent • But is this true? • Could a property hold for a problem under one encoding scheme, but not another? • Yes!

24. Encoding Schemes, cont. • Example #1: • Input a graph G = (V, E) and for each pair of vertices output an indication of whether or not the two vertices are adjacent. • Adjacency matrix: O(n) (n = input length) • Vertex-edge list: O(n2) (no edges) • Note that the encoding scheme does make a difference

25. Encoding Schemes, cont. • Example #2: • Input a positive integer n and output all binary numbers between 0 and n-1 • Input n in binary: O(m2m) (m = input length) • Input n in unary: O(log(m)2log(m)) = O(mlogm) • Note that the encoding scheme makes a dramatic difference in running time, i.e., polynomial vs. exponential • The second encoding scheme is unnecessarily padding the input, and is therefore said to be unreasonable

26. Encoding Schemes, cont. • Reasonable encoding schemes are: • Concise (relative to how a computer stores things) • Decodable (in polynomial time, pages 10, 21) • We do not want the fact that a problem is NP-complete or solvable in polynomial time to depend on a specific encoding scheme • Hence, we restrict ourselves to reasonable encoding schemes • Comes naturally, unreasonable encoding schemes will look suspicious

27. Encoding Schemes and Input Length. • The running time of an algorithm, or the computational complexity of a problem is typically expressed as a function of the length of a given instance. • O(n2) • Ω (n3) • In order to talk about the computational complexity of a problem or the running time of an algorithm, we need to associate a length with each instance.

28. Encoding Schemes and Input Length. • Every problem will have an associated length function: LENGTH: D Z+ which must be encoding-scheme independent, i.e., “polynomial related” to the input lengths we would obtain from a reasonable encoding scheme (see page 19 for a definition). • This ensures that the length function is reasonable and that complexity results for a problem are consistent with those for an associated encoded language.

29. Input Length Function • Recall the CLIQUE problem: CLIQUE INSTANCE: A Graph G = (V, E) and a positive integer J <= |V|. QUESTION: Does G contain a clique of size J or more? • What would be a reasonable length function? • # of vertices • # of edges • # of vertices + # of edges • Which would be polynomial related to a reasonable encoding scheme? • Note that the first and last differ quite substantially, although not exponentially.

30. Deterministic, One-Tape TM …….. …….. • Two-way, infinite tape, broken into cells, each containing one symbol. • Two-way, read/write tape head. • Finite control, i.e., a program, containing the position of the read head, current symbol being scanned, and the current state. • An input string is placed on the tape, padded to the left and right infinitely with blanks, read/write head is positioned at the left end of input string. • In one move, depending on the current state and the current symbol being scanned, the TM 1) changes state, 2) prints a symbol over the cell being scanned, and 3) moves its’ tape head one cell left or right. Finite Control

31. Deterministic, One-Tape TM • A DTM is a seven-tuple: M = (Q, Σ, Γ, δ, q0, B, F) Q A finite set of states Γ A finite tape alphabet B A distinguished blank symbol, which is in Γ Σ A finite input alphabet, which is a subset of Γ– {B} q0 The initial/starting state, q0 is in Q qn, qy Halt states, which are in Q δ A next-move function, which is a mapping (i.e., may be undefined) from (Q - {qn, qy}) x Γ –> Q x Γ x {-1,+1} • Intuitively, δ(q,s) specifies the next state, symbol to be written, and the direction of tape head movement by M after reading symbol s while in state q.

32. DTM Definitions • Let M be a DTM. • M accepts x  Σ* IFF M halts in qy when give x as input. • The language recognized by M is: LM = { x | x  Σ* and M accepts x} • M is an algorithm IFF it always halts • M solves a decision problem  under encoding scheme e if M always halts and LM = L[, e]

33. DTM Definitions, cont. • The time used by M on x  Σ* is the number of steps that occur up until a halt state is entered. • In what sense does this represent “time?” • Suppose M always halts. Then the time complexity function TM : Z+ Z+ is denoted by TM(n) • See page 26 • M is a polynomial time DTM program if there exists a polynomial p such that: TM(n) <= p(n) for all n  Z+

34. Definition of P • Definition: P = { L : There exists a polynomial-time DTM M such that L = LM } • If L[, e]  P then we say that  belongs to P under e. • What are the implications for  if L[, e]  P? • The problem can be solved in polynomial time

35. Non-DeterministicOne-Tape TM (Version #1) • A NDTM is a seven-tuple: M = (Q, Σ, Γ, δ, q0, B, F) Q A finite set of states Γ A finite tape alphabet B A distinguished blank symbol, which is in Γ Σ A finite input alphabet, which is a subset of Γ– {B} q0 The initial/starting state, q0 is in Q qn, qy Halt states, which are in Q δ A next-move function, which is a mapping (i.e., may be undefined) from (Q - {qn, qy}) x Γ –> finite subsets of Q x Γ x {-1,+1} • Intuitively, δ(q,s) specifies zero or more options, each of which includes the next state, symbol to be written, and the direction of tape head movement by M after reading symbol s while in state q.

36. NDTM Definitions • Let M be a NDTM. • M accepts x  Σ* IFF there exists some sequence of transitions that leads M to (halt in) qy. • The language recognized by M is: LM = { x | x  Σ* and M accepts x} • M solves a decision problem  under encoding scheme e if LM = L[, e]

37. NDTM Definitions • Why is there no requirement, as there is with a DTM, for a NDTM to always halt? • Under this definition, a non-deterministic algorithm might only go into infinite loops for some (non-accepted) inputs. • Why do we tolerate this? • Basically, for convenience and simplicity • Some authors require all computations to terminate • Others require all non-accepting computations to enter an infinite loop

38. DTMs vs. NDTMs • A DTM is both a language defining device and an intuitive model of computation. • A computer program can solve a problem in polynomial time IFF a DTM can. • Anything computable by a DTM is computable by a computer program. • A NDTM is a language defining device, but not so much a good intuitive model of computation, i.e., it is not clear on the surface that anything “doable” by a NDTM is “doable” by a computer program. • In fact, the first is NOT clear at all! • How a NDTM “computes” is not even clear, at least in a real-world sense. • For out purposes, it is not as important to think of a NDTM as a “practical” computing device, so much as a language defining device.

39. NDTM Definitions, cont. • The time used by M to accept x  LMis: min{ i | M accepts x in i transitions} • Then the time complexity function TM : Z+ Z+ is denoted by TM(n) • See page 31 • M is a polynomial time NDTM program if there exists a polynomial p such that: TM(n) <= p(n) for all n >= 1.

40. NDTM Definitions, cont. • Note that the time complexity function only depends on accepting computations. • In fact, if no inputs of a particular size n are accepted, the complexity function is 1, even if some computations go into an infinite loop. • Why? Aren’t we concerned about the time required for non-accepted strings? • The time complexity function could be modified to reflect all inputs • We could even require every path in the NDTM to terminate (as mentioned previously) • Neither of these changes would ultimately matter (I.e., we could convert a NDTM with the first type of polynomial complexity bound to a NDTM with the second type of polynomial complexity bound and visa versa)

41. NDTM Definitions, cont. • The books model of a NDTM allows infinite computations for convenience - its easy to describe their version of a NDTM, and showing problems are in NP. • To accommodate this, the time complexity function only reflects accepting computations • Conversely, we could have defined P based only on accepting computations. • Although equivalent, it would be less intuitive (some authors do this) • And it’s easy enough to require all DTM computations to halt.

42. Definition of NP • Definition: NP = { L : There exists a polynomial-time NDTM program M such that L = LM } • If L[, e]  NP then we say that  belongs to NP under e. • How would one show that a language is in NP? • Give a polynomial time NDTM for it (wow!) • What are the implications for  if L[, e]  NP? • How useful or important are problems in NP? • Not as clear in this case…

43. DTMs vs. NDTMs Lemma: Every polynomial-time DTM is a polynomial-time NDTM. Proof: • Observe that every DTM is a NDTM • If a DTM runs in TM(n) deterministic time, then it runs in TM(n) non-deterministic time, by definition (the reader can verify that the latter is no more than the former, i.e., TNDM(n) <= TdM(n). • Therefore, if a DTM runs in deterministic polynomial time, then it runs in non-deterministic polynomial time. Theorem: P  NP Proof: • Suppose L  P • By definition, there exists a polynomial-time DTM M such that L = LM • By the lemma, M is also a polynomial-time NDTM such that L = LM • Hence, L  NP

44. DTMs vs. NDTMs Theorem: If L  NP then there exists a polynomial p and a DTM M such that L = LM and TM(n) <= O(2p(n)). Proof: • Let M be a NDTM where L = LM • Let q(n) be a polynomial-time bound on M • Let r be the maximum number of options provided by δ, i.e: r = max{ |S| : q  Q, s  Γ, and δ(q,s) = S } Proof: If x  LM and |x| = n, then • There is a “road map” of length <= q(n) for getting to qy • There are rq(n) road maps of length <= q(n) • A (3-tape) DTM M’ generates and tries each road map • M’ operates in (approximately) O(q(n)rq(n)) time, which is O(2p(n)) for some p(n).

45. DTMs vs. NDTMs • Question: • Could this DTM simulation of a NDTM be improved to run in polynomial time? • Only if P = NP • Observations: • The preceding proof differs slightly from that given in the book, due to the different NDTM model • Non-deterministic algorithms are starting to look like they might be useful for realistic problems, i.e., those using deterministic exponential time algorithms. • Note the “exhaustive search” nature of the simulation • For many problem in NP, exhaustive search is the only non-deterministic algorithm • Additional question: • Why didn’t we just define P as problems solvable by polynomial time DTMS, and NP as problems solvable by exponential time DTMS, and then just observe that P is a subset of NP….hmmm….requires more thought…

46. Non-DeterministicOne-Tape TM (Version #2) • A NDTM is a seven-tuple: M = (Q, Σ, Γ, δ, q0, B, F) Q A finite set of states Γ A finite tape alphabet B A distinguished blank symbol, which is in Γ Σ A finite input alphabet, which is a subset of Γ– {B} q1 The initial/starting state, q0 is in Q qn, qy Halt states, which are in Q δ A next-move function, which is a mapping (i.e., may be undefined) from (Q - {qn, qy}) x Γ –> finite subsets of Q x Γ x {-1,+1} Exactly as before, however,…(this will make clear how NP problems are important)

47. write head read/write head Guessing Module Finite Control NDTM (Version #2) -3 -2 -1 0 1 2 3 …….. …….. • Just like a DTM, but with an additional non-deterministic guessing module • Initial configuration: • input string x is placed in squares 1 through |x| • write-only head is scanning square -1, read-write head is scanning square 1 • Computation on x takes place in two separate, distinct stages • Guessing stage: • at each step, the guessing module directs the write-only head to either write a symbol from Γ and move one square to the left, or to stop • guessing module decides when (or if!) to stop • Checking stage: • if the guessing module stops, then control is transferred to q0 the start state of the finite control • the finite control, which is deterministic, then computes using the same rules as a DTM • The finite control either halts and accepts, halts and rejects, or loops infinitely

48. NDTM (Version #2) Notes: • The guessing module can guess any string from Γ* • Hmmm…so what exactly does a guessing module look like? • The guessing module allows for an infinite # of computations • This explains the need for the time complexity function to only consider accepting computations. • We could put a bound on the guess to eliminate this, but its easier to describe the NDTM without it, and it doesn’t matter anyway (class NP stays the same). • The guessed string is usually used in the second stage • For a given input string x, there are an infinite number of computations of a NDTM

49. NDTM (Version #2) • M accepts x  Σ*if there exists a guess x  Γ*that brings M to qy • Note that this is basically the same as our our definition of string acceptance with Version #1 • The time to accept x  Σ* … same as before (see slide 1.21) • All definitions and theorems proved about the first verson of NDTMs still hold: • LM, TM(n), NP • P  NP (verify this one) • If L  NP then there exists a polynomial p and a DTM such that L = LM and TM(n) <= O(2p(n))

50. NDTM (Version #2) • Are the two different versions of NDTMs equivalent? • There exists a NDTM version #1 that accepts L IFF there exist a NDTM version #2 that accepts L • Prove this as an exercise • But are the models equivalent with respect to time complexity? • Do the two models define the same class NP? • Yes! • Why do Garey and Johnson use Version #2? • Convenience • At the problem level, it is easer to show a problem (or language) is in NP • How does one show a problem (or language) is in P? • How does one show a problem (or language) is in NP?