UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall, 2008

1. UMass Lowell Computer Science 91.503Analysis of AlgorithmsProf. Karen DanielsFall, 2008 Lecture 6 Friday, 10/10/08 NP-Completeness

2. Overview • Graham Videotape • Review: Reduction to Establish Lower Bounds • Basics • Practicing Some Reductions • How to Treat NP- Hard or Complete Problems

3. Resources • Recommended: • Computers & Intractability • by Garey & Johnson • W.H.Freeman • 1990 • ISBN 0716710455. • For further study… • Computational Complexity: A Conceptual Perspective • by Oded Goldreich • 2008 • ISBN 978-0-521-88473-0

4. Motivation

5. Why do we care?? • When faced with a problem in practice, need to know whether or not one can expect produce a “fast” algorithm (i.e. O(nk)) to solve it • “Intractable” problems occur frequently in practice • Surface differences between tractable and intractable problems are often hard to see • e.g. MST solvable in polynomial time but Dominating Set intractable

6. Dominating Set source: Garey & Johnson

7. Distance-Based Subdivision Maximum Rectangle Geometric Restriction Ordered Containment Limited Gaps Containment NP-complete Minimal Enclosure Core Algorithms Maximal Cover Application-Based Algorithms Lattice Packing Two-Phase Layout Column-Based Layout Industrial MotivationSupporting Apparel Manufacturing

8. P2 P2 P1 Q3 P1 Q2 Q2 Q1 Translational 2D Polygon Covering Translated Q Covers P Sample P and Q Q1 Q3 Covering Research Example: Translational 2D Covering [CCCG’01, CCCG’03 conference papers] STATIC • Input: • Covering polygons Q = {Q1, Q2 , ... , Qm} • Target polygons (or point-sets) P = {P1, P2 , ... , Pn} • Output: • Translations g = {g1, g2 , ... , gm} such that NP-COMPLETE

9. . . . covering decomposition: . . . . . . . . . . cover Covering Problem Taxonomy covering non-geometric covering geometric covering VERTEX-COVER, SET-COVER, EDGE-COVER, VLSI logic minimization, facility location 2D translational covering P: finite point sets P: shapes Q: identical • Thin coverings of the plane with congruent convex shapes • Translational covering of a convex set by a sequence of convex shapes • Translational covering of nonconvex set with nonidentical covering shapes Q: nonconvex Q: convex 1D interval covered by annuli BOX-COVER partition: • NP-hard/complete polygon problems • polynomial-time results for restricted orthogonal polygon covering and horizontally convex polygons • approximation algorithms for boundary, corner covers of orthogonal polygons Polynomial-time algorithms for triangulation and some tilings

10. Reduction to Establish Lower Bound Tightening a Lower Bound for Maximum-Area Axis-Parallel Rectangle Problem from W(n) to W(nlgn)

11. Approach • Go around lower bound brick wall by: • examining strong lower bounds for some similar problems • transforming a similar problem to the rectangle problem [this process is similar to what we do when we prove problems NP-complete] worst-case bounds on problem n5 2n 1 n2 n log2 n n

12. x2 x4 x1 x3 Lower Bound of W(n log n) by Transforming a (seemingly unrelated) Problem MAX-GAP instance: given n real numbers { x1, x2, ... xn } find the maximum difference between 2 consecutive numbers in the sorted list. O(n) time transformation Rectangle area is a solution to the MAX-GAP instance specialized polygon Rectangle algorithm must take as least as much time as MAX-GAP. MAX-GAP is known to be in W(n log n). Rectangle algorithm must take W(n log n) time for specialized polygons. [Transforming yet another different problem yields bound for unspecialized polygons.]

13. NP-Completeness Chapter 34

14. Basic Concepts Polynomial Time & Problem Formalization Polynomial-Time Verification NP-Completeness & Reducibility NP-Completeness Proofs Expanding List of Hard Problems via Reduction

15. Polynomial Time & Problem Formalization • Formalize notions of abstract and concrete decision problems • Encoding a problem • Define class of polynomial-time solvable decision problems • Define complexity class • Express relationship between decision problems and algorithms that solve them using framework of formal language theory

16. Polynomial Time & Problem Formalization (continued) • Solvable in Polynomial Time • i.e. O(nk) for some positive constant k • Tractable: k is typically small (e.g. 1, 2, 3) • Typically holds across sequential computation models • Closure properties (chaining/composition) • Abstract Problem: • Binary relation on set of problem instances & solutions • Decision Problem: yes/no answer • impose bound on optimization problem to recast • e.g. Dominating Set

17. Dominating Set source: Garey & Johnson

18. Polynomial Time & Problem Formalization (continued) • Encoding of set S of abstract objects is mapping e from S to set of binary strings • Concrete Problem: instance set is set of binary strings • Concrete Problem is Polynomial-Time Solvable if there exists an algorithm to solve it in time O(nk) for some positive constant k • Complexity Class P = set of polynomial-time solvable concrete decision problems • Given abstract decision problem Q that maps instance set I to {0,1} can induce a concrete decision problem e(Q) • If solution to abstract-problem instance • then solution to concrete-problem instance is also Q(i)

19. Lemma 34.1: Let Q be an abstract decision problem on an instance set I and let e1, e2 be polynomially related encodings on I. Then, Polynomial Time & Problem Formalization (continued) Goal: Extend definition of polynomial-time solvability from concrete to abstract using encodings as bridge • is polynomial-time computable if there exists a polynomial-time algorithm A that, given any input , produces f(x) as output. • For set I of problem instances, encodings are polynomially related if there exist 2 polynomial-time computable functions f12, f21 such that • If 2 encodings e1, e2 of abstract problem are polynomially related, then polynomial-time solvability of problem is encoding-independent:

20. Polynomial Time & Problem Formalization (continued) • Formal Language Framework • Alphabet S is finite set of symbols: • Language L over S is any set of strings consisting of symbols from S • Set of instances for decision problem Q is • View Q as language L over S = {0,1} • Express relation between decision problems and algorithms • Algorithm Aaccepts string if, given input x, output A(x) is 1 • Language accepted byA is set of strings A accepts

21. Polynomial Time & Problem Formalization (continued) • Formal Language Framework (continued) • Language L is decided by algorithm A if every binary string in L is accepted by A and every binary string not in L is rejected by A • Language L is accepted in polynomial time by algorithm A if it is accepted by A and if exists constant k such that, for any length-n string , A accepts x in time O(nk) • Language L is decided in polynomial time by algorithm A if exists constant k such that, for any length-n string , A correctly decides in time O(nk) if

22. Polynomial Time & Problem Formalization (continued) • Formal Language Framework (continued) • Complexity Class is a set of languages whose membership is determined by a complexity measure (i.e. running time) of algorithm that determines whether a given string belongs to a language. • We already defined: Complexity Class P = set of polynomial-time solvable concrete decision problems • Language Framework gives other definitions: P = { : there exists an algorithm A that decides L in polynomial time} = { L : L is accepted by a polynomial-time algorithm}

23. Polynomial-Time Verification • Examine algorithms that “verify” membership in languages • Define class of decision problems whose solutions can be verified in polynomial time

24. Polynomial-Time Verification (continued) • Verification Algorithms • algorithm A verifies language L if for any string certificate y that A can use to prove that • also, for any string certificate y that A can use to prove that • language verified by verification algorithm A is

25. Polynomial-Time Verification (continued) • The Complexity Class NP • languages that can be verified by a polynomial-time algorithm A • Averifies language Lin polynomial time • The Complexity Class co-NP • set of languages L such that

26. Polynomial-Time Verification (continued) P P P 34.3 source: 91.503 textbook Cormen et al.

27. NP-Completeness & Reducibility • Polynomial-Time Reducibility • Define complexity class of NP-complete problems • Relationships among complexity classes • Circuit satisfiability as an NP-complete problem

28. NP-Completeness & Reducibility (continued) • Q can be reduced to Q’ if any instance of Q can be “easily rephrased as an instance of Q’, the solution to which provides solution to instance of Q • L1is polynomial-time reducible to L2 : if exists polynomial-time computable function 34.4 source: 91.503 textbook Cormen et al.

29. NP-Completeness & Reducibility (continued) 34.5 34.3 source: 91.503 textbook Cormen et al.

30. without this condition, language is only NP-hard popular theoretical computer science view set of all NP-complete languages NP Language is NP-complete if and NP-Completeness & Reducibility (continued) Theorem: - If any NP-complete problem is polynomial-time solvable, then P=NP. - Equivalently, if any problem in NP is not polynomial-time solvable, then no NP-complete problem is polynomial-time solvable. source: 91.503 textbook Cormen et al.

31. Expanding List of Hard Problems via Reduction • Relationships among some NP-complete problems First NP-complete problem: Boolean Formula Satisfiability (Cook ’71) Circuit-SAT source: Garey & Johnson source: 91.503 textbook Cormen et al. Need to show that starting problem is in NP

32. single circuit output Boolean Combinational Gate Types: AND, NOT, OR source: 91.503 textbook Cormen et al. NP-Completeness & Reducibility (continued): Circuit Satisfiability circuit inputs • Satisfying Assignment: truth assignment inducing output = 1 • GOAL: Show Circuit Satisfiability is NP-complete Language Def: CIRCUIT-SAT = {<C>:C is a satisfiable boolean combinational circuit.} 34.8

33. NP-Completeness & Reducibility (continued) Circuit Satisfiability • CIRCUIT-SAT is in NP: • Construct 2-input polynomial-time algorithm A that verifies CIRCUIT-SAT • Input 1: encoding of circuit C • Input 2: certificate = assignment of boolean values to all wires of C • Algorithm A : • For each logic gate A checks if certificate value for gate’s output wire correctly computes gate’s function based on input wire values • If output of entire circuit is 1, algorithm outputs 1 • Otherwise, algorithm outputs 0 • For satisfiable circuit, there is a (polynomial-length) certificate that causes A to output 1 • For unsatisfiable circuit, no certificate can cause A to output 1 • Algorithm A runs in polynomial time source: 91.503 textbook Cormen et al.

34. NP-Completeness & Reducibility (continued) Circuit Satisfiability • CIRCUIT-SAT is NP-hard: • L represents some language in NP • Create polynomial-time algorithm F computing reduction function f that maps every binary string x to a circuit C=f(x) such that • Must exist algorithm A verifying L in polynomial time. (A from NP) • F uses A to compute f. • Represent computation of A as sequence of configurations • F constructs single circuit that computes all configurations produced by initial configuration Circuit is tailor-made for instance string x. source: 91.503 textbook Cormen et al.

35. NP-Completeness Proofs (continued) • Proving a Language NP-Complete • Proving a Language NP-Hard • all steps except (1) source: 91.503 textbook Cormen et al.

36. NP-Completeness Proofs (continued) • Reducing Boolean Circuit Satisfiability to Boolean Formula Satisfiability • Boolean Formula Satisfiability: Instance of language SAT is a boolean formula f consisting of: • n boolean variables: x1, x2, ... , xn • m boolean connectives: boolean function with 1 or 2 inputs and 1 output • e.g. AND, OR, NOT, implication, iff • parentheses • truth, satisfying assignments notions apply source: 91.503 textbook Cormen et al.

37. NP-Completeness Proofs (continued) • Show: SAT is NP-Complete • SAT is in NP via argument similar to circuit case • Reduce Boolean Circuit Satisfiability to Boolean Formula Satisfiability • Careful! Naive approach might have shared subformulas and cause formula size to grow exponentially! source: 91.503 textbook Cormen et al.

38. NP-Completeness Proofs (continued) • Show: SAT is NP-Complete (continued) • Reduce Boolean Circuit Satisfiability to Boolean Formula Satisfiability (continued) • Approach: For each wire xi in circuit C, formula f has variable xi. Express gate operation. C 34.10 source: 91.503 textbook Cormen et al.

39. NP-Completeness Proofs (continued) • Reducing Formula Satisfiability to 3-CNF-Satisfiability • Boolean Formula Satisfiability: Instance of language SAT is a boolean formula f • CNF = conjunctive normal form • conjunction: AND of clauses • clause: OR of literal(s) • 3-CNF: each clause has exactly 3 distinct literals • Show: 3-CNF is NP-Complete source: 91.503 textbook Cormen et al.

40. NP-Completeness Proofs (continued) • Show: 3-CNF is NP-Complete (continued) • 3CNF is in NP via argument similar to circuit case • Reduce Formula Satisfiability to 3-CNF-Satisfiability • 3 steps progressively transforming formula f closer to 3CNF • 1) Similar to reduction Binary “ParseTree” 34.11 source: 91.503 textbook Cormen et al.

41. NP-Completeness Proofs (continued) • Show: 3-CNF is NP-Complete (continued) • 3CNF is in NP via argument similar to circuit case • Reduce Formula Satisfiability to 3-CNF-Satisfiability • 3 steps progressively transforming formula f closer to 3CNF • 2) Convert each clause fi’ into conjunctive normal form • construct truth table for fi’ • using entries for 0, build DNF formula for • convert DNF into CNF formula fi” by applying DeMorgan’s laws • each clause has at most 3 literals * truth table for f1 * * * source: 91.503 textbook Cormen et al. * use to build DNF

42. NP-Completeness Proofs (continued) • Show: 3-CNF is NP-Complete (continued) • 3CNF is in NP via argument similar to circuit case • Reduce Formula Satisfiability to 3-CNF-Satisfiability • 3 steps progressively transforming formula f closer to 3CNF • 3) Convert each clause fi” so it has exactly 3 distinct literals • add “padding variables” • create fi”’ • Size of resulting formula is polynomial in length of original formula and reduction takes only polynomial time and 3-CNF formula source: 91.503 textbook Cormen et al.

43. Expanding List of Hard Problems via Reduction • Relationships among some NP-complete problems First NP-complete problem: Boolean Formula Satisfiability (Cook ’71) Circuit-SAT source: Garey & Johnson source: 91.503 textbook Cormen et al.

44. A B C D F E Clique Cliqueof an undirected graph G=(V,E) is a complete subgraph of G. cliqueof size 3 source: Garey & Johnson

45. x1 For each clause place triple of vertices v1r , v2r , v3r into V. Put edge for vir , vjs if corresponding literals consistent and r not= s. Reducing 3-CNF-SAT to Clique source: 91.503 textbook Cormen et al. Construct graph G such that f is satisfiable iff G has clique of size k x1 x3 x1 x2 x2 34.14

46. C D E source: Garey & Johnson Vertex Cover A B vertex coverof size 2 Vertex Coverof an undirected graph G=(V,E) is a subset F

47. u v z w y x Reducing Clique to VertexCover Construct complement G =(V, E ) of graph G such that G has clique of size k iff graph G has a vertex cover of size |V|-k. 34.15 source: 91.503 textbook Cormen et al.

48. Dominating Set source: Garey & Johnson

49. Hamiltonian Cycle 34.2 Hamiltonian Cycleof an undirected graph G=(V,E) is a simple cycle that contains each vertex in V. NP-Complete: see textbook’s reduction from VertexCover source: 91.503 textbook Cormen et al.

50. How to Treat NP-Hard or Complete Problems