CS 3343: Analysis of Algorithms

CS 3343: Analysis of Algorithms P and NP

Have seen so far • Algorithms for various problems • Sorting, order statistics, LCS, scheduling, MST, shortest path, etc • Running times O(n2) ,O(n log n), O(n),O(nm), etc. • i.e., polynomial in the input size • Those are called tractable problems • Can we solve all (or most of) interesting problems in polynomial time?

2 4 9 5 10 11 8 3 6 5 9 7 Example difficult problem • Traveling Salesperson Problem (TSP) • Input: undirected graph with lengths on edges • Output: shortest tour that visits each vertex exactly once • An optimization problem • Best known algorithm: O(n 2n) time. • Intractable

Comparison of functions For a super computer that does 1 trillion operations per second, it will be longer than 1 billion years

Another difficult problem • Maximum Clique: • Input: undirected graph G=(V,E) • Output: largest subset C of V such that every pair of vertices in C has an edge between them • An optimization problem • Best known algorithm: O(n 2n) time

What can we do ? • Spend more time designing algorithms for those problems • People tried for a few decades, no luck • Prove there is no polynomial time algorithm for those problems • Would be great • Seems really difficult

What else can we do ? • Show that those hard problems are essentially equivalent. i.e., if we can solve one of them in polynomial time, then all others can be solved in polynomial time as well. • Works for at least 10 000 hard problems

The benefits of equivalence • Combines research efforts • If one problem has polynomial time solution, then all of them do • More realistically:Once an exponential lower bound is shown for one problem, it holds for all of them P1 P2 P3

Summing up • If we show that a problem ∏ is equivalent to ten thousand other well studied problems without efficient algorithms, then we get a very strong evidence that ∏ is hard. • We need to: • Identify the class of problems of interest • Define the notion of equivalence • Prove the equivalence(s)

Classes of problems: P and NP • Implicitly, we mean decision problems: answer YES or NO • Other problems can be converted into decision problems • k-clique: is there a clique of size = k? • P is set of problems that can be solved in polynomial time • NP (nondeterministic polynomial time) is the set of problems that can be solved in polynomial time by a nondeterministic computer

Nondeterminism • Think of a non-deterministic computer as a computer that magically “guesses” a solution, then has to verify that it is correct • If a solution exists, computer always guesses it • One way to imagine it: a parallel computer that can freely spawn an infinite number of processes • Have one processor work on each possible solution • All processors attempt to verify that their solution works • If a processor finds it has a working solution • So: NP = problems verifiable in polynomial time

P and NP • P = problems that can be solved in polynomial time • NP = problems for which a solution can be verified in polynomial time • K-clique problem is in NP: • Easy to verify solution in polynomial time • Is sorting in NP? • Not a decision problem • Is sortedness in NP? • Yes. Easy to verify

P and NP • Is P = NP? • The biggest open problem in CS • Most suspect not • the Clay Mathematics Institute has offered a $1 million prize for the first proof NP P = NP? or P

NP-Complete Problems • We will see that NP-Complete problems are the “hardest” problems in NP: • If any one NP-Complete problem can be solved in polynomial time… • …then every NP-Complete problem can be solved in polynomial time… • …and in fact every problem in NP can be solved in polynomial time (which would show P = NP) • Thus: solve TSP in O(n100) time, you’ve proved that P = NP. Retire rich & famous.

Reduction • The crux of NP-Completeness is reducibility • Informally, a problem ’ can be reduced to another problem  if any instance of ’ can be “easily rephrased” as an instance of , the solution to which provides a solution to the instance of ’ • What do you suppose “easily” means? • This rephrasing is called transformation • Intuitively: If ’ reduces to , ’ is “no harder to solve” than 

x’ Reductions: ∏’ to ∏ YES x A for ∏ NO YES A’ for ∏’ NO

x’ Reductions: ∏’ to ∏ YES f(x’)= x f A for ∏ NO YES A’ for ∏’ NO

Reduction: an example • ’: Given a set of Booleans, is at least one TRUE? • : Given a set of integers, is their sum positive? • Transformation: (x1, x2, …, xn) = (y1, y2, …, yn) where yi = 1 if xi = TRUE, yi = 0 if xi = FALSE

Reductions • ∏’ is polynomial time reducible to ∏ ( ∏’≤p ∏) iff there is a polynomial time function f that maps inputs x’ for ∏’ into inputs x for ∏, such that for any x’ ∏’(x’)=∏(f(x’)) • Fact 1: if ∏P and ∏’ ≤p ∏ then ∏’P • Fact 2: if ∏NP and ∏’ ≤p ∏ then ∏’NP • Fact 3 (transitivity): if ∏’’ ≤p ∏’ and ∏’ ≤p ∏ then ∏” ≤p ∏

Recap • We defined a large class of interesting problems, namely NP • We have a way of saying that one problem is not harder than another (∏’ ≤p ∏) • Our goal: show equivalence between hard problems

  Showing equivalence between difficult problems • Options: • Show reductions between all pairs of problems • Reduce the number of reductions using transitivity of “≤” TSP Clique P3 P4 P5 ∏’

Showing equivalence between difficult problems • Options: • Show reductions between all pairs of problems • Reduce the number of reductions using transitivity of “≤” • Show that all problems in NP are reducible to a fixed ∏. To show that some problem ∏’NP is equivalent to all difficult problems, we only show ∏ ≤p ∏’. TSP  Clique ∏ P3 P4   P5 ∏’

The first problem ∏ • Boolean satisfiability problem (SAT): • Given: a formula φ with m clauses over n variables, e.g., x1v x2 vx5, x3 v¬x5 • Check if there exists TRUE/FALSE assignments to the variables that makes the formula satisfiable • Any SAT can be rewritten into conjunctive normal form (CNF) • a conjunction of clauses, where a clause is a disjunction of literals • A literal is a Boolean variable or its negation • E.g. (AB)  (BCD) is in CNF (AB) (ACD) is not in CNF

SAT is NP-complete • Fact:SAT NP • Theorem [Cook’71]: For any ∏’NP ,we have ∏’ ≤ SAT. • Definition: A problem ∏ such that for any ∏’NP we have ∏’ ≤p ∏, is called NP-hard • Not necessarily decision problem • Definition: An NP-hard problem that belongs to NP is called NP-complete • Corollary: SAT is NP-complete.

SAT: the boolean satisfiability problem for formulas in conjunctive normal form • 0-1 INTEGER PROGRAMMING • CLIQUE (see also independent set problem) • SET PACKING • VERTEX COVER • SET COVERING • FEEDBACK NODE SET • FEEDBACK ARC SET • DIRECTED HAMILTONIAN CIRCUIT • UNDIRECTED HAMILTONIAN CIRCUIT • 3-SAT • CHROMATIC NUMBER (also called the Graph Coloring Problem) • CLIQUE COVER • EXACT COVER • HITTING SET • STEINER TREE • 3-dimensional MATCHING • KNAPSACK (Karp's definition of Knapsack is closer to Subset sum) • JOB SEQUENCING • PARTITION • MAX-CUT Karp's 21 NP-complete problems, 1972 Since then thousands problems have been proved to be NP-complete

Clique again • Clique (decision variant): • Input: undirected graph G=(V,E), K • Output: is there a subset C of V, |C|≥K, such that every pair of vertices in C has an edge between them

SAT ≤p Clique x’ • Given a SAT formula φ=C1,…,Cm over x1,…,xn, we need to produce G=(V,E) and K, such that φsatisfiable iff G has a clique of size ≥ K. f(x’)=x

SAT ≤p Clique reduction • For each literal t occurring in φ, create a vertex vt • Create an edge vt– vt’iff: • t and t’ are not in the same clause, and • t is not the negation of t’

t and t’ are not in the same clause, and • t is not the negation of t’ Edge vt – vt’ SAT ≤p Clique example • Formula: (x1 V x2 Vx3) ( ¬x2 v¬x3)  (¬x1 vx2) • Graph: ¬x2 x1 ¬ x3 x2 x3 x2 ¬ x1 • Claim:φsatisfiable iff G has a clique of size ≥ m

t and t’ are not in the same clause, and • t is not the negation of t’ Edge vt – vt’ Proof • “→” part: • Take any assignment that satisfies φ. E.g., x1=F, x2=T, x3=F • Let the set C contain one satisfied literal per clause • C is a clique ¬x2 x1 ¬ x3 x2 x3 x2 ¬ x1

t and t’ are not in the same clause, and • t is not the negation of t’ Edge vt – vt’ Proof • “←” part: • Take any clique C of size ≥ m (i.e., = m) • Create a set of equations that satisfies selected literals. E.g., x3=T, x2=F, x1=F • The set of equations is consistent and the solution satisfies φ ¬x2 x1 ¬ x3 x2 x3 x2 ¬ x1

Altogether • We constructed a reduction that maps: • YES inputs to SAT to YES inputs to Clique • NO inputs to SAT to NO inputs to Clique • The reduction works in polynomial time • Therefore, SAT ≤p Clique →Clique NP-hard • Clique is in NP → Clique is NP-complete

Independent set (IS) • Input: undirected graph G=(V,E) • Output: is there a subset S of V, |S|≥K such that no pair of vertices in S has an edge between them • Want to show: IS is NP-complete

Clique ≤ IS x’ • Given an input G=(V,E), K to Clique, need to construct an input G’=(V’,E’), K’ to IS,such that G has clique of size ≥K iff G’ has IS of size ≥K’. • Construction: K’=K,V’=V,E’=E • Reason: C is a clique in G iff it is an IS in G’s complement. f(x’)=x

Classes of problems

CS 3343: Analysis of Algorithms