Computability & Complexity II

1. Computability & Complexity II Chris Umans Caltech

2. Complexity Theory Classify problems according to the computational resources required • running time • storage space • parallelism • randomness • rounds of interaction, communication, others… Attempt to answer: what is computationally feasible with limited resources? CBSSS

3. The central questions • Is finding a solution as easy as recognizing one? P = NP? • Is every sequential algorithm parallelizable? P = NC? • Can every efficient algorithm be converted into one that uses a tinyamount of memory? P = L? • Are there small Boolean circuits for all problems that require exponential running time? EXP  P/poly? • Can every randomized algorithm be converted into a deterministic algorithm one? P = BPP? CBSSS

4. Central Questions We think we know the answers to all of these questions … … but no one has been able to prove that even a small part of this “world-view” is correct. If we’re wrong on any one of these then computer science will change dramatically CBSSS

5. Outline • We’ll look at three of these problems: • P vs. NP “the power of nondeterminism” • L vs. P “the power of sequential computation” • P vs. BPP “the power of randomness” CBSSS

6. P vs. NP CBSSS

7. Completeness • In an ideal world, given language L 1. state an algorithm deciding 2.prove that no algorithm does better • we are pretty good at part 1 • we are currently completely helpless when it comes to part 2, for most problems that we care about CBSSS

8. Completeness • in place of part 2 we can • relate the difficulty of problems to each other via reductions • prove that a problem is a “hardest” problem in a complexity class via completeness • powerful, successful surrogate for lower bounds CBSSS

9. Recall: reductions • “many-one” reduction: • now, require f computable in polynomial time A B f yes yes reduction from language A to language B f no no CBSSS

10. Completeness • complexity class C (set of languages) • language L is C-complete if • L is in C • every language in C reduces to L • formalizes “L is hardest problem in complexity class C”: • L in P implies every language in C is in P • related concept: language L is C-hardif • every language in C reduces to L CBSSS

11. Some problems we care about • 3SAT = { : 3-CNF  is satisfiable} • TSP = {<G, k>: there is a tour of all nodes of G with weight ≤ k} • SUBSET-SUM = {<x1, x2, … xn, B> : some subset of {x1, x2, … xn} sums to B} • IS= { <G, k> : there is a subset of at least k vertices of G with no edges between them} Only know exponential time algorithms for these problems CBSSS

12. From last week… • We have defined the complexity classes P (polynomial time), EXP (exponential time) some language decidable all languages RE P HALT EXP CBSSS

13. Why not EXP • 3-SAT, TSP, SUBSET-SUM, IS, … • Why not show these are EXP-complete? • all possess positive feature that some problems in EXP probably don’t have: a solution can be recognized in polynomial time CBSSS

14. NP “witness” or “certificate” Definition: NP = all languages decidable by a Nondeterministic TM in Polynomial time Theorem: language L is in NP if and only if it is expressible as: L = { x |  y, |y| ≤ |x|k, (x, y)  R } where R is a language in P. • poly-time TM MR deciding R is a “verifier” efficiently verifiable CBSSS

15. Poly-time verifiers • Example: 3SAT expressible as 3SAT = {φ : φ is a 3-CNF formula for which  assignment A for which (φ, A)  R} R = {(φ, A) : A is a sat. assign. for φ} • satisfying assignment A is a “witness” of the satisfiability of φ (it “certifies” satisfiability of φ) • R is decidable in poly-time CBSSS

16. EXP all languages decidable P NP P  NP  EXP believe all containments proper assuming P ≠ NP, can prove a problem is hard by showing it is NP-complete CBSSS

17. NP-complete problems • L is NP-complete if • L is in NP • every problem in NP reduces to L Theorem (Cook): 3SAT is NP-complete. • opened door to showing thousands of problems we care about are NP-complete CBSSS

18. NP-complete problems • Example reduction: • 3SAT = { φ : φ is a 3-CNF Boolean formula that has a satisfying assignment } (3-CNF = AND of OR of ≤ 3 literals) • IS = { <G, k> | G is a graph with an independent set V’  V of size ≥ k } (ind. set = set of vertices no 2 of which are connected by an edge) CBSSS

19. Ind. Set is NP-complete The reduction f: given φ = (x  y  z)  (x  w  z)  …  (…) we produce graph Gφ: x x … y z w z • one triangle for each of m clauses • edge between every pair of contradictory literals • set k = m CBSSS

20. Ind. Set is NP-complete φ = (x  y  z)  (x  w  z)  …  (…) x x … y z w z • Claim: φ has a satisfying assignment if and only if G has an independent set of size at least k • Proof? • IS  NP. Reduction shows NP-complete. CBSSS

21. Interlude: Transforming Turing Machine computations into Boolean circuits CBSSS

22. Configurations • Useful convention: Turing Machine configurations. Any point in computation . . . x1 x2 … xi xi+1 … xm state = q represented by string: C = x1 x2 …xi q xi+1 xi+2… xm • start configuration for single-tape TM on input x: qstartx1x2…xn CBSSS

23. Turing Machine Tableau • Tableau (configurations written in an array) for machine M on input x=x1…xn: … x1/qs x2 … xn _ • height = time taken • width = space used … x1 x2/q1 … xn _ … x1/q1 a … xn _ ... ... … _/qa _ … _ _ CBSSS

24. Turing Machine Tableau • Important observation: contents of cell in tableau determined by 3 others above it: a/q1 b a a b/q1 a b/q1 a a b a b CBSSS

25. Turing Machine Tableau • Can build Boolean circuit STEP • input (binary encoding of) 3 cells • output (binary encoding of) 1 cell • each output bit is some function of inputs • can build circuit for each • size is independent of size of tableau a b/q1 a STEP a CBSSS

26. Turing Machine Tableau width w tableau for M on input x … x1/qs x2 … xn _ • w copies of STEP compute row i from i-1 … x1 x2/q1 … xn _ ... ... … STEP STEP STEP STEP STEP … CBSSS

27. x1 x2 xn … x1/qs x2 … xn _ Size = O(width x height) STEP STEP STEP STEP STEP ... ... STEP STEP STEP STEP STEP ignore these 1 iff cell contains qaccept Circuit C built from TM M C(x) = 1 iff M accepts input x CBSSS

28. Back to NP-completeness CBSSS

29. Circuit-SAT is NP-complete • Circuit SAT: given a Boolean circuit (gates , , ), with variables y1, y2, …, ym is there some assignment that makes it output 1? Theorem: Circuit SAT is NP-complete. • Proof: • clearly in NP CBSSS

30. NP-completeness • Given L  NP of form L = { x |  y such that (x, y)  R } x1 x2 … xn y1 y2 … ym 1 iff (x,y)  R circuit produced from TM deciding R • hardwire input x; leave y as variables CBSSS

31. 3SAT is NP-complete • Idea: auxiliary variable for each gate in circuit x1 x2 x3 x4 … xn gi gi  • (z1 gi) • (z2 gi) • (gi  z1  z2) • (gi z) • (z  gi)  (z  gi) (z1z2 gi) z z1 z2 gi • (gi z1) • (gi z2) • (z1  z2  gi)  (z1 z2 gi) z1 z2 CBSSS

32. P vs. NP • Most “hard” problems we care about have turned out to be NP-complete • L is in NP • every problem in NP reduces to L • Not known that P ≠ NP arguably most significant open problem in Computer Science and Math. CBSSS

33. L vs. P CBSSS

34. Multitape TMs • A useful variant: k-tape TM read-only input tape … 1 1 0 0 1 1 0 0 0 0 1 1 finite control k read/write heads q0 … 1 1 0 0 1 0 … 1 1 0 0 1 1 0 0 0 0 1 1 k-1 “work tapes” … … 0 CBSSS

35. Space complexity • SPACE(f(n)) = languages decidable by a multi-tape TM that touches at most f(n) squares of its work tapes, where n is the input length, and f :N N • Important space class: L = SPACE(O(log n)) CBSSS

36. L and P • configuration graph: nodes are configurations, edge (C, C’) iff C yields C’ in one step • # configurations for a TM that runs in space O(log n) is nk for some constant k • can determine if reach qaccept or qreject from start configuration by exploring configuration graph (e.g. use DFS) • Conclude: L  P CBSSS

37. EXP L all languages decidable P NP L  P  NP  EXP believe all containments proper CBSSS

38. Logspace A question: • Boolean formula with n nodes • evaluate using O(log n) space? • depth-first traversal requires storing intermediate values • idea: short-circuit ANDs and ORs when possible    1 0 1 CBSSS

39. Logspace • Can we evaluate an n node Boolean circuit using O(log n) space?       1 0 1 0 1 CBSSS

40. A P-complete problem • We don’t know how to prove L ≠ P • But, can identify problems in Pleast likely to be in L using P-completeness. • need stronger reduction (why?) f yes yes f no no L2 L1 CBSSS

41. A P-complete problem • logspace reduction: f computable by TM that uses O(log n) space Theorem: If L2is P-complete, then L2in L implies L = P CBSSS

42. A P-complete problem • Circuit Value (CVAL): given a variable-free Boolean circuit (gates , , , 0, 1), does it output 1? Theorem: CVAL is P-complete. • Proof: • already argued in P CBSSS

43. A P-complete problem • L arbitrary language in P • TM M decides L in nk steps x1 x2 x3 … xn 1 iff x  L circuit produced from TM deciding L • hardwire input x • poly-size circuit; logspace reduction CBSSS

44. L vs. P • Not known that L ≠ P • Interesting connection to parallelizability: • small-space algorithm automatically gives efficient parallel algorithm • efficient parallel algorithm automatically gives small-space algorithm • P-complete problems least likely to be parallelizable CBSSS

45. P vs. BPP CBSSS

46. Communication complexity two parties: Alice and Bob function f:{0,1}n x {0,1}n  {0,1} Alice holds x {0,1}n; Bob holds y {0,1}n • Goal: compute f(x, y) while communicating as few bits as possible between Alice and Bob • count number of bits exchanged (computation free) • at each step: one party sends bits that are a function of held input and received bits so far CBSSS

47. Communication complexity • simple function (equality): EQ(x, y) = 1 iff x = y • simple protocol: • Alice sends x to Bob (n bits) • Bob sends EQ(x, y) to Alice (1 bit) • total: n + 1 bits CBSSS

48. Communication complexity • Can we do better? • deterministic protocol? • probabilistic protocol? • at each step: one party sends bits that are a function of held input and received bits so far and the result of some coin tosses • required to output f(x, y) with high probability over all coin tosses CBSSS

49. Communication complexity Theorem: no deterministic protocol can compute EQ(x, y) while exchanging fewer than n+1 bits. • Proof: • “input matrix”: Y = {0,1}n X = {0,1}n f(x,y) CBSSS

50. Communication complexity • assume without loss of generality 1 bit sent at a time • A sends 1 bit depending only on x: Y = {0,1}n inputs x causing A to send 1 X = {0,1}n inputs x causing A to send 0 CBSSS