CS151 Complexity Theory

1. CS151Complexity Theory Lecture 6 April 15, 2004

2. Outline • CLIQUE • monotone circuits and problems • Razborov’s lower bound for monotone circuits computing CLIQUE CS151 Lecture 6

3. Clique Recall… IS = { (G, k) | G is a graph with an independent set V’  V of size ≥ k } (independent set = set of vertices no 2 of which are connected by an edge) • IS is NP-complete. CS151 Lecture 6

4. Clique CLIQUE = { (G, k) | G is a graph with a clique of size ≥ k } (clique = set of vertices every pair of which are connected by an edge) • CLIQUE is NP-complete. • reduction? CS151 Lecture 6

5. Circuit lower bounds • We think that NP requires exponential-size circuits. • Where should we look for a problem to attempt to prove this? • Intuition: “hardest problems” – i.e., NP-complete problems CS151 Lecture 6

6. Circuit lower bounds • Formally: • if any problem in NP requires super-polynomial size circuits • then everyNP-complete problem requires super-polynomial size circuits • Proof idea: poly-time reductions can be performed by poly-size circuits using a variant of CVAL construction CS151 Lecture 6

7. Monotone problems • Definition: monotone language = language L  {0,1}* such that x  L implies x’  L for all x  x’. • flipping a bit of the input from 0 to 1 can only change the output from “no” to “yes” (or not at all) CS151 Lecture 6

8. Monotone problems • some NP-complete languages are monotone • e.g. CLIQUE (given as adjacency matrix): • others: HAMILTON CYCLE, SET COVER… • but not SAT, KNAPSACK… CS151 Lecture 6

9. Monotone circuits A restricted class of circuits: • Definition: monotone circuit = circuit whose gates are ANDs (), ORs (), but no NOTs • can only compute monotone functions • monotone functions closed under AND, OR CS151 Lecture 6

10. Monotone circuits • A question: Do all poly-time computable monotone functions have poly-size monotone circuits? • recall: true in non-monotone case CS151 Lecture 6

11. Monotone circuits A monotone circuit for CLIQUEn,k • Input: graph G = (V,E) as adj. matrix, |V|=n • variable xi,j for each possible edge (i,j) • ISCLIQUE(S) = monotone circuit that = 1 iff S  V is a clique: i,j  S xi,j • CLIQUEn, k computed by monotone circuit: S  V, |S| = k ISCLIQUE(S) CS151 Lecture 6

12. Monotone circuits • Size of this monotone circuit for CLIQUEn,k: • when k = n1/4, size is approximately: CS151 Lecture 6

13. Monotone circuits • Theorem (Razborov 85): monotone circuits for CLIQUEn, k with k = n1/4 must have size at least 2Ω(n1/8). • Proof: • rest of lecture CS151 Lecture 6

14. Proof idea • “method of approximation” • suppose C is a monotone circuit for CLIQUEn, k • build another monotone circuit CC that “approximates” C gate-by-gate  CS151 Lecture 6

15. Proof idea • on test collection of positive/negative instances of CLIQUEn,k: • local property: few errors at each gate • global property: many errors on test collection • Conclude: C has many gates CS151 Lecture 6

16. Notation • input: graph G = (V, E) • variable xj,k for each potential edge (j, k) • CC(X1, X2, … Xm), where Xi V, means: i ( j,k  Xixj,k) • For example: CC(X1, X2, … Xm) where the Xi range over all k-subsets of V • this is the obvious monotone circuit for CLIQUEn,k from a previous slide. CS151 Lecture 6

17. Preview • approximate circuit CC(X1, X2, … Xm) • n = # nodes • k = n1/4 = size of clique • h = n1/8 = max. size of subsets Xi • this is “global property” that ensures lots of errors • many graphs G with no k-cliques, but clique on Xiof size h G Xi CS151 Lecture 6

18. Preview • approximate circuit CC(X1, X2, … Xm) • p = n1/8log n • M = (p – 1)hh! • max # of subsets is M (so m ≤ M) • critical for “local property” that ensures few errors at each gate CS151 Lecture 6

19. Building CC • CC (“crude circuit”) for circuit C defined inductively as follows: • CC for single variable x is just CC({x}) • no errors yet! • CC for circuit C of form: • “approximate OR” of CC for C’, CC for C’’  C’ C’’ CS151 Lecture 6

20. Building CC • CC for circuit C of form: • “approximate AND” of CC for C’, CC for C’’ • “approximate OR” and “approximate AND” steps introduce errors  C’ C’’ CS151 Lecture 6

21. Approximate OR  CC’(X1,X2,…Xm’) CC’’(Y1,Y2,…Ym’’) • exact OR: CC(X1,X2,…Xm’,Y1,Y2,…Ym’’) • set sizes still ≤ h • may be up to 2M sets; need to reduce to M C’ C’’ CS151 Lecture 6

22. Approximate OR • throw away sets? bad:many errors • throw away overlapping sets? – better • throw away special configuration of overlapping sets – best CS151 Lecture 6

23. Sunflowers • Definition: (h, p)-sunflower is a family of p sets (“petals”) each of size at most h, such that intersection of every pair is a subset S (the “core”). CS151 Lecture 6

24. Sunflowers Lemma (Erdös-Rado): Every family of more than M = (p-1)hh! sets, each of size at most h, contains an (h, p)-sunflower. • Proof: • not hard • in Papadimitriou CS151 Lecture 6

25. Approximate OR • CC’(X1,X2,…Xm’) • CC’’(Y1,Y2,…Ym’’) • exact OR: CC(X1,X2,…Xm’,Y1,Y2,…Ym’’) • while more than M sets, find (h, p)-sunflower; replace with its core (“pluck”) • approximate OR: CC(pluck(X1,X2,…Xm’,Y1,Y2,…Ym’’) )  C’ C’’ CS151 Lecture 6

26. Approximate AND  • CC’(X1,X2,…Xm’) • CC’’(Y1,Y2,…Ym’’) • exact AND: CC( {(Xi Yj) : 1 ≤ i ≤ m’, 1 ≤ j ≤ m’’} ) • some sets may be larger than h; discard them • may be up to M2 sets. While > M sets, find (h, p)-sunflower; replace with its core (“pluck”) • approximate AND: CC( pluck ( {(XiYj) : |XiYj| ≤ h } )) C’ C’’ CS151 Lecture 6

27. Test collection • Positive instances: all graphs G on n nodes with a k-clique and no other edges. G CS151 Lecture 6

28. Test collection • Negative instances: • k-1 colors • color each node uniformly at random with one of the colors • edge (x, y) iff x, y different colors • no k-clique • include graphs in their multiplicities • makes analysis easier (k-1)-partite graph CS151 Lecture 6

29. Analysis • “false positive”: • negative example • gate is supposed to output 0, but our CC outputs 1 Lemma: each approximation step introduces at most M2(k-1)n/2pfalse positives. CS151 Lecture 6

30. Analysis  • Proof: • case 1: OR CC’(X1,X2,…Xm’) CC’’(Y1,Y2,…Ym’’) CC(pluck(X1,X2,…Xm’,Y1,Y2,…Ym’’)) • given “plucking”: replace Z1… Zp with Z • bad case: clique on Z, and each petal is missing at least one edge C’ C’’ CS151 Lecture 6

31. Analysis • what is the probability of a repeated color in each Zibut no repeated colors in Z? Pr[R(Z1)R(Z2)…R(Zp) R(Z)] ≤ Pr[R(Z1)R(Z2)…R(Zp)|R(Z)] (definition of conditional probability) = i Pr[R(Zi) | R(Z)] (independent events given no repeats in Z) ≤ i Pr[R(Zi)] (obviously larger) event R(S) = repeated colors in S CS151 Lecture 6

32. Analysis • for every pair of vertices in Zi, probability of same color is 1/(k-1) • R(Zi) ≤ (h choose 2)/(k-1) ≤ ½ • i Pr[R(Zi)] ≤ (½)p • # negative examples is (k-1)n • # false positives in given plucking step is at most (½)p(k-1)n • at most M plucking steps • # false positives at OR ≤ M(½)p(k-1)n CS151 Lecture 6

33. Analysis  • case 2: AND CC’(X1,X2,…Xm’) CC’’(Y1,Y2,…Ym’’) CC(pluck( {(XiYj) : |XiYj| ≤ h } )) • discarding sets (XiYj) larger than h can only make circuit accept fewer examples • no false positives here C’ C’’ CS151 Lecture 6

34. Analysis • up to M2 pluckings • each introduces at most (½)p(k-1)n false positives (previous slides) • # false positives at AND ≤ M2(½)p(k-1)n CS151 Lecture 6

35. Analysis • “false negative”: • positive example; • gate is supposed to output 1, but our CC outputs 0 Lemma: each approximation step introduces at most false negatives. CS151 Lecture 6

36. Analysis • Proof: • Case 1: OR • plucking can only make circuit accept more examples • no false negatives here. • Case 2: AND CC’(X1,X2,…Xm’) CC’’(Y1,Y2,…Ym’’) CC(pluck( {(XiYj) : |XiYj| ≤ h } ))  C’ C’’ CS151 Lecture 6

37. Analysis • discarding set Z = (XiYj) larger than h may introduce false negatives • any clique that includes Z is a problem; there are at most such positive examples, since |Z|>h • at most M2 such deletions • we’ve seen plucking doesn’t matter CS151 Lecture 6

38. Analysis Lemma: every non-trivial CC outputs 1 on at least ½ of the negative examples. • Proof: • CC contains some set X of size at most h • accepts all neg. examples with different colors in X • probability X has repeated colors is R(X) ≤ (h choose 2)/(k-1) ≤ ½ • probability over negative examples that CC accepts is at least ½. CS151 Lecture 6

39. Finishing up • First possibility: trivial CC, rejects all positive examples • every positive example must have been false negative at some gate • number of gates must be at least: CS151 Lecture 6

40. Finishing up • Second possibility: CC accepts at least ½ of negative examples • every negative example must have been false positive at some gate • number of gates must be at least: CS151 Lecture 6

41. Finishing up Both quantities are at least 2Ω(n1/8) CS151 Lecture 6

42. Conclusions • A question (true in non-monotone case): Do all poly-time computable monotone functions have poly-size monotone circuits? • if yes, then we would have just proved P ≠ NP • why? CS151 Lecture 6

43. Conclusions • unfortunately, answer is no • Razborov later showed similar (super-polynomial) lower bound for MATCHING, which is in P… CS151 Lecture 6