Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds

1. Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds DCFS 2008

2. Today’s Goal: • To raise awareness of the tight connection between circuit complexity and Kolmogorov complexity. • And to show that this is useful. • To plant seeds of optimism, regarding the prospects of proving lower bounds in circuit complexity.

3. Kolmogorov Complexity • C(x) = min{|d| : U(d) = x} • Important property • Invariance: The choice of the universal Turing machine U is unimportant. • x is random if C(x) ≥ |x|. • CA(x) = min{|d| : UA(d) = x}

4. Circuit Complexity • Let D be a circuit of AND and OR gates (with negations at the inputs). Size(D) = # of wires in D. • Size(f) = min{Size(D) : D computes f} • We may allow oracle gates for a set A, along with AND and OR gates. • SizeA(f) = min{Size(D) : DA computes f}

5. K-complexity ≈ Circuit Complexity • There are some obvious similarities in the definitions. What are some differences? • A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings. • Given any string x, let fx be the function whose truth table is the string of length 2log|x|, padded out with 0’s, and define Size(x) to be Size(fx).

6. K-complexity ≈ Circuit Complexity • There are some obvious similarities in the definitions. What are some differences? • A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings. • A more fundamental difference: • C(x) is not computable; Size(x) is. • The Minimum Circuit Size Problem (MCSP): {(x,i) : Size(x) ≤ i}.

7. MCSP • MCSP is in NP, but is not known to be NP-complete. • MCSP is not believed to be in P. • Factoring is in BPPMCSP. • Every cryptographically-secure one-way function can be inverted in PMCSP/poly.

8. So how can K-complexity and Circuit complexity be the same? • C(x) ≈ SizeH(x), where H is the halting problem. • For one direction, let U(d) = x. We need a small circuit (with oracle gates for H) for fx, where fx(i) is the i-th bit of x. This is easy, since {(d,i,b) : U(d) outputs a string whose i-th bit is b} is computably-enumerable. • For the other direction, let SizeH(fx) = m. No oracle gate has more than m wires coming into it. Given a description of D (size not much bigger than m) and the m-bit number giving the size of {y in H : |y| ≤ m}, U can simulate DH and produce fx

9. So how can K-complexity and Circuit complexity be the same? • C(x) ≈ SizeH(x), where H is the halting problem. • So there is a connection between C(x) and Size(x) … • …but is it useful? • First, let’s look at decidable versions of Kolmogorov complexity.

10. Time-Bounded Kolmogorov Complexity • The usual definition: • Ct(x) = min{|d| : U(d) = x in time t(|d|)}. • Problems with this definition • No invariance! If U and U’ are different universal Turing machines, CtU and CtU’ have no clear relationship. • (One can bound CtU by Ct’U’ for t’ slightly larger than t – but nothing can be done for t’=t.) • No nice connection to circuit complexity!

11. Time-Bounded Kolmogorov Complexity • Levin’s definition: • Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}. • Invariance holds! If U and U’ are different universal Turing machines, KtU(x) and KtU’(x) are within log |x| of each other. • Let A be complete for E = Dtime(2O(n)). Then Kt(x) ≈ SizeA(x).

12. Time-Bounded Kolmogorov Complexity • Levin’s definition: • Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}. • Why log t? • This gives an optimal search order for NP search problems. • Adding t instead of log t would give every string complexity ≥ |x|. • …So let’s look at how to make the run-time be much smaller.

13. Revised Kolmogorov Complexity • C(x) = min{|d| : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x} (where bit # i+1 of x is *). • This is identical to the original definition. • Kt(x) = min{|d|+log t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}. • The new and old definitions are within O(log |x|) of each other. • Define KT(x) = min{|d|+t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}.

14. Kolmogorov Complexity is Circuit Complexity • C(x) ≈ SizeH(x). • Kt(x) ≈ SizeE(x). • KT(x) ≈ Size(x). • Other measures of complexity can be captured in this way, too: • Branching Program Size ≈ KB(x) = min{|d|+2s : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in space s(|d|)}.

15. Kolmogorov Complexity is Circuit Complexity • C(x) ≈ SizeH(x). • Kt(x) ≈ SizeE(x). • KT(x) ≈ Size(x). • Other measures of complexity can be captured in this way, too: • Formula Size ≈ KF(x) = min{|d|+2t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.

16. …but is this interesting? • The result that Factoring is in BPPMCSP was first proved by observing that, in PMCSP, one can accept a large set of strings having large KT complexity (and by making use of many important results in the theory of pseudorandom generators and derandomization). • (Basic Idea): There is a pseudorandom generator based on factoring, such that factoring is in BPPT for any test T that distinguishes truly random strings from pseudorandom strings. MCSP is such a test.

17. This idea has many variants. • Consider RKT, RKt, and RC. • RKT is in coNP, and not known to be coNP hard. • RC is not hard for NP under poly-time many-one reductions, unless P=NP. • How about more powerful reductions? • Is there anything interesting that we could compute quickly if C were computable, that we can’t already compute quickly? • Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in • But RC is undecidable! Perhaps H is in P relative to RC?

18. This idea has many variants. • Consider RKT, RKt, and RC. • RKT is in coNP, and not known to be coNP hard. • RC, is not hard for NP under poly-time many-one reductions, unless P=NP. • How about more powerful reductions? • PSPACE is in P relative to RC. • NEXP is in NP relative to RC. • Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in poly time, relative to RC [BFNV]. • But RC is undecidable! Perhaps H is in P relative to RC?

19. Relationship between H and RC • Perhaps H is in P relative to RC? • This is still open. It is known that there is a computable time bound t such that H is in DTime(t) relative to RC [Kummer]. • …but the bound t depends on the choice of U in the definition of C(x). • We also know that H is in P/poly relative to RC.

20. This idea has many variants. • Consider RKT, RKt, and RC. • What about RKt? • RKt, is not hard for NP under poly-time many-one reductions, unless E=NE. • How about more powerful reductions? • EXP = NP(RKt). • RKt is complete for EXP under P/poly reductions. • Open if RKt is in P!

21. Kolmogorov Complexity is Circuit Complexity • C(x) ≈ SizeH(x). • Kt(x) ≈ SizeE(x). • KT(x) ≈ Size(x). • Other measures of complexity can be captured in this way, too: • Formula Size ≈ KF(x) = min{|d|+2t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.

22. Kolmogorov Complexity is Circuit Complexity • C(x) ≈ SizeH(x). • Kt(x) ≈ SizeE(x). • KT(x) ≈ Size(x). • Other measures of complexity can be captured in this way, too: • A similar definition captures depth k threshold circuit size. • [This is the clever transition to start the discussion of circuit lower bounds…]

23. Big Complexity Classes • NP • P • . • . • NC • L (Deterministic Logspace)

24. The Main Objects of Interest:Small Complexity Classes • TC0O(1)-Depth Circuits of MAJ gates • AC0 [6] • NC1 Log-Depth Circuits • AC0 can’t compute Mod 2 [FSS,A] • AC0 O(1)-Depth Circuits of AND/OR gates

25. The Main Objects of Interest:Small Complexity Classes • TC0O(1)-Depth Circuits of MAJ gates • AC0 [6] • NC1 Log-Depth Circuits • AC0 can’t compute Mod 2 [FSS,A] • AC0 O(1)-Depth Circuits of AND/OR gates

26. The Main Objects of Interest:Small Complexity Classes • TC0O(1)-Depth Circuits of MAJ gates • NC1 Log-Depth Circuits • AC0 [2]can’t compute Mod 3 [R,S] • AC0 [2] • AC0 O(1)-Depth Circuits of AND/OR gates

27. The Main Objects of Interest:Small Complexity Classes • NC1 Log-Depth Circuits • TC0O(1)-Depth Circuits of MAJ gates • AC0 [6] • AC0 [2] • AC0 O(1)-Depth Circuits of AND/OR gates

28. The Main Objects of Interest:Small Complexity Classes • NC1 poly-size formulae • TC0O(1)-Depth Circuits of MAJ gates • AC0 [6] • AC0 [2] • AC0 O(1)-Depth Circuits of AND/OR gates

29. Complete Problems • NP has complete sets (under polynomial time reducibility ≤P) • These small classes have complete sets, too (under ≤AC°)

30. Reductions • A ≤AC°B means that there is a constant-depth circuit computing A that has the usual AND and OR gates, and also has ‘oracle gates’ for B. B

31. Complete Problems • NC1 • TC0 • AC0 [6] • AC0 [2] • AC0 • sorting, multiplication, division • [Naor,Reingold] Pseudorandom Generator

32. Complete Problems • NC1 • TC0 • AC0 [6] • AC0 [2] • AC0 • BFE: Balanced Boolean Formula Evaluation (AND,OR,XOR) • Word problem over S5

33. The Word Problem Over S5 • A regular set complete for NC1 =

34. Complexity Classes are not Invented – They’re Discovered • NP (SAT, Clique, TSP,…) • P (Linear Programming, CVP, …) • NL (Connectivity, Shortest Paths, 2SAT, …) • L (Undirected Connectivity, Acyclicity, …) • NC1 (BFE, Regular Sets) • TC0 (Sorting, Multiplication, Division) We’re interested in NC1 (for instance) not because we want to build formulae for these functions…

35. Complexity Classes are not Invented – They’re Discovered • NP (SAT, Clique, TSP,…) • P (Linear Programming, CVP, …) • NL (Connectivity, Shortest Paths, 2SAT, …) • L (Undirected Connectivity, Acyclicity, …) • NC1 (BFE, Regular Sets) • TC0 (Sorting, Multiplication, Division) … but because we want to know if the blocks of this partition are distinct.

36. Complexity Classes are not Invented – They’re Discovered • NP (SAT, Clique, TSP,…) • P (Linear Programming, CVP, …) • NL (Connectivity, Shortest Paths, 2SAT, …) • L (Undirected Connectivity, Acyclicity, …) • NC1 (BFE, Regular Sets) • TC0 (Sorting, Multiplication, Division) These classes are real. They’re important.

37. Longstanding Open Problems • Is P = NP? • Is AC0[6] = NP? • Is depth 3 AC0[6] = NP? We’ll focus on questions such as: Is BFE in TC0? Is BFE in AC0[6]?

38. How Close Are We to Proving Circuit Lower Bounds? • Conventional Wisdom: Not Close At All! • No new superpolynomial size lower bounds in over two decades. • Razborov and Rudich: Any “natural” argument proving a lower bound against a circuit class C yields a proof that C can’t compute a pseudorandom function generator. • Since the [Naor, Reingold] generator is computable in TC0, this is bad news.

39. More Modest Goals • Problems requiring formulae of size n3 [Håstad] • Problems requiring branching programs of size nearly n loglog n [Beame, Saks, Sun, Vee] • Problems requiring depth d TC0 circuits of size n1+c [Impagliazzo, Paturi, Saks] • Time-Space Tradeoffs [Fortnow, Lipton, Van Melkebeek, Viglas] • There is little feeling that these results bring us any closer to separating complexity classes.

40. How Close Are We to Proving Circuit Lower Bounds? • How close are the following two statements? • TC0 Circuits for BFE must be of size n1+Ω(1) • For some c>0, TC0 Circuits for BFE must be of size n1+c.

41. How Close Are We to Proving Circuit Lower Bounds? • How close are the following two statements? • TC0 Circuits for BFE must be of size n1+Ω(1) • For some c>0, FTC0 Circuits for BFE must be of size n1+c This is known [IPS’97] This implies TC0≠ NC1 [A, Koucky]

42. Self-Reducibility • A set B is said to be “self-reducible” if B≤PB

43. Self-Reducibility • A set B is said to be “self-reducible” if B≤PB via a reduction that, on input x, does not ask about whether x is in B. • Very well-studied notion. • For example, φ is in SAT if and only if (φ0 is in SAT) or(φ1 is in SAT)

44. Self-Reducibility • Many of the important problems in (or near) NC1 have a special self-reducibility property:

45. Self-Reducibility • Many of the important problems in (or near) NC1 have a special self-reducibility property: Instances of length n are AC0-Turing (or TC0-Turing) reducible to instances of length n½ via reductions of linear size. • Examples: • BFE • the word problem over S5 • MAJORITY • Iterated Product of 3-by-3 Integer Matrices

46. Self Reducibility • BFE A subformula near the root Subformulae near inputs

47. Self Reducibility • S5

48. Self Reducibility • The self-reduction of S5, on inputs of size n, uses (n½ + 1) oracle gates of size n½. • Thus if S5 has TC0 circuits of size nk, it also has circuits of size (n½ + 1)nk/2= O(n(k+1)/2). • Similar arguments hold for other classes (such as AC0[6] and NC1). • More complicated self-reductions can be presented for MAJORITY and Iterated Product of 3-by-3 matrices.

49. A Corollary • If BFE has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size. • If S5 has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size. • If MAJ has AC0[6] circuits, then it has such circuits of nearly linear size. (Etc.) • Thus, e.g., to separate NC1 from TC0, it suffices to show that BFE requires TC0 circuits of size n1.0000001.

50. A Corollary • If BFE has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size. • If S5 has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size. • If MAJ has AC0[6] circuits, then it has such circuits of nearly linear size. (Etc.) • How widespread is this phenomenon? Is it true for SAT? (I.e., can we show NP ≠ TC0 by proving that SAT requires TC0 circuits of size n1.0000001?)