Poly-logarithmic independence fools circuits, a survey

1. Poly-logarithmic independence fools circuits, a survey Mirmojtaba (Mojtaba) Gharibi December 2010

2. Overview • In 1990, Linial and Nisan conjectured that no circuit can distinguish the uniform distribution from any poly-logarithmic independent distribution • In a recent breakthrough, after about two decades of no progress, the conjecture was settled by Mark Braverman

3. Overview : circuits • A circuit consisting of polynomial number of , and gates. • gates only appear at the input nodes • Depth of the circuit is constant and is the number of gates from the input to the output

4. Overview:R-independent distribution • Consider a set of random variables distributed according to an r-independent distribution over . • Looking at any subset of size at most , the probability that any bit in that subset be is independent of any other outcomes in that subset

5. An example • Consider the following probability distribution over : where are set uniformly and independently at random. • This is an example of a -independent distribution, since choosing any subset of size at most , each variable is totally independent from any outcome within that set:

6. An example • Subsets of size at most of are:

7. Overview:R-independent distribution • So, -independent distributions are in some sense locally random • Whereas on the other hand, the uniform distribution is globally random

8. Linial and Nisan 1990’s conjecture • circuit cannot distinguish local randomness from global randomness. • i.e. any function computable by an circuit aiming to distinguish the Uniform distribution from an -independent distribution output the same thing on inputs drawn from both distributions except with a negligible bias.

9. A Definition • A distribution is said to the Boolean function if: Or equivalently

10. Linial and Nisan 1990’s conjecture • We are interested to know how large needs to be in order for to any circuit of size operating on • LN conjecture (with relaxed parameters):

11. Small bias • So for a polynomially small , a polylogarithmic is good enough. • We cannot take for granted that a polynomially small , can be boosted to a constant probability by taking majority. Why? circuits are not capable of taking majority. Polynomially small bias is taken as negligible for circuits.

12. Motivation and applications • It says that if an circuit accepts truly random bits, then it also accepts pseudorandom bits. So if one wants to distinguish random bits from pseudorandom bits, he needs a more powerful circuit, possibly with exponentially more gates or more powerful gates like XOR and MAJORITY or with greater depth (e.g. logarithmic). • So it gives a better understanding of limitations of an important complexity class. The result may be later used as a tool for proving lower bounds.

13. Motivation and applications • Since 1980’s we have known many serious limitations of circuits like many specific pseudorandom distributions that fool circuits. The conjecture actually says that anyr-independent distribution will fool them. So this gives a large class of distributions that look random to circuits. • For instance, linear codes with poly-logarithmic seed length can be PRGs for .

14. History • In 2007, in the first noticeable development, Bazzi settled the conjecture for circuits. His proof was about 50 pages. • In 2008, Razborov simplified Bazzi’s proof to a 3-page proof. • Finally, in 2009, Mark Braverman settled the conjecture with a short proof.

15. Bazzi’s results • Bazzi’s theorem: - independence depth 2 circuits where

16. Bazzi’s results • Bazzi’s proof was based on harmonic and poset analysis techniques. He also used Linial, Mansour and Nisan celebrated result of 1993 of low-degree real polynomial approximation. Razborov’s proof does not use Fourier analysis techniques except of making connection to Linial, Mansour, and Nisan’s theorem.

17. Rest of this presentation • Razborov-Smolensky’s approximation technique by low-degree polynomials over finite fields • Linial, Mansour and Nisan’s approximation technique by low-degree real polynomials • Mark Braverman’s proof of the conjecture

18. Razborov-Smolensky’s approximation • Recall : the technique was used in the class to prove • It involves approximating a Boolean function using a low-degree polynomial over finite fields. • Then, knowing the properties of the low-degree polynomials, we can talk about the properties of F.

19. Razborov-Smolensky’s approximation • For example, for PARITY: • Any function can be well approximated by low-degree polynomials • can be represented with a high-degree polynomial • A low-degree polynomial cannot approximate a high-degree polynomial • Hence

20. Razborov-Smolensky’s approximation • Denote the approximation of Boolean function with a low-degree polynomial . • In Rozborov-Smolenskey’s technique, the criteria for a good approximation is that for a large fraction of inputs. However, when , they may largely disagree.

21. Razborov-Smolensky’s approximation 1 0 1 0

22. Linial, Mansour and Nisan’s approximation • Denote the approximator of the Boolean function with a low-degree real polynomial • LMN says that approximation of is possible via low-degree real polynomials. But there is no guarantee that on any inputs. Most likely for any inputs,

23. Linial, Mansour and Nisan’s approximation • [LMN93]: Every Boolean function computable by an circuit of size and depth , can be approximated by a real low-degree polynomial of degree :

24. Linial, Mansour and Nisan’s approximation 1 0

25. The conjecture We wish to prove : • For any -independent distribution where :

26. The basic idea • If we can find a “good” low-degree polynomial approximation of F we are done. Because: • Low-degree polynomials are composed of low-degree terms. • The expectation of any polynomial is the sum of the expectation of its terms. • Each term’s expectation is exactly the same under and distribution. So the polynomial’s expectation is also the same.

27. The first step in the proof • We will construct a distribution on the polynomial over a proper finite field such that with high probability agrees with on any given input. So for any given measure , with high probability we have an approximator having a small error, which implies that there exists a specific approximator having a small error with respect to .

28. The first step in the proof • If the polynomial is a good approximator (i.e. for some small ), one can transform to a good approximator and show that the conjecture holds. • However, most likely it is not the case! Why?

29. 1 0 1 0 Small fraction

30. The first step in the proof • We want a good approximation too, but is behaving wildly in its bad region. • Let’s first construct , then we will deal with this problem in the second step of the proof!

31. Approximator’s Construction • NOT gates will all appear at input nodes which are easy to approximate. • Anywhere else, we have AND/OR gates. • As usual we use induction on the depth of the circuit. • We describe the construction of AND approximator. OR’s construction follows from the symmetry between 1 & 0 and AND & OR.

32. AND’s approximator • Consider set of indexes . For a parameter , we prepare a collection of of its subsets in the following way: • For each of we prepare at least random subsets of • We include in each subset each of the indexes with independent probability . We also include . Denote these subsets with For convenience let us assume is a power of 2, e.g. . …

33. AND’s approximator • Construct: is the approximator of . Let us for now focus on the case that all have approximated correctly.We later bound all the errors by the union bound.

34. AND’s approximator • approximates correctly when . • However, we may err when .

35. AND’s approximator • When , let us say of the ’s have had been zero.( is the number of zeros). approximates correctly if at least one of hits exactly one zero. • The probability of a wrong approximation can be shown to be at most . Since it is true for any value of , we can actually find a collection of that yields that error bound. • By the union bound • The degree of the polynomial is

36. Insight • Our goal was to make our behave nicely in its bad region. • Here is the idea: • Given our choices for , there exist another Boolean formula computable by an circuit of slightly more depth and size which can determine if has erred or not. Denote it with .

37. 1 0 1 0 1 0

38. Insight • How can we use this to make have a better behaviour in its bad region? • Set • Compute .

39. 1 0 1 0 1 0 1 0

40. Insight • One can show that is a good approximator of with respect to both measures and . and are small: • Also is a good approximator of . But it is Boolean, not a polynomial. We wish was exactly behaving like !!!

41. The second step in the proof • Here is our new strategy: • Since we failed to find a good approximator of directly, we try to find a good low-degree approximator of which we denote by . • Since is a good approximator of , is also a good approximator of . • By a good approximator we mean an approximator which by known techniques can be transformed into an approximator.

42. The second step in the proof • We approximate with a low-degree real polynomial of degree based on Linial, Nisan and Mansour technique. Denote the approximation by . We use this approximation to form . We choose t large enough to have close to 0 when is close to 1.

43. 1 0 1 0 1 0 1 0 1 0

44. The second step in the proof • Since we have used LMN technique, we can just say that is a good approximation of with respect to uniform distribution. (though it is also good with respect to , since we believe in the conjecture that a Boolean formula like outputs the same things most of the times with respect to measure or )

45. The second step in the proof • However, this is enough for us, since we can transform into by • And then using and playing with inequalities -which is skipped for our present purpose- will lead us to the proof.

46. Finally • By setting the parameter properly (i.e. ) one can settle the conjecture: • Any independent distribution circuits of size and depth :

47. References • [1] Linial, N., and Nisan, N. "Approximate inclusion exclusion." Combinatorica, 1990: 349- 365. • [2] Braverman, M. "Poly-logarithmic independence fools AC0 circuits." IEEE conference on Computational Complexity, 2009: 3-8. • [3] Bazzi, L. M. J. "Polylogarithmic independence can fool DNF formulas." Proceedings of the 48th annual IEEE symposium on Foundations of Computer Science, 2007: 63-73. • [4] Bazzi, L. M. J. "Polylogarithmic independence can fool DNF formulas." SIAM Journal on Computing (SICOMP), 2009. • [5] Razborov, A. A. "A simple proof of Bazzi's theorem." Electronic Colloquium on Computational Complexity, 2008: Report No. 81. • [6] Razborov, A. A. "Lower bounds for the size of circuits of bounded depth with basis {&,⊕}." Mathematicheskie Zametki, 1987: 598–607. English translation in Math. Notes. Acad.Sci. USSR, 1987: 333–338. • [7] Smolensky, R. "Algebraic methods in the theory of lower bounds for Boolean circuit complexity." Proceedings of the nineteenth annual ACM symposium on Theory of computing, 1987: 77-82. • [8] Linial, N., Mansour, Y. and Nisan, N. "Constant depth circuits, Fourier transform, and learnability." Journal of the ACM, 1993: 607-620. • [9] www.scottaaronson.com/blog/ retrieved on Nov 25th, 2010