Boolean Circuits of Depth-Three and Arithmetic Circuits with General Gates

1. Boolean Circuits of Depth-Three and Arithmetic Circuits with General Gates OdedGoldreich Weizmann Institute of Science Based on Joint work with AviWigderson Original title: “On the Size of Depth-Three Boolean Circuits for Computing MultilinearFunctions”, ECCC TR13-043.

2. Constant Depth Boolean Circuits Paritynrequires depth dcircuits of size exp((n1/(d-1))). Famous frontier: Stronger circuit models. Another frontier: Stronger lower bounds (i.e., exp((n))). Multi-linear functions : x=(x(1),…,x(t)), x(i)0,1n F(x(1),…,x(t)) = (i_1,…,i_t)Txi_1(1)  xi_t(t)associated with tensor T  [n]t Think of t=2,… log n Conj (sanity check): For every t>1, there exists a t-linear function that requires depth-three circuits of sizeexp((tnt/(t+1))). [holds for t=1…]

3. The Program* t-linear functions x=(x(1),…,x(t)), |x(i)|=n F(x(1),…,x(t)) = (i_1,…,i_t)Txi_1(1)  xi_t(t) Conj (1st sanity check): For every t>1, there exists a t-linear function that requires depth-three circuits of sizeexp((tnt/(t+1))). [holds for t=1] Goal: For every t>1, present an explicit t-linear function that requires depth-three circuits of sizeexp((tnt/(t+1))). [holds for t=1] A 2nd sanity check: Consider a restricted model of (depth-three) circuits, and prove the L.B. in it. *) Taking advantage of Avi’s absence.

4. Arithmetic Circuits with General Gates Motivation: Depth-three Boolean Circuits for Paritynare obtained by implementing asqrt(n)-way sum of sqrt(n)-way sums. In general, depth-three BC are obtained via depth-two AC with general ML-gates. We get depth-three BC for F of size exponential in C2(F) Model: Depth-two (set-)multi-linear circuits with arbitrary (set-)multi-linear gates. Complexity measure (C2) = the (max.) arity of a gate. Recall: We use a fix partition of the variables, and multi-linear means being linear in each variable-block. Depth-three BC obtained this way are restricted in (1) their structure arising from direct composition, and (2) ML gates.

5. Arithmetic Circuits with General Gates (cont.) Model: Unbounded-depth (set-)multi-linear circuitswith arbitrary (set-)multi-linear gates. Complexity measure (C) = max(arity, #gates). PROP: Every ML function F has a depth-three BC of sizeexp(O(C(F)). PF: guess & verify. THM: There exist bilinear functions F such that C(F)=sqrt(n) but C2(F)=(n2/3). OBS: For every t-linearF,Ct+1(F) ≤ 2C(F).

6. Arith. Circuits with General Gates: Results Model: Unbounded-depth (set-)multi-linear circuitswith arbitrary (set-)multi-linear gates. Complexity measure (C) = max(arity, #gates);C2for depth-two. THM 1: There exist bilinear functions F such that C(F)=sqrt(n) but C2(F)=(n2/3). THM 2: For every t-linear function F it holds that C(F) ≤ C2(F) = O(tnt/(t+1)). THM 3: Almost all t-linear functions F satisfy C2(F) ≥ C(F) = (tnt/(t+1)). Open: An explicit function as in Thm 3; for starters (tn0.51).

7. Arith. Circuits with General Gates: Results (cont.) Model: Unbounded-depth (set-)multi-linear circuitswith arbitrary (set-)multi-linear gates. Complexity measure (C) = max(arity, #gates);C2for depth-two. Open: An explicit function as in Thm 3; for starters (tn0.51). An approach (a candidate): The 3-linear function assoc. with tensor T=(i,j,k): |i-(n/2)|+|j-(n/2)|+|k-(n/2)|≤n/2. PROP: The complexity of the above 3-linear function is lower bounded by the maximum complexity of all bilinear functions associated w. Toeplitz matrices. THM: If matrix M has rigidity m3 for rank m, then the corresponding bilinear function has complexity (m). Note: A restricted notion of (“structured”) rigidity suffices. Open:Show that Toeplitz matrix w. rigidity n1.51for rank n0.51.

8. Comments on the proofs Model: Multi-linear circuits with arbitrary multi-linear gates. Complexity measure (C) = max(arity, #gates);C2for depth-two. THM 1: There exist bilinear functions F such that C(F)=sqrt(n) but C2(F)=(n2/3). PF idea:s=sqrt(n), f(x,y)=g(x,L1(y),…,Ls(y)). THM 2: For every t-linear function F it holds that C(F) ≤ C2(F) = O(tnt/(t+1)). PF: Covering by m cubes of side m. THM 3: Almost all t-linear functions F satisfy C2(F) ≥ C(F) = (tnt/(t+1)). PF: A counting argument. THM 4: If matrix M has rigidity m3 for rank m, then the corresponding bilinear function has complexity (m). PF idea: The m linear function yield a rank m matrix, whereas the m quadratic forms (in variables) cover m3 entries.

9. Add’l comments on the proof of THM 1 Model: Multi-linear circuits with arbitrary multi-linear gates. Complexity measure (C) = max(arity, #gates);C2for depth-two. THM 1: There exist bilinear functions F such that C(F)=sqrt(n) but C2(F)=(n2/3). PF:For s=sqrt(n), let f(x,y)=g(x,L1(y),…,Ls(y)), where g is generic (over n+s bits), each Licomputes the sum of s variables in y. A generic depth-two ML circuit of complexity m computes f as B(F1(x),…,Fm(x),G1(y),…,Gm(y)) + i[m]Bi(x,y) where the Bi’s are quadratic and each function has aritym. Hitting y with a random restriction that leaves one variable alive in each block, we get B(F1(x),…,Fm(x),G’1(y’),…,G’m(y’)) + i[m]B’i(x,y’) where each B’I(and G’I) depends on O(m/s) variables. Hence, the description length is O(m3/s) ; cf. to ns=n2/s.

10. Structured Rigidity DEF: Matrix M has (m1,m2,m3)-structured rigidity for rank r if matrix R of rank r the non-zeros of M-R cannot be covered by m1(gen.) m2-by-m3rectangles. Rigidity m1m2m3 implies (m1,m2,m3) structured rigidity for the same rank, but not vice versa. THM 5: There exist matrices of (m,m,m)-structured rigidity for rank m that do not have rigidity 3mn for rank 0(let alone for rank m). For every m[n0.51,n0.66]. PF: Consider a random matrix with 3mn one-entries. THM 4’: If matrix M has (m,m,m)-structured rigidity for rank m, then the corresponding bilinear function has complexity (m). PF idea: The proof of Thm 4 goes through w.o. any change.

11. END Slides available athttp://www.wisdom.weizmann.ac.il/~oded/T/kk.pptx Paper available at http://www.wisdom.weizmann.ac.il/~oded/p_kk.html