27 and Still Counting: Iterated Product, Inductive Counting, and the Structure of P

1. 27 and Still Counting: Iterated Product, Inductive Counting, and the Structure of P ImmermanFest, Vienna, July 13, 2014

2. Why 27?? 33 years ago this month, the earth shifted:

3. Why 27?? Let us recall the landscape prior to July, 1987.

4. In the Beginning (1956) … • …there was the Chomsky Hierarchy. co-c.e. c.e. co-CSL?? CSL co-CFL CFL Regular

5. In the Beginning (1956) … • …there was the Chomsky Hierarchy. Π01 Σ01 co-CSL?? CSL co-CFL CFL Regular

6. In the very Beginning (1943) … • …there was the Arithmetic Hierarchy. Π03 Σ03 Π02 Σ02 Π01 Σ01

7. …and it was good! • Alternative characterizations in terms of • Logic (Alternating quantifiers and recursive predicates) • Alternating Turing machines. • Oracle Turing machines. • FO(Halting Problem) [not really] • AC0-Turing reductions to the Halting Problem [not really]

8. AC0 Reductions B B B • A ≤AC°B means that there is a constant-depth circuit computing A that has the usual AND, OR, and NOT gates, and also has ‘oracle gates’ for B.

9. The Arithmetic Hierarchy begat • …the Polynomial Hierarchy. Πp3 Σp3 Πp2 Σp2 coNP=Πp1 Σp1=NP

10. …which was also pretty good! • Alternative characterizations in terms of • Logic (Alternating quantifiers and recursive predicates) • Alternating Turing machines. • Oracle Turing machines. • FO(SAT) [not really] • AC0-Turing reductions to SAT [not really] • Some fairly natural complete problems at levels 2 and 3.

11. The Polynomial Hierarchy begat • …the NL Alternation Hierarchy. Πlog3 Σlog3 Πlog2 Σlog2 coNL=Πlog1 Σlog1=NL

12. …and it was not so great. • Alternative characterizations in terms of • Logic [if you played with the definitions] • Alternating Turing machines. • Oracle Turing machines. • FO(GAP) • AC0-Turing reductions to GAP • Some fairly natural complete problems at levels 2 and 3. [Rosier]

13. So what was the problem? • You’d like NLNL to be a subclass of P. • Unfortunately, it’s NP! • So Ruzzo, Simon, and Tompa introduced “RST” relativization. (The oracle machine must work deterministically while writing a query.) • May seem artificial – but it corresponds to AC0- and FO-Turing reducibility. • So of course, this gives us another hierarchy:

14. The NL Oracle Hierarchy • Where is the Alternation Hierarchy? coNLNLNL NLNLNL coNLNL NLNL ALH coNL NL

15. The NL Oracle Hierarchy • A lovely structure? coNLNLNL NLNLNL coNLNL NLNL ALH coNL NL

16. The NL Oracle Hierarchy • A lovely structure? Or a fine mess? coNLNLNL NLNLNL coNLNL NLNL ALH coNL NL

17. And the walls came a tumblin’ down • And here’s where you expect me to mention NL=coNL… • …but this collapse happened in 1986! • In two phases: • The NL Alternation Hierarchy = LNL [Lange, Jenner, Kirsig] • The NL Oracle Hierarchy = LNL [Schöning, Wagner][Buss, Cook, Dymond, Hay]

18. The NL “Hierarchy” • But true enlightenment had not yet arrived. • Within the year, the world would know that NL=coNL. LNL coNL NL

19. Impact of Inductive Counting • The discovery that NL=coNL provided the single most significant insight into the nature of space-bounded computation since the 60’s. • The list of complexity classes that have been impacted by these new insights into nondeterminism includes • LogCFL, VP, VP(2), DET, PL, #L, UL, ModkL, SAC1(log), RUL, CNL, … • Some of these are not so important…but some assuredly are!

20. The NC Hierarchy NC2 TC1 AC1 NL L NC1 TC0 AC0

21. The NC Hierarchy Determinant These …not so much. NC2 TC1 CFLs AC1 But there are other important problems in the vicinity. These have natural complete sets. NL L NC1 TC0 AC0

22. Linear Algebra and Logspace • The connection between linear algebra and logspace-bounded computation was discovered rather late, and via excessively difficult arguments – primarily because inductive counting was discovered so late. • The relevant logspace classes were initially studied without any motivation from natural problems. • What are these classes? • PL, #L, GapL, C=L

23. Probabilistic Logspace • PL was introduced by [Gill, 1977], by analogy with PP (defined in the same paper). • The history of upper bounds on the complexity of PL: • PSPACE [Gill, 1977] • SPACE(log6n) [Simon, 1981] • NC2 [Borodin Cook Pippenger, 1982] • L#L [Jung, 1985] • What was the problem??

24. The problem with PL • In a nutshell, the problem is that PL machines can continue to do useful work after exponential time. • For example: NL = RL!

25. The problem with PL • In a nutshell, the problem is that PL machines can continue to do useful work after exponential time. • For example: NL = RL! (If “RL” is defined without a polynomial time bound.) • But with Inductive Counting as a tool, it’s easy to see that PL is the same class, with or without a polynomial-time restriction. (Jung did this the hard way, in 1985.) • Thus PL is characterized by NL machines with more accepting than rejecting paths.

26. Linear Algebra and #L • The connection between #P and the Permanent was made in 1979. • #L was explicitly defined and studied in 1990. • The fact that Determinant is complete for GapL (= #L - #L) was not discovered until 1991-1992. • An immediate consequence was: {M : Det(M) > 0} is complete for PL. • …but we much more often ask: Is Det(M)=0?

27. Singular matrices • The set of singular matrices is complete for C=L (characterized by NL machines where the number of accepting and rejecting computations are equal.) • Hierarchies: • AC0(C=L) = C=L U C=LC=L U … • AC0(PL) • AC0(#L)

28. Singular matrices • The set of singular matrices is complete for C=L (characterized by NL machines where the number of accepting and rejecting computations are equal.) • Hierarchies: • AC0(C=L) = C=L U C=LC=L U … • AC0(PL) = PL Collapse! [Beigel, Fu, 1997] • AC0(#L)

29. Singular matrices • The set of singular matrices is complete for C=L (characterized by NL machines where the number of accepting and rejecting computations are equal.) • Hierarchies: • AC0(C=L) = LC=L Collapse! [A. Beals, Ogihara] • AC0(PL) = PL Collapse! [Beigel, Fu, 1997] • AC0(#L)

30. Singular matrices • The set of singular matrices is complete for C=L (characterized by NL machines where the number of accepting and rejecting computations are equal.) • Hierarchies: • AC0(C=L) = LC=L Collapse! [A. Beals, Ogihara] • AC0(PL) = PL Collapse! [Beigel, Fu, 1997] • AC0(#L) = ???? (No collapse known.)

31. The C=L Hierarchy LC=L C=L coC=L

32. The C=L Hierarchy Rank Nonsingular Matrices Singular Matrices So this is strong evidence that these three classes are distinct.

33. Another view of #L • A complete problem for #L is: Counting the number of paths from s to t in a directed graph. • Equivalently: it’s the problem of computing the (1,1) entry of a product of several nxn matrices with entries in the Natural Numbers. • Similarly, iterated product of integer matrices is complete for GapL (i.e., the determinant class). • Immerman and Landau highlighted the connection between complexity classes and iterated product over several algebras.

34. Immerman & Landau Curiously, missing from this list is the most “algebraic” class: VP

35. VP and Polynomial Degree • Valiant introduced the study of the arithmetic complexity of n-variate polynomials with algebraic degree bounded by poly(n). Later, this came to be called VP (or VPR for algebra R).

36. VP and Polynomial Degree • VP has a nice circuit characterization: log depth poly-size circuits with unbounded fan-in + gates, fan-in 2 * gates. (Semiunbounded fan-in circuits). [VSBR ‘83, Vinay ‘91] • Over the Boolean ring, this was discovered earlier [Ruzzo ‘79, Venkateswaran ‘87]. • VP({0,1},V,Λ) is also known as LogCFL and SAC1. • LogCFL = problems logspace-reducible to CFLs. • SAC1 = Semi-unbounded fan-in circuits of depth log1 n.

37. SemiUnbounded Fan-In • Immediately after NL=coNL was established, SAC1 was shown to be closed under complement, too (using inductive counting). [Borodin, Cook, Dymond, Ruzzo, Tompa] • Thus the following two models are equivalent: • Unbounded fan-in V, bounded fan-in Λ • Bounded fan-in V, unbounded fan-in Λ • How about other algebras? • A similar result over, say GF2 would be remarkable: VP(2) would equal AC1!

38. VP and Iterated “Product” • There aren’t that many “natural” complete problems for VP. (Using the connection to LogCFL, one class of complete problems is counting # of parse trees showing that x is in L, for certain CFLs L.) • Recently, a complete problem was added to this list, that looks a lot like an “iterated product”: • Tensor Contraction [Capelli, Durand, Mengel]

39. VP and Iterated “Product” • Matrices with b rows and c columns are 2-dimensional tensors of order [b,c]. • Given 2 tensors A and B (of orders, say [a,b,c] and [c,d,e]), their contraction has order [a,b,d,e] • Given a list of 1-, 2-, and 3-dimensional tensors (or even poly-dimensional tensors), computing an entry of their iterated contraction is complete for VP. • But this is non-associative. (The nesting must be given, say, as a tree.)

40. Undirected Reachability • The same paper that showed LogCFL = coLogCFL also applied inductive counting to SL (the problems reducible to reachability in undirected graphs). • SL had its own hierarchy, inside the NL hierarchy. • SL was known to be in RL (with a poly run-time), and the new insight was a coRL algorithm. Many developments followed. • Reingold ended this saga, showing SL=L.

41. Where do these classes fit? AC1 VP(2) VP(3) VP(5) VP SAC1 Mod2L Mod3L Mod5L #L NL

42. Gal and Wigderson Lots of nonuniform inclusions: AC1 VP(2) VP(3) VP(5) VP SAC1 Mod2L Mod3L Mod5L #L NL

43. NL in #L is Open But UL is in #L (by definition). AC1 VP(2) VP(3) VP(5) VP SAC1 Mod2L Mod3L Mod5L #L NL UL

44. [GW] + Inductive Counting More nonuniform inculsions [Reihnardt, A] AC1 VP(2) VP(3) VP(5) VP SAC1 Mod2L Mod3L Mod5L #L UL/poly = NL/poly

45. [GW] + Inductive Counting These hold uniformly if SAT needs size 2n/100. AC1 VP(2) VP(3) VP(5) VP SAC1 Mod2L Mod3L Mod5L #L UL/poly = NL/poly

46. A sampling of open questions • Inductive counting (and related techniques) have thus far failed to show: • UL = coUL • C=L = coC=L • NL = UL • A collapse of the #L hierarchy • Any relationship between AC1 and #L. (Immerman and Landau conjecture that TC1 reduces to #L.)

47. AC1 and VP • TC1 = #AC1(mod pn) [Reif, Tate] • Arithmetic degree nlog n. • AC1 is contained in #AC1(mod pn) where all multiplication gates have fan-in log n. [A, Jiao, Mahajan, Vinay] • Arithmetic degree nloglog n. • Thus if the degree could be lowered to poly(n), working over Q instead of [Z mod pn], one would have AC1 contained in VP. • If also GapL = VP, we’d have AC1 in GapL.

48. A Continuing Legacy • Inductive Counting continues to shape the research agenda. • Case in point: Catalytic Computing (STOC ‘14 [Buhrman, Cleve, Koucky, Loff, Speelman]) • CL = problems solvable using logarithmic space augmented with a “full memory” that must be restored to its original state. • TC1 is contained in CL, which is contained in ZPP. • What about CNL?

49. CNL • A new “nondeterministic” class. An unfamiliar model. How can one program in this model? • There is currently exactly one nondeterministic algorithm known in this model. It is used in order to show… • CNL = coCNL.

50. Legacy • A lessening of confidence in the framework of complexity classes, and an increase in humility regarding popular conjectures. • An invaluable insight into the nature of nondeterminism, and space-bounded computation in general. • A fundamental shift in the way that we approach questions in complexity theory.