Cracks in the Defenses: Scouting Out Approaches on Circuit Lower Bounds

1. Cracks in the Defenses:Scouting Out Approaches on Circuit Lower Bounds CSR 2008

2. Introduction • How far are we from proving circuit lower bounds? • I have no idea! • There is a lot of pessimism, based on • The lack of any good circuit lower bounds • The [Razborov,Rudich] “natural proofs” obstacle • Today, we’ll make some observations that may cause some of you to be less pessimistic.

3. But First…Why Circuits? • 2 Basic models of computation • Programs (one program – works for every input length) • Circuits (different circuit for each input length) • One crucial difference: circuit lower bounds can be used to prove intractability results for fixed input sizes. • Program run-time lower bounds can’t.

4. An example: the Game of Checkers • Computing strategies for Checkers requires exponential time. • More precisely, given an n-by-n Checkers board with checkers on it, no program can compute an optimal next move in fewer than c2n – d steps, for some constants c and d. • n-by-n Checkers is complete for EXP.

5. An example: the Game of Checkers • Computing strategies for Checkers requires exponential time. • More precisely, given an n-by-n Checkers board with checkers on it, no program can compute an optimal next move in fewer than c2n – d steps, for some constants c and d. • Thus any program solving this problem must run very slowly on large inputs. This is the essence of asymptotic analysis.

6. An example: the Game of Checkers • Computing strategies for Checkers requires exponential time. • More precisely, given an n-by-n Checkers board with checkers on it, no program can compute an optimal next move in fewer than c2n – d steps, for some constants c and d. • This is a much stronger statement about complexity than we are able to prove for most problems (such as NP-complete problems).

7. An example: the Game of Checkers • Computing strategies for Checkers requires exponential time. • More precisely, given an n-by-n Checkers board with checkers on it, no program can compute an optimal next move in fewer than c2n – d steps, for some constants c and d. • but…Conceivably, there is a hand-held device that computes optimal moves, even for Checker boards of size 1000-by-1000! • …because we don’t know if EXP is in P/poly (the class of problems with small circuits).

8. An Example of what can be done, given a circuit size lower bound • Theorem: Any circuit that takes as input a logical formula (in WS1S) of length 616 and produces as output a correct answer, saying if the formula is valid or not, has at least 10123 gates. (Stockmeyer, 1974) • (Proof sketch): There is a problem requiring exponential circuit size that is efficiently reducible to WS1S.

9. An Example of what can be done, given a circuit size lower bound • Theorem: Any circuit that takes as input a logical formula (in WS1S) of length 616 and produces as output a correct answer, saying if the formula is valid or not, has at least 10123 gates. (Stockmeyer, 1974) • What we need: Similar lower bounds, but for problems in NP such as SAT or FACTORING. • We would even be glad to get lower bounds for restricted classes of circuits.

10. Big Complexity Classes • NP • P • . • . • NC • L (Deterministic Logspace)

11. The Main Objects of Interest:Small Complexity Classes • TC0O(1)-Depth Circuits of MAJ gates • AC0 [6] • NC1 Log-Depth Circuits • AC0 can’t compute Mod 2 [FSS,A] • AC0 O(1)-Depth Circuits of AND/OR gates

12. The Main Objects of Interest:Small Complexity Classes • TC0O(1)-Depth Circuits of MAJ gates • AC0 [6] • NC1 Log-Depth Circuits • AC0 can’t compute Mod 2 [FSS,A] • AC0 O(1)-Depth Circuits of AND/OR gates

13. The Main Objects of Interest:Small Complexity Classes • TC0O(1)-Depth Circuits of MAJ gates • NC1 Log-Depth Circuits • AC0 [2]can’t compute Mod 3 [R,S] • AC0 [2] • AC0 O(1)-Depth Circuits of AND/OR gates

14. The Main Objects of Interest:Small Complexity Classes • NC1 Log-Depth Circuits • TC0O(1)-Depth Circuits of MAJ gates • AC0 [6] • AC0 [2] • AC0 O(1)-Depth Circuits of AND/OR gates

15. The Main Objects of Interest:Small Complexity Classes • NC1 poly-size formulae • TC0O(1)-Depth Circuits of MAJ gates • AC0 [6] • AC0 [2] • AC0 O(1)-Depth Circuits of AND/OR gates

16. Complete Problems • NP has complete sets (under polynomial time reducibility ≤P) • These small classes have complete sets, too (under ≤AC°)

17. Reductions • A ≤AC°B means that there is a constant-depth circuit computing A that has the usual AND and OR gates, and also has ‘oracle gates’ for B. B

18. Complete Problems • NC1 • TC0 • AC0 [6] • AC0 [2] • AC0 • sorting, multiplication, division • [Naor,Reingold] Pseudorandom Generator

19. Complete Problems • NC1 • TC0 • AC0 [6] • AC0 [2] • AC0 • BFE: Balanced Boolean Formula Evaluation (AND,OR,XOR) • Word problem over S5

20. The Word Problem Over S5 • A regular set complete for NC1 =

21. Complexity Classes are not Invented – They’re Discovered • NP (SAT, Clique, TSP,…) • P (Linear Programming, CVP, …) • NL (Connectivity, Shortest Paths, 2SAT, …) • L (Undirected Connectivity, Acyclicity, …) • NC1 (BFE, Regular Sets) • TC0 (Sorting, Multiplication, Division) We’re interested in NC1 (for instance) not because we want to build formulae for these functions…

22. Complexity Classes are not Invented – They’re Discovered • NP (SAT, Clique, TSP,…) • P (Linear Programming, CVP, …) • NL (Connectivity, Shortest Paths, 2SAT, …) • L (Undirected Connectivity, Acyclicity, …) • NC1 (BFE, Regular Sets) • TC0 (Sorting, Multiplication, Division) … but because we want to know if the blocks of this partition are distinct.

23. Complexity Classes are not Invented – They’re Discovered • NP (SAT, Clique, TSP,…) • P (Linear Programming, CVP, …) • NL (Connectivity, Shortest Paths, 2SAT, …) • L (Undirected Connectivity, Acyclicity, …) • NC1 (BFE, Regular Sets) • TC0 (Sorting, Multiplication, Division) These classes are real. They’re important.

24. How far are we in this talk? • We’ve explained why circuit lower bounds are important. • …even for restricted classes of circuits. • What is currently known about these classes?

25. Longstanding Open Problems • Is P = NP? • Is AC0[6] = NP? • Is depth 3 AC0[6] = NP? We’ll focus on questions such as: Is BFE in TC0? Is BFE in AC0[6]?

26. How Close Are We to Proving Circuit Lower Bounds? • Conventional Wisdom: Not Close At All! • No new superpolynomial size lower bounds in over two decades. • Razborov and Rudich: Any “natural” argument proving a lower bound against a circuit class C yields a proof that C can’t compute a pseudorandom function generator. • Since the [Naor, Reingold] generator is computable in TC0, this is bad news.

27. More Modest Goals • Problems requiring formulae of size n3 [Håstad] • Problems requiring branching programs of size nearly n loglog n [Beame, Saks, Sun, Vee] • Problems requiring depth d TC0 circuits of size n1+c [Impagliazzo, Paturi, Saks] • Time-Space Tradeoffs [Fortnow, Lipton, Van Melkebeek, Viglas] • There is little feeling that these results bring us any closer to separating complexity classes.

28. How Close Are We to Proving Circuit Lower Bounds? • How close are the following two statements? • TC0 Circuits for BFE must be of size n1+Ω(1) • For some c>0, TC0 Circuits for BFE must be of size n1+c.

29. How Close Are We to Proving Circuit Lower Bounds? • How close are the following two statements? • TC0 Circuits for BFE must be of size n1+Ω(1) • For some c>0, TC0 Circuits for BFE must be of size n1+c This is known [IPS’97] This implies TC0≠ NC1 [A, Koucky]

30. Self-Reducibility • A set B is said to be “self-reducible” if B≤PB

31. Self-Reducibility • A set B is said to be “self-reducible” if B≤PB via a reduction that, on input x, does not ask about whether x is in B. • Very well-studied notion. • For example, φ is in SAT if and only if (φ0 is in SAT) or(φ1 is in SAT)

32. Self-Reducibility • Many of the important problems in (or near) NC1 have a special self-reducibility property:

33. Self-Reducibility • Many of the important problems in (or near) NC1 have a special self-reducibility property: Instances of length n are AC0-Turing (or TC0-Turing) reducible to instances of length n½ via reductions of linear size. • Examples: • BFE • the word problem over S5 • MAJORITY • Iterated Product of 3-by-3 Integer Matrices

34. Self Reducibility • BFE A subformula near the root Subformulae near inputs

35. Self Reducibility • S5

36. Self Reducibility • The self-reduction of S5, on inputs of size n, uses (n½ + 1) oracle gates of size n½. • Thus if S5 has TC0 circuits of size nk, it also has circuits of size (n½ + 1)nk/2= O(n(k+1)/2). • Similar arguments hold for other classes (such as AC0[6] and NC1). • More complicated self-reductions can be presented for MAJORITY and Iterated Product of 3-by-3 matrices.

37. A Corollary • If BFE has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size. • If S5 has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size. • If MAJ has AC0[6] circuits, then it has such circuits of nearly linear size. (Etc.) • Thus, e.g., to separate NC1 from TC0, it suffices to show that BFE requires TC0 circuits of size n1.0000001.

38. A Corollary • If BFE has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size. • If S5 has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size. • If MAJ has AC0[6] circuits, then it has such circuits of nearly linear size. (Etc.) • How widespread is this phenomenon? Is it true for SAT? (I.e., can we show NP ≠ TC0 by proving that SAT requires TC0 circuits of size n1.0000001?)

39. Limitations of Self-Reducibility • Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC. • Thus there is no obvious way to apply these techniques to SAT or to problems complete for P. • …but perhaps, rather than showing directly that SAT has this strong form of self-reducibility, one can argue that if SAT is in TC0 then it has TC0 circuits of nearly-linear size.

40. Limitations of Self-Reducibility • Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC.

41. Limitations of Self-Reducibility • Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC. d levels of oracle gates

42. Limitations of Self-Reducibility • Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC. d2 levels of oracle gates

43. Limitations of Self-Reducibility • Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC. After log log rounds, the depth is logO(1)n d3 levels of oracle gates

44. Prospects for Progress • We have seen that existing techniques prove bounds that are “nearly” good enough to separate NC1 and TC0. Some of these proofs are “natural”. • Don’t the results of [Razborov & Rudich] indicate that further progress will require very different approaches? • Not necessarily!

45. Prospects for Progress • The [Razborov & Rudich] framework of natural proofs assumes that a “natural” proof of a lower bound will make use of a combinatorial property that (among other things) is shared by a large fraction of the functions on n bits. • In contrast, we are making use of a self-reducibility property that allows us to boost a n1+ε lower bound to a superpolynomial lower bound. This self-reducibility property holds for only a vanishingly small fraction of all functions.

46. Prospects for Progress • These observations are simple, but … • they have forever changed the way that we look at quadratic (and smaller) lower bounds. • We are not claiming to have found a way around the obstacles identified by [Razborov & Rudich]. (Such a claim will have to wait until someone proves that NC1≠ TC0.) But we do believe that this avenue deserves further exploration.

47. Other Avenues for Progress • Diagonalization + Algebraic Tools • The Mulmuley-Sohoni Approach • Lower Bounds via Derandomization

48. Diagonalization • The archtype of a “relativizable” proof technique – unable to prove P ≠ NP, or even NEXP not contained in P/poly. • Non-relativizing proof techniques have been developed, using algebraic techniques that were useful in analyzing interactive and probabilistically checkable proof systems. • These proof techniques “algebrize” [Aaronson, Wigderson], and hence also cannot prove that NEXP is not contained in P/poly.

49. Diagonalization + Algebraic Techniques • There is no evidence that these techniques are unable to prove that NEXP is not contained in TC0. • …but there is also no evidence that they can. • Even “simple” results such as “AC0 can’t compute Mod 2” are not known to be provable using these techniques.

50. Other Avenues for Progress • Diagonalization + Algebraic Tools • The Mulmuley-Sohoni Approach • Lower Bounds via Derandomization