1 / 63

Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds

Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds. DCFS 2008. Today’s Goal:. To raise awareness of the tight connection between circuit complexity and Kolmogorov complexity. And to show that this is useful.

issac
Download Presentation

Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds DCFS 2008

  2. Today’s Goal: • To raise awareness of the tight connection between circuit complexity and Kolmogorov complexity. • And to show that this is useful. • To plant seeds of optimism, regarding the prospects of proving lower bounds in circuit complexity.

  3. Kolmogorov Complexity • C(x) = min{|d| : U(d) = x} • Important property • Invariance: The choice of the universal Turing machine U is unimportant. • x is random if C(x) ≥ |x|. • CA(x) = min{|d| : UA(d) = x}

  4. Circuit Complexity • Let D be a circuit of AND and OR gates (with negations at the inputs). Size(D) = # of wires in D. • Size(f) = min{Size(D) : D computes f} • We may allow oracle gates for a set A, along with AND and OR gates. • SizeA(f) = min{Size(D) : DA computes f}

  5. K-complexity ≈ Circuit Complexity • There are some obvious similarities in the definitions. What are some differences? • A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings. • Given any string x, let fx be the function whose truth table is the string of length 2log|x|, padded out with 0’s, and define Size(x) to be Size(fx).

  6. K-complexity ≈ Circuit Complexity • There are some obvious similarities in the definitions. What are some differences? • A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings. • A more fundamental difference: • C(x) is not computable; Size(x) is. • The Minimum Circuit Size Problem (MCSP): {(x,i) : Size(x) ≤ i}.

  7. MCSP • MCSP is in NP, but is not known to be NP-complete. • MCSP is not believed to be in P. • Factoring is in BPPMCSP. • Every cryptographically-secure one-way function can be inverted in PMCSP/poly.

  8. So how can K-complexity and Circuit complexity be the same? • C(x) ≈ SizeH(x), where H is the halting problem. • For one direction, let U(d) = x. We need a small circuit (with oracle gates for H) for fx, where fx(i) is the i-th bit of x. This is easy, since {(d,i,b) : U(d) outputs a string whose i-th bit is b} is computably-enumerable. • For the other direction, let SizeH(fx) = m. No oracle gate has more than m wires coming into it. Given a description of D (size not much bigger than m) and the m-bit number giving the size of {y in H : |y| ≤ m}, U can simulate DH and produce fx

  9. So how can K-complexity and Circuit complexity be the same? • C(x) ≈ SizeH(x), where H is the halting problem. • So there is a connection between C(x) and Size(x) … • …but is it useful? • First, let’s look at decidable versions of Kolmogorov complexity.

  10. Time-Bounded Kolmogorov Complexity • The usual definition: • Ct(x) = min{|d| : U(d) = x in time t(|d|)}. • Problems with this definition • No invariance! If U and U’ are different universal Turing machines, CtU and CtU’ have no clear relationship. • (One can bound CtU by Ct’U’ for t’ slightly larger than t – but nothing can be done for t’=t.) • No nice connection to circuit complexity!

  11. Time-Bounded Kolmogorov Complexity • Levin’s definition: • Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}. • Invariance holds! If U and U’ are different universal Turing machines, KtU(x) and KtU’(x) are within log |x| of each other. • Let A be complete for E = Dtime(2O(n)). Then Kt(x) ≈ SizeA(x).

  12. Time-Bounded Kolmogorov Complexity • Levin’s definition: • Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}. • Why log t? • This gives an optimal search order for NP search problems. • Adding t instead of log t would give every string complexity ≥ |x|. • …So let’s look at how to make the run-time be much smaller.

  13. Revised Kolmogorov Complexity • C(x) = min{|d| : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x} (where bit # i+1 of x is *). • This is identical to the original definition. • Kt(x) = min{|d|+log t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}. • The new and old definitions are within O(log |x|) of each other. • Define KT(x) = min{|d|+t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}.

  14. Kolmogorov Complexity is Circuit Complexity • C(x) ≈ SizeH(x). • Kt(x) ≈ SizeE(x). • KT(x) ≈ Size(x). • Other measures of complexity can be captured in this way, too: • Branching Program Size ≈ KB(x) = min{|d|+2s : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in space s(|d|)}.

  15. Kolmogorov Complexity is Circuit Complexity • C(x) ≈ SizeH(x). • Kt(x) ≈ SizeE(x). • KT(x) ≈ Size(x). • Other measures of complexity can be captured in this way, too: • Formula Size ≈ KF(x) = min{|d|+2t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.

  16. …but is this interesting? • The result that Factoring is in BPPMCSP was first proved by observing that, in PMCSP, one can accept a large set of strings having large KT complexity (and by making use of many important results in the theory of pseudorandom generators and derandomization). • (Basic Idea): There is a pseudorandom generator based on factoring, such that factoring is in BPPT for any test T that distinguishes truly random strings from pseudorandom strings. MCSP is such a test.

  17. This idea has many variants. • Consider RKT, RKt, and RC. • RKT is in coNP, and not known to be coNP hard. • RC is not hard for NP under poly-time many-one reductions, unless P=NP. • How about more powerful reductions? • Is there anything interesting that we could compute quickly if C were computable, that we can’t already compute quickly? • Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in • But RC is undecidable! Perhaps H is in P relative to RC?

  18. This idea has many variants. • Consider RKT, RKt, and RC. • RKT is in coNP, and not known to be coNP hard. • RC, is not hard for NP under poly-time many-one reductions, unless P=NP. • How about more powerful reductions? • PSPACE is in P relative to RC. • NEXP is in NP relative to RC. • Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in poly time, relative to RC [BFNV]. • But RC is undecidable! Perhaps H is in P relative to RC?

  19. Relationship between H and RC • Perhaps H is in P relative to RC? • This is still open. It is known that there is a computable time bound t such that H is in DTime(t) relative to RC [Kummer]. • …but the bound t depends on the choice of U in the definition of C(x). • We also know that H is in P/poly relative to RC.

  20. This idea has many variants. • Consider RKT, RKt, and RC. • What about RKt? • RKt, is not hard for NP under poly-time many-one reductions, unless E=NE. • How about more powerful reductions? • EXP = NP(RKt). • RKt is complete for EXP under P/poly reductions. • Open if RKt is in P!

  21. Kolmogorov Complexity is Circuit Complexity • C(x) ≈ SizeH(x). • Kt(x) ≈ SizeE(x). • KT(x) ≈ Size(x). • Other measures of complexity can be captured in this way, too: • Formula Size ≈ KF(x) = min{|d|+2t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.

  22. Kolmogorov Complexity is Circuit Complexity • C(x) ≈ SizeH(x). • Kt(x) ≈ SizeE(x). • KT(x) ≈ Size(x). • Other measures of complexity can be captured in this way, too: • A similar definition captures depth k threshold circuit size. • [This is the clever transition to start the discussion of circuit lower bounds…]

  23. Big Complexity Classes • NP • P • . • . • NC • L (Deterministic Logspace)

  24. The Main Objects of Interest:Small Complexity Classes • TC0O(1)-Depth Circuits of MAJ gates • AC0 [6] • NC1 Log-Depth Circuits • AC0 can’t compute Mod 2 [FSS,A] • AC0 O(1)-Depth Circuits of AND/OR gates

  25. The Main Objects of Interest:Small Complexity Classes • TC0O(1)-Depth Circuits of MAJ gates • AC0 [6] • NC1 Log-Depth Circuits • AC0 can’t compute Mod 2 [FSS,A] • AC0 O(1)-Depth Circuits of AND/OR gates

  26. The Main Objects of Interest:Small Complexity Classes • TC0O(1)-Depth Circuits of MAJ gates • NC1 Log-Depth Circuits • AC0 [2]can’t compute Mod 3 [R,S] • AC0 [2] • AC0 O(1)-Depth Circuits of AND/OR gates

  27. The Main Objects of Interest:Small Complexity Classes • NC1 Log-Depth Circuits • TC0O(1)-Depth Circuits of MAJ gates • AC0 [6] • AC0 [2] • AC0 O(1)-Depth Circuits of AND/OR gates

  28. The Main Objects of Interest:Small Complexity Classes • NC1 poly-size formulae • TC0O(1)-Depth Circuits of MAJ gates • AC0 [6] • AC0 [2] • AC0 O(1)-Depth Circuits of AND/OR gates

  29. Complete Problems • NP has complete sets (under polynomial time reducibility ≤P) • These small classes have complete sets, too (under ≤AC°)

  30. Reductions • A ≤AC°B means that there is a constant-depth circuit computing A that has the usual AND and OR gates, and also has ‘oracle gates’ for B. B

  31. Complete Problems • NC1 • TC0 • AC0 [6] • AC0 [2] • AC0 • sorting, multiplication, division • [Naor,Reingold] Pseudorandom Generator

  32. Complete Problems • NC1 • TC0 • AC0 [6] • AC0 [2] • AC0 • BFE: Balanced Boolean Formula Evaluation (AND,OR,XOR) • Word problem over S5

  33. The Word Problem Over S5 • A regular set complete for NC1 =

  34. Complexity Classes are not Invented – They’re Discovered • NP (SAT, Clique, TSP,…) • P (Linear Programming, CVP, …) • NL (Connectivity, Shortest Paths, 2SAT, …) • L (Undirected Connectivity, Acyclicity, …) • NC1 (BFE, Regular Sets) • TC0 (Sorting, Multiplication, Division) We’re interested in NC1 (for instance) not because we want to build formulae for these functions…

  35. Complexity Classes are not Invented – They’re Discovered • NP (SAT, Clique, TSP,…) • P (Linear Programming, CVP, …) • NL (Connectivity, Shortest Paths, 2SAT, …) • L (Undirected Connectivity, Acyclicity, …) • NC1 (BFE, Regular Sets) • TC0 (Sorting, Multiplication, Division) … but because we want to know if the blocks of this partition are distinct.

  36. Complexity Classes are not Invented – They’re Discovered • NP (SAT, Clique, TSP,…) • P (Linear Programming, CVP, …) • NL (Connectivity, Shortest Paths, 2SAT, …) • L (Undirected Connectivity, Acyclicity, …) • NC1 (BFE, Regular Sets) • TC0 (Sorting, Multiplication, Division) These classes are real. They’re important.

  37. Longstanding Open Problems • Is P = NP? • Is AC0[6] = NP? • Is depth 3 AC0[6] = NP? We’ll focus on questions such as: Is BFE in TC0? Is BFE in AC0[6]?

  38. How Close Are We to Proving Circuit Lower Bounds? • Conventional Wisdom: Not Close At All! • No new superpolynomial size lower bounds in over two decades. • Razborov and Rudich: Any “natural” argument proving a lower bound against a circuit class C yields a proof that C can’t compute a pseudorandom function generator. • Since the [Naor, Reingold] generator is computable in TC0, this is bad news.

  39. More Modest Goals • Problems requiring formulae of size n3 [Håstad] • Problems requiring branching programs of size nearly n loglog n [Beame, Saks, Sun, Vee] • Problems requiring depth d TC0 circuits of size n1+c [Impagliazzo, Paturi, Saks] • Time-Space Tradeoffs [Fortnow, Lipton, Van Melkebeek, Viglas] • There is little feeling that these results bring us any closer to separating complexity classes.

  40. How Close Are We to Proving Circuit Lower Bounds? • How close are the following two statements? • TC0 Circuits for BFE must be of size n1+Ω(1) • For some c>0, TC0 Circuits for BFE must be of size n1+c.

  41. How Close Are We to Proving Circuit Lower Bounds? • How close are the following two statements? • TC0 Circuits for BFE must be of size n1+Ω(1) • For some c>0, FTC0 Circuits for BFE must be of size n1+c This is known [IPS’97] This implies TC0≠ NC1 [A, Koucky]

  42. Self-Reducibility • A set B is said to be “self-reducible” if B≤PB

  43. Self-Reducibility • A set B is said to be “self-reducible” if B≤PB via a reduction that, on input x, does not ask about whether x is in B. • Very well-studied notion. • For example, φ is in SAT if and only if (φ0 is in SAT) or(φ1 is in SAT)

  44. Self-Reducibility • Many of the important problems in (or near) NC1 have a special self-reducibility property:

  45. Self-Reducibility • Many of the important problems in (or near) NC1 have a special self-reducibility property: Instances of length n are AC0-Turing (or TC0-Turing) reducible to instances of length n½ via reductions of linear size. • Examples: • BFE • the word problem over S5 • MAJORITY • Iterated Product of 3-by-3 Integer Matrices

  46. Self Reducibility • BFE A subformula near the root Subformulae near inputs

  47. Self Reducibility • S5

  48. Self Reducibility • The self-reduction of S5, on inputs of size n, uses (n½ + 1) oracle gates of size n½. • Thus if S5 has TC0 circuits of size nk, it also has circuits of size (n½ + 1)nk/2= O(n(k+1)/2). • Similar arguments hold for other classes (such as AC0[6] and NC1). • More complicated self-reductions can be presented for MAJORITY and Iterated Product of 3-by-3 matrices.

  49. A Corollary • If BFE has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size. • If S5 has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size. • If MAJ has AC0[6] circuits, then it has such circuits of nearly linear size. (Etc.) • Thus, e.g., to separate NC1 from TC0, it suffices to show that BFE requires TC0 circuits of size n1.0000001.

  50. A Corollary • If BFE has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size. • If S5 has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size. • If MAJ has AC0[6] circuits, then it has such circuits of nearly linear size. (Etc.) • How widespread is this phenomenon? Is it true for SAT? (I.e., can we show NP ≠ TC0 by proving that SAT requires TC0 circuits of size n1.0000001?)

More Related