Download Presentation

Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds

Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds

378 Views

Download Presentation
## Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Circuit Complexity, Kolmogorov Complexity, and Prospects for**Lower Bounds DCFS 2008**Today’s Goal:**• To raise awareness of the tight connection between circuit complexity and Kolmogorov complexity. • And to show that this is useful. • To plant seeds of optimism, regarding the prospects of proving lower bounds in circuit complexity.**Kolmogorov Complexity**• C(x) = min{|d| : U(d) = x} • Important property • Invariance: The choice of the universal Turing machine U is unimportant. • x is random if C(x) ≥ |x|. • CA(x) = min{|d| : UA(d) = x}**Circuit Complexity**• Let D be a circuit of AND and OR gates (with negations at the inputs). Size(D) = # of wires in D. • Size(f) = min{Size(D) : D computes f} • We may allow oracle gates for a set A, along with AND and OR gates. • SizeA(f) = min{Size(D) : DA computes f}**K-complexity ≈ Circuit Complexity**• There are some obvious similarities in the definitions. What are some differences? • A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings. • Given any string x, let fx be the function whose truth table is the string of length 2log|x|, padded out with 0’s, and define Size(x) to be Size(fx).**K-complexity ≈ Circuit Complexity**• There are some obvious similarities in the definitions. What are some differences? • A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings. • A more fundamental difference: • C(x) is not computable; Size(x) is. • The Minimum Circuit Size Problem (MCSP): {(x,i) : Size(x) ≤ i}.**MCSP**• MCSP is in NP, but is not known to be NP-complete. • MCSP is not believed to be in P. • Factoring is in BPPMCSP. • Every cryptographically-secure one-way function can be inverted in PMCSP/poly.**So how can K-complexity and Circuit complexity be the same?**• C(x) ≈ SizeH(x), where H is the halting problem. • For one direction, let U(d) = x. We need a small circuit (with oracle gates for H) for fx, where fx(i) is the i-th bit of x. This is easy, since {(d,i,b) : U(d) outputs a string whose i-th bit is b} is computably-enumerable. • For the other direction, let SizeH(fx) = m. No oracle gate has more than m wires coming into it. Given a description of D (size not much bigger than m) and the m-bit number giving the size of {y in H : |y| ≤ m}, U can simulate DH and produce fx**So how can K-complexity and Circuit complexity be the same?**• C(x) ≈ SizeH(x), where H is the halting problem. • So there is a connection between C(x) and Size(x) … • …but is it useful? • First, let’s look at decidable versions of Kolmogorov complexity.**Time-Bounded Kolmogorov Complexity**• The usual definition: • Ct(x) = min{|d| : U(d) = x in time t(|d|)}. • Problems with this definition • No invariance! If U and U’ are different universal Turing machines, CtU and CtU’ have no clear relationship. • (One can bound CtU by Ct’U’ for t’ slightly larger than t – but nothing can be done for t’=t.) • No nice connection to circuit complexity!**Time-Bounded Kolmogorov Complexity**• Levin’s definition: • Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}. • Invariance holds! If U and U’ are different universal Turing machines, KtU(x) and KtU’(x) are within log |x| of each other. • Let A be complete for E = Dtime(2O(n)). Then Kt(x) ≈ SizeA(x).**Time-Bounded Kolmogorov Complexity**• Levin’s definition: • Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}. • Why log t? • This gives an optimal search order for NP search problems. • Adding t instead of log t would give every string complexity ≥ |x|. • …So let’s look at how to make the run-time be much smaller.**Revised Kolmogorov Complexity**• C(x) = min{|d| : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x} (where bit # i+1 of x is *). • This is identical to the original definition. • Kt(x) = min{|d|+log t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}. • The new and old definitions are within O(log |x|) of each other. • Define KT(x) = min{|d|+t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}.**Kolmogorov Complexity is Circuit Complexity**• C(x) ≈ SizeH(x). • Kt(x) ≈ SizeE(x). • KT(x) ≈ Size(x). • Other measures of complexity can be captured in this way, too: • Branching Program Size ≈ KB(x) = min{|d|+2s : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in space s(|d|)}.**Kolmogorov Complexity is Circuit Complexity**• C(x) ≈ SizeH(x). • Kt(x) ≈ SizeE(x). • KT(x) ≈ Size(x). • Other measures of complexity can be captured in this way, too: • Formula Size ≈ KF(x) = min{|d|+2t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.**…but is this interesting?**• The result that Factoring is in BPPMCSP was first proved by observing that, in PMCSP, one can accept a large set of strings having large KT complexity (and by making use of many important results in the theory of pseudorandom generators and derandomization). • (Basic Idea): There is a pseudorandom generator based on factoring, such that factoring is in BPPT for any test T that distinguishes truly random strings from pseudorandom strings. MCSP is such a test.**This idea has many variants.**• Consider RKT, RKt, and RC. • RKT is in coNP, and not known to be coNP hard. • RC is not hard for NP under poly-time many-one reductions, unless P=NP. • How about more powerful reductions? • Is there anything interesting that we could compute quickly if C were computable, that we can’t already compute quickly? • Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in • But RC is undecidable! Perhaps H is in P relative to RC?**This idea has many variants.**• Consider RKT, RKt, and RC. • RKT is in coNP, and not known to be coNP hard. • RC, is not hard for NP under poly-time many-one reductions, unless P=NP. • How about more powerful reductions? • PSPACE is in P relative to RC. • NEXP is in NP relative to RC. • Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in poly time, relative to RC [BFNV]. • But RC is undecidable! Perhaps H is in P relative to RC?**Relationship between H and RC**• Perhaps H is in P relative to RC? • This is still open. It is known that there is a computable time bound t such that H is in DTime(t) relative to RC [Kummer]. • …but the bound t depends on the choice of U in the definition of C(x). • We also know that H is in P/poly relative to RC.**This idea has many variants.**• Consider RKT, RKt, and RC. • What about RKt? • RKt, is not hard for NP under poly-time many-one reductions, unless E=NE. • How about more powerful reductions? • EXP = NP(RKt). • RKt is complete for EXP under P/poly reductions. • Open if RKt is in P!**Kolmogorov Complexity is Circuit Complexity**• C(x) ≈ SizeH(x). • Kt(x) ≈ SizeE(x). • KT(x) ≈ Size(x). • Other measures of complexity can be captured in this way, too: • Formula Size ≈ KF(x) = min{|d|+2t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.**Kolmogorov Complexity is Circuit Complexity**• C(x) ≈ SizeH(x). • Kt(x) ≈ SizeE(x). • KT(x) ≈ Size(x). • Other measures of complexity can be captured in this way, too: • A similar definition captures depth k threshold circuit size. • [This is the clever transition to start the discussion of circuit lower bounds…]**Big Complexity Classes**• NP • P • . • . • NC • L (Deterministic Logspace)**The Main Objects of Interest:Small Complexity Classes**• TC0O(1)-Depth Circuits of MAJ gates • AC0 [6] • NC1 Log-Depth Circuits • AC0 can’t compute Mod 2 [FSS,A] • AC0 O(1)-Depth Circuits of AND/OR gates**The Main Objects of Interest:Small Complexity Classes**• TC0O(1)-Depth Circuits of MAJ gates • AC0 [6] • NC1 Log-Depth Circuits • AC0 can’t compute Mod 2 [FSS,A] • AC0 O(1)-Depth Circuits of AND/OR gates**The Main Objects of Interest:Small Complexity Classes**• TC0O(1)-Depth Circuits of MAJ gates • NC1 Log-Depth Circuits • AC0 [2]can’t compute Mod 3 [R,S] • AC0 [2] • AC0 O(1)-Depth Circuits of AND/OR gates**The Main Objects of Interest:Small Complexity Classes**• NC1 Log-Depth Circuits • TC0O(1)-Depth Circuits of MAJ gates • AC0 [6] • AC0 [2] • AC0 O(1)-Depth Circuits of AND/OR gates**The Main Objects of Interest:Small Complexity Classes**• NC1 poly-size formulae • TC0O(1)-Depth Circuits of MAJ gates • AC0 [6] • AC0 [2] • AC0 O(1)-Depth Circuits of AND/OR gates**Complete Problems**• NP has complete sets (under polynomial time reducibility ≤P) • These small classes have complete sets, too (under ≤AC°)**Reductions**• A ≤AC°B means that there is a constant-depth circuit computing A that has the usual AND and OR gates, and also has ‘oracle gates’ for B. B**Complete Problems**• NC1 • TC0 • AC0 [6] • AC0 [2] • AC0 • sorting, multiplication, division • [Naor,Reingold] Pseudorandom Generator**Complete Problems**• NC1 • TC0 • AC0 [6] • AC0 [2] • AC0 • BFE: Balanced Boolean Formula Evaluation (AND,OR,XOR) • Word problem over S5**The Word Problem Over S5**• A regular set complete for NC1 =**Complexity Classes are not Invented – They’re Discovered**• NP (SAT, Clique, TSP,…) • P (Linear Programming, CVP, …) • NL (Connectivity, Shortest Paths, 2SAT, …) • L (Undirected Connectivity, Acyclicity, …) • NC1 (BFE, Regular Sets) • TC0 (Sorting, Multiplication, Division) We’re interested in NC1 (for instance) not because we want to build formulae for these functions…**Complexity Classes are not Invented – They’re Discovered**• NP (SAT, Clique, TSP,…) • P (Linear Programming, CVP, …) • NL (Connectivity, Shortest Paths, 2SAT, …) • L (Undirected Connectivity, Acyclicity, …) • NC1 (BFE, Regular Sets) • TC0 (Sorting, Multiplication, Division) … but because we want to know if the blocks of this partition are distinct.**Complexity Classes are not Invented – They’re Discovered**• NP (SAT, Clique, TSP,…) • P (Linear Programming, CVP, …) • NL (Connectivity, Shortest Paths, 2SAT, …) • L (Undirected Connectivity, Acyclicity, …) • NC1 (BFE, Regular Sets) • TC0 (Sorting, Multiplication, Division) These classes are real. They’re important.**Longstanding Open Problems**• Is P = NP? • Is AC0[6] = NP? • Is depth 3 AC0[6] = NP? We’ll focus on questions such as: Is BFE in TC0? Is BFE in AC0[6]?**How Close Are We to Proving Circuit Lower Bounds?**• Conventional Wisdom: Not Close At All! • No new superpolynomial size lower bounds in over two decades. • Razborov and Rudich: Any “natural” argument proving a lower bound against a circuit class C yields a proof that C can’t compute a pseudorandom function generator. • Since the [Naor, Reingold] generator is computable in TC0, this is bad news.**More Modest Goals**• Problems requiring formulae of size n3 [Håstad] • Problems requiring branching programs of size nearly n loglog n [Beame, Saks, Sun, Vee] • Problems requiring depth d TC0 circuits of size n1+c [Impagliazzo, Paturi, Saks] • Time-Space Tradeoffs [Fortnow, Lipton, Van Melkebeek, Viglas] • There is little feeling that these results bring us any closer to separating complexity classes.**How Close Are We to Proving Circuit Lower Bounds?**• How close are the following two statements? • TC0 Circuits for BFE must be of size n1+Ω(1) • For some c>0, TC0 Circuits for BFE must be of size n1+c.**How Close Are We to Proving Circuit Lower Bounds?**• How close are the following two statements? • TC0 Circuits for BFE must be of size n1+Ω(1) • For some c>0, FTC0 Circuits for BFE must be of size n1+c This is known [IPS’97] This implies TC0≠ NC1 [A, Koucky]**Self-Reducibility**• A set B is said to be “self-reducible” if B≤PB**Self-Reducibility**• A set B is said to be “self-reducible” if B≤PB via a reduction that, on input x, does not ask about whether x is in B. • Very well-studied notion. • For example, φ is in SAT if and only if (φ0 is in SAT) or(φ1 is in SAT)**Self-Reducibility**• Many of the important problems in (or near) NC1 have a special self-reducibility property:**Self-Reducibility**• Many of the important problems in (or near) NC1 have a special self-reducibility property: Instances of length n are AC0-Turing (or TC0-Turing) reducible to instances of length n½ via reductions of linear size. • Examples: • BFE • the word problem over S5 • MAJORITY • Iterated Product of 3-by-3 Integer Matrices**Self Reducibility**• BFE A subformula near the root Subformulae near inputs**Self Reducibility**• S5**Self Reducibility**• The self-reduction of S5, on inputs of size n, uses (n½ + 1) oracle gates of size n½. • Thus if S5 has TC0 circuits of size nk, it also has circuits of size (n½ + 1)nk/2= O(n(k+1)/2). • Similar arguments hold for other classes (such as AC0[6] and NC1). • More complicated self-reductions can be presented for MAJORITY and Iterated Product of 3-by-3 matrices.**A Corollary**• If BFE has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size. • If S5 has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size. • If MAJ has AC0[6] circuits, then it has such circuits of nearly linear size. (Etc.) • Thus, e.g., to separate NC1 from TC0, it suffices to show that BFE requires TC0 circuits of size n1.0000001.**A Corollary**• If BFE has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size. • If S5 has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size. • If MAJ has AC0[6] circuits, then it has such circuits of nearly linear size. (Etc.) • How widespread is this phenomenon? Is it true for SAT? (I.e., can we show NP ≠ TC0 by proving that SAT requires TC0 circuits of size n1.0000001?)