Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds

1 / 63

# Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds - PowerPoint PPT Presentation

Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds. DCFS 2008. Today’s Goal:. To raise awareness of the tight connection between circuit complexity and Kolmogorov complexity. And to show that this is useful.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds' - issac

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds

DCFS 2008

Today’s Goal:
• To raise awareness of the tight connection between circuit complexity and Kolmogorov complexity.
• And to show that this is useful.
• To plant seeds of optimism, regarding the prospects of proving lower bounds in circuit complexity.
Kolmogorov Complexity
• C(x) = min{|d| : U(d) = x}
• Important property
• Invariance: The choice of the universal Turing machine U is unimportant.
• x is random if C(x) ≥ |x|.
• CA(x) = min{|d| : UA(d) = x}
Circuit Complexity
• Let D be a circuit of AND and OR gates (with negations at the inputs). Size(D) = # of wires in D.
• Size(f) = min{Size(D) : D computes f}
• We may allow oracle gates for a set A, along with AND and OR gates.
• SizeA(f) = min{Size(D) : DA computes f}
K-complexity ≈ Circuit Complexity
• There are some obvious similarities in the definitions. What are some differences?
• A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings.
• Given any string x, let fx be the function whose truth table is the string of length 2log|x|, padded out with 0’s, and define Size(x) to be Size(fx).
K-complexity ≈ Circuit Complexity
• There are some obvious similarities in the definitions. What are some differences?
• A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings.
• A more fundamental difference:
• C(x) is not computable; Size(x) is.
• The Minimum Circuit Size Problem (MCSP): {(x,i) : Size(x) ≤ i}.
MCSP
• MCSP is in NP, but is not known to be NP-complete.
• MCSP is not believed to be in P.
• Factoring is in BPPMCSP.
• Every cryptographically-secure one-way function can be inverted in PMCSP/poly.
So how can K-complexity and Circuit complexity be the same?
• C(x) ≈ SizeH(x), where H is the halting problem.
• For one direction, let U(d) = x. We need a small circuit (with oracle gates for H) for fx, where fx(i) is the i-th bit of x. This is easy, since {(d,i,b) : U(d) outputs a string whose i-th bit is b} is computably-enumerable.
• For the other direction, let SizeH(fx) = m. No oracle gate has more than m wires coming into it. Given a description of D (size not much bigger than m) and the m-bit number giving the size of {y in H : |y| ≤ m}, U can simulate DH and produce fx
So how can K-complexity and Circuit complexity be the same?
• C(x) ≈ SizeH(x), where H is the halting problem.
• So there is a connection between C(x) and Size(x) …
• …but is it useful?
• First, let’s look at decidable versions of Kolmogorov complexity.
Time-Bounded Kolmogorov Complexity
• The usual definition:
• Ct(x) = min{|d| : U(d) = x in time t(|d|)}.
• Problems with this definition
• No invariance! If U and U’ are different universal Turing machines, CtU and CtU’ have no clear relationship.
• (One can bound CtU by Ct’U’ for t’ slightly larger than t – but nothing can be done for t’=t.)
• No nice connection to circuit complexity!
Time-Bounded Kolmogorov Complexity
• Levin’s definition:
• Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}.
• Invariance holds! If U and U’ are different universal Turing machines, KtU(x) and KtU’(x) are within log |x| of each other.
• Let A be complete for E = Dtime(2O(n)). Then Kt(x) ≈ SizeA(x).
Time-Bounded Kolmogorov Complexity
• Levin’s definition:
• Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}.
• Why log t?
• This gives an optimal search order for NP search problems.
• Adding t instead of log t would give every string complexity ≥ |x|.
• …So let’s look at how to make the run-time be much smaller.
Revised Kolmogorov Complexity
• C(x) = min{|d| : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x} (where bit # i+1 of x is *).
• This is identical to the original definition.
• Kt(x) = min{|d|+log t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}.
• The new and old definitions are within O(log |x|) of each other.
• Define KT(x) = min{|d|+t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}.
Kolmogorov Complexity is Circuit Complexity
• C(x) ≈ SizeH(x).
• Kt(x) ≈ SizeE(x).
• KT(x) ≈ Size(x).
• Other measures of complexity can be captured in this way, too:
• Branching Program Size ≈ KB(x) = min{|d|+2s : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in space s(|d|)}.
Kolmogorov Complexity is Circuit Complexity
• C(x) ≈ SizeH(x).
• Kt(x) ≈ SizeE(x).
• KT(x) ≈ Size(x).
• Other measures of complexity can be captured in this way, too:
• Formula Size ≈ KF(x) = min{|d|+2t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.
…but is this interesting?
• The result that Factoring is in BPPMCSP was first proved by observing that, in PMCSP, one can accept a large set of strings having large KT complexity (and by making use of many important results in the theory of pseudorandom generators and derandomization).
• (Basic Idea): There is a pseudorandom generator based on factoring, such that factoring is in BPPT for any test T that distinguishes truly random strings from pseudorandom strings. MCSP is such a test.
This idea has many variants.
• Consider RKT, RKt, and RC.
• RKT is in coNP, and not known to be coNP hard.
• RC is not hard for NP under poly-time many-one reductions, unless P=NP.
• How about more powerful reductions?
• Is there anything interesting that we could compute quickly if C were computable, that we can’t already compute quickly?
• Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in
• But RC is undecidable! Perhaps H is in P relative to RC?
This idea has many variants.
• Consider RKT, RKt, and RC.
• RKT is in coNP, and not known to be coNP hard.
• RC, is not hard for NP under poly-time many-one reductions, unless P=NP.
• How about more powerful reductions?
• PSPACE is in P relative to RC.
• NEXP is in NP relative to RC.
• Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in poly time, relative to RC [BFNV].
• But RC is undecidable! Perhaps H is in P relative to RC?
Relationship between H and RC
• Perhaps H is in P relative to RC?
• This is still open. It is known that there is a computable time bound t such that H is in DTime(t) relative to RC [Kummer].
• …but the bound t depends on the choice of U in the definition of C(x).
• We also know that H is in P/poly relative to RC.
This idea has many variants.
• Consider RKT, RKt, and RC.
• RKt, is not hard for NP under poly-time many-one reductions, unless E=NE.
• How about more powerful reductions?
• EXP = NP(RKt).
• RKt is complete for EXP under P/poly reductions.
• Open if RKt is in P!
Kolmogorov Complexity is Circuit Complexity
• C(x) ≈ SizeH(x).
• Kt(x) ≈ SizeE(x).
• KT(x) ≈ Size(x).
• Other measures of complexity can be captured in this way, too:
• Formula Size ≈ KF(x) = min{|d|+2t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.
Kolmogorov Complexity is Circuit Complexity
• C(x) ≈ SizeH(x).
• Kt(x) ≈ SizeE(x).
• KT(x) ≈ Size(x).
• Other measures of complexity can be captured in this way, too:
• A similar definition captures depth k threshold circuit size.
• [This is the clever transition to start the discussion of circuit lower bounds…]
Big Complexity Classes
• NP
• P
• .
• .
• NC
• L (Deterministic Logspace)
The Main Objects of Interest:Small Complexity Classes
• TC0O(1)-Depth Circuits of MAJ gates
• AC0 [6]
• NC1 Log-Depth Circuits
• AC0 can’t compute Mod 2 [FSS,A]
• AC0 O(1)-Depth Circuits of AND/OR gates
The Main Objects of Interest:Small Complexity Classes
• TC0O(1)-Depth Circuits of MAJ gates
• AC0 [6]
• NC1 Log-Depth Circuits
• AC0 can’t compute Mod 2 [FSS,A]
• AC0 O(1)-Depth Circuits of AND/OR gates
The Main Objects of Interest:Small Complexity Classes
• TC0O(1)-Depth Circuits of MAJ gates
• NC1 Log-Depth Circuits
• AC0 [2]can’t compute Mod 3 [R,S]
• AC0 [2]
• AC0 O(1)-Depth Circuits of AND/OR gates
The Main Objects of Interest:Small Complexity Classes
• NC1 Log-Depth Circuits
• TC0O(1)-Depth Circuits of MAJ gates
• AC0 [6]
• AC0 [2]
• AC0 O(1)-Depth Circuits of AND/OR gates
The Main Objects of Interest:Small Complexity Classes
• NC1 poly-size formulae
• TC0O(1)-Depth Circuits of MAJ gates
• AC0 [6]
• AC0 [2]
• AC0 O(1)-Depth Circuits of AND/OR gates
Complete Problems
• NP has complete sets (under polynomial time reducibility ≤P)
• These small classes have complete sets, too (under ≤AC°)
Reductions
• A ≤AC°B means that there is a constant-depth circuit computing A that has the usual AND and OR gates, and also has ‘oracle gates’ for B.

B

Complete Problems
• NC1
• TC0
• AC0 [6]
• AC0 [2]
• AC0
• sorting, multiplication, division
• [Naor,Reingold] Pseudorandom Generator
Complete Problems
• NC1
• TC0
• AC0 [6]
• AC0 [2]
• AC0
• BFE: Balanced Boolean Formula Evaluation (AND,OR,XOR)
• Word problem over S5
The Word Problem Over S5
• A regular set complete for NC1

=

Complexity Classes are not Invented – They’re Discovered
• NP (SAT, Clique, TSP,…)
• P (Linear Programming, CVP, …)
• NL (Connectivity, Shortest Paths, 2SAT, …)
• L (Undirected Connectivity, Acyclicity, …)
• NC1 (BFE, Regular Sets)
• TC0 (Sorting, Multiplication, Division)

We’re interested in NC1 (for instance) not because we want to build formulae for these functions…

Complexity Classes are not Invented – They’re Discovered
• NP (SAT, Clique, TSP,…)
• P (Linear Programming, CVP, …)
• NL (Connectivity, Shortest Paths, 2SAT, …)
• L (Undirected Connectivity, Acyclicity, …)
• NC1 (BFE, Regular Sets)
• TC0 (Sorting, Multiplication, Division)

… but because we want to know if the blocks of this partition are distinct.

Complexity Classes are not Invented – They’re Discovered
• NP (SAT, Clique, TSP,…)
• P (Linear Programming, CVP, …)
• NL (Connectivity, Shortest Paths, 2SAT, …)
• L (Undirected Connectivity, Acyclicity, …)
• NC1 (BFE, Regular Sets)
• TC0 (Sorting, Multiplication, Division)

These classes are real.

They’re important.

Longstanding Open Problems
• Is P = NP?
• Is AC0[6] = NP?
• Is depth 3 AC0[6] = NP?

We’ll focus on questions such as:

Is BFE in TC0?

Is BFE in AC0[6]?

How Close Are We to Proving Circuit Lower Bounds?
• Conventional Wisdom: Not Close At All!
• No new superpolynomial size lower bounds in over two decades.
• Razborov and Rudich: Any “natural” argument proving a lower bound against a circuit class C yields a proof that C can’t compute a pseudorandom function generator.
• Since the [Naor, Reingold] generator is computable in TC0, this is bad news.
More Modest Goals
• Problems requiring formulae of size n3 [Håstad]
• Problems requiring branching programs of size nearly n loglog n [Beame, Saks, Sun, Vee]
• Problems requiring depth d TC0 circuits of size n1+c [Impagliazzo, Paturi, Saks]
• Time-Space Tradeoffs [Fortnow, Lipton, Van Melkebeek, Viglas]
• There is little feeling that these results bring us any closer to separating complexity classes.
How Close Are We to Proving Circuit Lower Bounds?
• How close are the following two statements?
• TC0 Circuits for BFE must be of size n1+Ω(1)
• For some c>0, TC0 Circuits for BFE must be of size n1+c.
How Close Are We to Proving Circuit Lower Bounds?
• How close are the following two statements?
• TC0 Circuits for BFE must be of size n1+Ω(1)
• For some c>0, FTC0 Circuits for BFE must be of size n1+c

This is known [IPS’97]

This implies TC0≠ NC1 [A, Koucky]

Self-Reducibility
• A set B is said to be “self-reducible” if B≤PB
Self-Reducibility
• A set B is said to be “self-reducible” if B≤PB via a reduction that, on input x, does not ask about whether x is in B.
• Very well-studied notion.
• For example, φ is in SAT if and only if (φ0 is in SAT) or(φ1 is in SAT)
Self-Reducibility
• Many of the important problems in (or near) NC1 have a special self-reducibility property:
Self-Reducibility
• Many of the important problems in (or near) NC1 have a special self-reducibility property: Instances of length n are AC0-Turing (or TC0-Turing) reducible to instances of length n½ via reductions of linear size.
• Examples:
• BFE
• the word problem over S5
• MAJORITY
• Iterated Product of 3-by-3 Integer Matrices
Self Reducibility
• BFE

A subformula near the root

Subformulae near inputs

Self Reducibility
• The self-reduction of S5, on inputs of size n, uses (n½ + 1) oracle gates of size n½.
• Thus if S5 has TC0 circuits of size nk, it also has circuits of size (n½ + 1)nk/2= O(n(k+1)/2).
• Similar arguments hold for other classes (such as AC0[6] and NC1).
• More complicated self-reductions can be presented for MAJORITY and Iterated Product of 3-by-3 matrices.
A Corollary
• If BFE has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.
• If S5 has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.
• If MAJ has AC0[6] circuits, then it has such circuits of nearly linear size. (Etc.)
• Thus, e.g., to separate NC1 from TC0, it suffices to show that BFE requires TC0 circuits of size n1.0000001.
A Corollary
• If BFE has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.
• If S5 has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.
• If MAJ has AC0[6] circuits, then it has such circuits of nearly linear size. (Etc.)
• How widespread is this phenomenon? Is it true for SAT? (I.e., can we show NP ≠ TC0 by proving that SAT requires TC0 circuits of size n1.0000001?)
Limitations of Self-Reducibility
• Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC.
• Thus there is no obvious way to apply these techniques to SAT or to problems complete for P.
• …but perhaps, rather than showing directly that SAT has this strong form of self-reducibility, one can argue that if SAT is in TC0 then it has TC0 circuits of nearly-linear size.
Limitations of Self-Reducibility
• Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC.
Limitations of Self-Reducibility
• Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC.

d levels of oracle gates

Limitations of Self-Reducibility
• Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC.

d2 levels of oracle gates

Limitations of Self-Reducibility
• Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC.

After log log rounds,

the depth is logO(1)n

d3 levels of oracle gates

Prospects for Progress
• We have seen that existing techniques prove bounds that are “nearly” good enough to separate NC1 and TC0. Some of these proofs are “natural”.
• Don’t the results of [Razborov & Rudich] indicate that further progress will require very different approaches?
• Not necessarily!
Prospects for Progress
• The [Razborov & Rudich] framework of natural proofs assumes that a “natural” proof of a lower bound will make use of a combinatorial property that (among other things) is shared by a large fraction of the functions on n bits.
• In contrast, we are making use of a self-reducibility property that allows us to boost a n1+ε lower bound to a superpolynomial lower bound. This self-reducibility property holds for only a vanishingly small fraction of all functions.
Prospects for Progress
• These observations are simple, but …
• they have forever changed the way that we look at quadratic (and smaller) lower bounds.
• We are not claiming to have found a way around the obstacles identified by [Razborov & Rudich]. (Such a claim will have to wait until someone proves that NC1≠ TC0.) But we do believe that this avenue deserves further exploration.
Conclusion
• There are good reasons to develop and explore the connections between Kolmogorov complexity and circuit complexity.
• There are many open problems in this area that I will be delighted to discuss with you in more detail.
• There are two bad typos in the proceedings version of the paper. (“P” should be “NP”.) A corrected version is available at my home page.
Speculation

Connections between Kolmogorov Complexity and Circuit Complexity might be relevant to the question of whether NEXP is contained in (non-uniform) TC0 (depth 3).

Speculation
• [IKW] showed that NEXP is in P/poly iff NEXP = MA iff MA cannot be derandomized
• The proof shows that NEXP is in P/poly iff every set in P contains strings of KT-complexity O(log n) iff NEXP = IP[P/poly].
Speculation

Similar techniques show:

• NEXP is in nonuniform NC1 iff every set in P contains strings of KF-complexity O(log n) iff NEXP = MIPNC1 iff MIPNC1 cannot be derandomized.
• NEXP is in nonuniform TC0 iff every set in P contains strings of small complexity iff NEXP = MIPTC0 iff MIPTC0 cannot be derandomized.
Speculation

What else happens in such a collapse?

• If NP = uniform TC0, then #P is not contained in non-uniform TC0 (so NEXP is not in non-uniform TC0).
• So let’s consider NEXP = MIPTC0 and NP ≠ uniform TC0. If this “hardness assumption” were sufficient to “derandomize” MIPTC0 then this would give the desired lower bound on NEXP…
• [Fortnow, Klivans], [van Melkebeek, Santhanam]