Loading in 2 Seconds...

Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds

Loading in 2 Seconds...

- By
**issac** - Follow User

- 350 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about 'Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds' - issac

Download Now**An Image/Link below is provided (as is) to download presentation**

Download Now

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds

DCFS 2008

Today’s Goal:

- To raise awareness of the tight connection between circuit complexity and Kolmogorov complexity.
- And to show that this is useful.
- To plant seeds of optimism, regarding the prospects of proving lower bounds in circuit complexity.

Kolmogorov Complexity

- C(x) = min{|d| : U(d) = x}
- Important property
- Invariance: The choice of the universal Turing machine U is unimportant.
- x is random if C(x) ≥ |x|.
- CA(x) = min{|d| : UA(d) = x}

Circuit Complexity

- Let D be a circuit of AND and OR gates (with negations at the inputs). Size(D) = # of wires in D.
- Size(f) = min{Size(D) : D computes f}
- We may allow oracle gates for a set A, along with AND and OR gates.
- SizeA(f) = min{Size(D) : DA computes f}

K-complexity ≈ Circuit Complexity

- There are some obvious similarities in the definitions. What are some differences?
- A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings.
- Given any string x, let fx be the function whose truth table is the string of length 2log|x|, padded out with 0’s, and define Size(x) to be Size(fx).

K-complexity ≈ Circuit Complexity

- There are some obvious similarities in the definitions. What are some differences?
- A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings.
- A more fundamental difference:
- C(x) is not computable; Size(x) is.
- The Minimum Circuit Size Problem (MCSP): {(x,i) : Size(x) ≤ i}.

MCSP

- MCSP is in NP, but is not known to be NP-complete.
- MCSP is not believed to be in P.
- Factoring is in BPPMCSP.
- Every cryptographically-secure one-way function can be inverted in PMCSP/poly.

So how can K-complexity and Circuit complexity be the same?

- C(x) ≈ SizeH(x), where H is the halting problem.
- For one direction, let U(d) = x. We need a small circuit (with oracle gates for H) for fx, where fx(i) is the i-th bit of x. This is easy, since {(d,i,b) : U(d) outputs a string whose i-th bit is b} is computably-enumerable.
- For the other direction, let SizeH(fx) = m. No oracle gate has more than m wires coming into it. Given a description of D (size not much bigger than m) and the m-bit number giving the size of {y in H : |y| ≤ m}, U can simulate DH and produce fx

So how can K-complexity and Circuit complexity be the same?

- C(x) ≈ SizeH(x), where H is the halting problem.
- So there is a connection between C(x) and Size(x) …
- …but is it useful?
- First, let’s look at decidable versions of Kolmogorov complexity.

Time-Bounded Kolmogorov Complexity

- The usual definition:
- Ct(x) = min{|d| : U(d) = x in time t(|d|)}.
- Problems with this definition
- No invariance! If U and U’ are different universal Turing machines, CtU and CtU’ have no clear relationship.
- (One can bound CtU by Ct’U’ for t’ slightly larger than t – but nothing can be done for t’=t.)
- No nice connection to circuit complexity!

Time-Bounded Kolmogorov Complexity

- Levin’s definition:
- Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}.
- Invariance holds! If U and U’ are different universal Turing machines, KtU(x) and KtU’(x) are within log |x| of each other.
- Let A be complete for E = Dtime(2O(n)). Then Kt(x) ≈ SizeA(x).

Time-Bounded Kolmogorov Complexity

- Levin’s definition:
- Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}.
- Why log t?
- This gives an optimal search order for NP search problems.
- Adding t instead of log t would give every string complexity ≥ |x|.
- …So let’s look at how to make the run-time be much smaller.

Revised Kolmogorov Complexity

- C(x) = min{|d| : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x} (where bit # i+1 of x is *).
- This is identical to the original definition.
- Kt(x) = min{|d|+log t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}.
- The new and old definitions are within O(log |x|) of each other.
- Define KT(x) = min{|d|+t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}.

Kolmogorov Complexity is Circuit Complexity

- C(x) ≈ SizeH(x).
- Kt(x) ≈ SizeE(x).
- KT(x) ≈ Size(x).
- Other measures of complexity can be captured in this way, too:
- Branching Program Size ≈ KB(x) = min{|d|+2s : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in space s(|d|)}.

Kolmogorov Complexity is Circuit Complexity

- C(x) ≈ SizeH(x).
- Kt(x) ≈ SizeE(x).
- KT(x) ≈ Size(x).
- Other measures of complexity can be captured in this way, too:
- Formula Size ≈ KF(x) = min{|d|+2t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.

…but is this interesting?

- The result that Factoring is in BPPMCSP was first proved by observing that, in PMCSP, one can accept a large set of strings having large KT complexity (and by making use of many important results in the theory of pseudorandom generators and derandomization).
- (Basic Idea): There is a pseudorandom generator based on factoring, such that factoring is in BPPT for any test T that distinguishes truly random strings from pseudorandom strings. MCSP is such a test.

This idea has many variants.

- Consider RKT, RKt, and RC.
- RKT is in coNP, and not known to be coNP hard.
- RC is not hard for NP under poly-time many-one reductions, unless P=NP.
- How about more powerful reductions?
- Is there anything interesting that we could compute quickly if C were computable, that we can’t already compute quickly?
- Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in
- But RC is undecidable! Perhaps H is in P relative to RC?

This idea has many variants.

- Consider RKT, RKt, and RC.
- RKT is in coNP, and not known to be coNP hard.
- RC, is not hard for NP under poly-time many-one reductions, unless P=NP.
- How about more powerful reductions?
- PSPACE is in P relative to RC.
- NEXP is in NP relative to RC.
- Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in poly time, relative to RC [BFNV].
- But RC is undecidable! Perhaps H is in P relative to RC?

Relationship between H and RC

- Perhaps H is in P relative to RC?
- This is still open. It is known that there is a computable time bound t such that H is in DTime(t) relative to RC [Kummer].
- …but the bound t depends on the choice of U in the definition of C(x).
- We also know that H is in P/poly relative to RC.

This idea has many variants.

- Consider RKT, RKt, and RC.
- What about RKt?
- RKt, is not hard for NP under poly-time many-one reductions, unless E=NE.
- How about more powerful reductions?
- EXP = NP(RKt).
- RKt is complete for EXP under P/poly reductions.
- Open if RKt is in P!

Kolmogorov Complexity is Circuit Complexity

- C(x) ≈ SizeH(x).
- Kt(x) ≈ SizeE(x).
- KT(x) ≈ Size(x).
- Other measures of complexity can be captured in this way, too:
- Formula Size ≈ KF(x) = min{|d|+2t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.

Kolmogorov Complexity is Circuit Complexity

- C(x) ≈ SizeH(x).
- Kt(x) ≈ SizeE(x).
- KT(x) ≈ Size(x).
- Other measures of complexity can be captured in this way, too:
- A similar definition captures depth k threshold circuit size.
- [This is the clever transition to start the discussion of circuit lower bounds…]

Big Complexity Classes

- NP
- P
- .
- .
- NC
- L (Deterministic Logspace)

The Main Objects of Interest:Small Complexity Classes

- TC0O(1)-Depth Circuits of MAJ gates
- AC0 [6]
- NC1 Log-Depth Circuits
- AC0 can’t compute Mod 2 [FSS,A]
- AC0 O(1)-Depth Circuits of AND/OR gates

The Main Objects of Interest:Small Complexity Classes

- TC0O(1)-Depth Circuits of MAJ gates
- AC0 [6]
- NC1 Log-Depth Circuits
- AC0 can’t compute Mod 2 [FSS,A]
- AC0 O(1)-Depth Circuits of AND/OR gates

The Main Objects of Interest:Small Complexity Classes

- TC0O(1)-Depth Circuits of MAJ gates
- NC1 Log-Depth Circuits
- AC0 [2]can’t compute Mod 3 [R,S]
- AC0 [2]
- AC0 O(1)-Depth Circuits of AND/OR gates

The Main Objects of Interest:Small Complexity Classes

- NC1 Log-Depth Circuits
- TC0O(1)-Depth Circuits of MAJ gates
- AC0 [6]
- AC0 [2]
- AC0 O(1)-Depth Circuits of AND/OR gates

The Main Objects of Interest:Small Complexity Classes

- NC1 poly-size formulae
- TC0O(1)-Depth Circuits of MAJ gates
- AC0 [6]
- AC0 [2]
- AC0 O(1)-Depth Circuits of AND/OR gates

Complete Problems

- NP has complete sets (under polynomial time reducibility ≤P)
- These small classes have complete sets, too (under ≤AC°)

Reductions

- A ≤AC°B means that there is a constant-depth circuit computing A that has the usual AND and OR gates, and also has ‘oracle gates’ for B.

B

Complete Problems

- NC1
- TC0
- AC0 [6]
- AC0 [2]
- AC0

- sorting, multiplication, division
- [Naor,Reingold] Pseudorandom Generator

Complete Problems

- NC1
- TC0
- AC0 [6]
- AC0 [2]
- AC0

- BFE: Balanced Boolean Formula Evaluation (AND,OR,XOR)
- Word problem over S5

Complexity Classes are not Invented – They’re Discovered

- NP (SAT, Clique, TSP,…)
- P (Linear Programming, CVP, …)
- NL (Connectivity, Shortest Paths, 2SAT, …)
- L (Undirected Connectivity, Acyclicity, …)
- NC1 (BFE, Regular Sets)
- TC0 (Sorting, Multiplication, Division)

We’re interested in NC1 (for instance) not because we want to build formulae for these functions…

Complexity Classes are not Invented – They’re Discovered

- NP (SAT, Clique, TSP,…)
- P (Linear Programming, CVP, …)
- NL (Connectivity, Shortest Paths, 2SAT, …)
- L (Undirected Connectivity, Acyclicity, …)
- NC1 (BFE, Regular Sets)
- TC0 (Sorting, Multiplication, Division)

… but because we want to know if the blocks of this partition are distinct.

Complexity Classes are not Invented – They’re Discovered

- NP (SAT, Clique, TSP,…)
- P (Linear Programming, CVP, …)
- NL (Connectivity, Shortest Paths, 2SAT, …)
- L (Undirected Connectivity, Acyclicity, …)
- NC1 (BFE, Regular Sets)
- TC0 (Sorting, Multiplication, Division)

These classes are real.

They’re important.

Longstanding Open Problems

- Is P = NP?
- Is AC0[6] = NP?
- Is depth 3 AC0[6] = NP?

We’ll focus on questions such as:

Is BFE in TC0?

Is BFE in AC0[6]?

How Close Are We to Proving Circuit Lower Bounds?

- Conventional Wisdom: Not Close At All!
- No new superpolynomial size lower bounds in over two decades.
- Razborov and Rudich: Any “natural” argument proving a lower bound against a circuit class C yields a proof that C can’t compute a pseudorandom function generator.
- Since the [Naor, Reingold] generator is computable in TC0, this is bad news.

More Modest Goals

- Problems requiring formulae of size n3 [Håstad]
- Problems requiring branching programs of size nearly n loglog n [Beame, Saks, Sun, Vee]
- Problems requiring depth d TC0 circuits of size n1+c [Impagliazzo, Paturi, Saks]
- Time-Space Tradeoffs [Fortnow, Lipton, Van Melkebeek, Viglas]
- There is little feeling that these results bring us any closer to separating complexity classes.

How Close Are We to Proving Circuit Lower Bounds?

- How close are the following two statements?
- TC0 Circuits for BFE must be of size n1+Ω(1)
- For some c>0, TC0 Circuits for BFE must be of size n1+c.

How Close Are We to Proving Circuit Lower Bounds?

- How close are the following two statements?
- TC0 Circuits for BFE must be of size n1+Ω(1)
- For some c>0, FTC0 Circuits for BFE must be of size n1+c

This is known [IPS’97]

This implies TC0≠ NC1 [A, Koucky]

Self-Reducibility

- A set B is said to be “self-reducible” if B≤PB

Self-Reducibility

- A set B is said to be “self-reducible” if B≤PB via a reduction that, on input x, does not ask about whether x is in B.
- Very well-studied notion.
- For example, φ is in SAT if and only if (φ0 is in SAT) or(φ1 is in SAT)

Self-Reducibility

- Many of the important problems in (or near) NC1 have a special self-reducibility property:

Self-Reducibility

- Many of the important problems in (or near) NC1 have a special self-reducibility property: Instances of length n are AC0-Turing (or TC0-Turing) reducible to instances of length n½ via reductions of linear size.
- Examples:
- BFE
- the word problem over S5
- MAJORITY
- Iterated Product of 3-by-3 Integer Matrices

Self Reducibility

- The self-reduction of S5, on inputs of size n, uses (n½ + 1) oracle gates of size n½.
- Thus if S5 has TC0 circuits of size nk, it also has circuits of size (n½ + 1)nk/2= O(n(k+1)/2).
- Similar arguments hold for other classes (such as AC0[6] and NC1).
- More complicated self-reductions can be presented for MAJORITY and Iterated Product of 3-by-3 matrices.

A Corollary

- If BFE has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.
- If S5 has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.
- If MAJ has AC0[6] circuits, then it has such circuits of nearly linear size. (Etc.)
- Thus, e.g., to separate NC1 from TC0, it suffices to show that BFE requires TC0 circuits of size n1.0000001.

A Corollary

- If BFE has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.
- If S5 has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.
- If MAJ has AC0[6] circuits, then it has such circuits of nearly linear size. (Etc.)
- How widespread is this phenomenon? Is it true for SAT? (I.e., can we show NP ≠ TC0 by proving that SAT requires TC0 circuits of size n1.0000001?)

Limitations of Self-Reducibility

- Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC.
- Thus there is no obvious way to apply these techniques to SAT or to problems complete for P.
- …but perhaps, rather than showing directly that SAT has this strong form of self-reducibility, one can argue that if SAT is in TC0 then it has TC0 circuits of nearly-linear size.

Limitations of Self-Reducibility

- Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC.

Limitations of Self-Reducibility

- Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC.

d levels of oracle gates

Limitations of Self-Reducibility

d2 levels of oracle gates

Limitations of Self-Reducibility

After log log rounds,

the depth is logO(1)n

d3 levels of oracle gates

Prospects for Progress

- We have seen that existing techniques prove bounds that are “nearly” good enough to separate NC1 and TC0. Some of these proofs are “natural”.
- Don’t the results of [Razborov & Rudich] indicate that further progress will require very different approaches?
- Not necessarily!

Prospects for Progress

- The [Razborov & Rudich] framework of natural proofs assumes that a “natural” proof of a lower bound will make use of a combinatorial property that (among other things) is shared by a large fraction of the functions on n bits.
- In contrast, we are making use of a self-reducibility property that allows us to boost a n1+ε lower bound to a superpolynomial lower bound. This self-reducibility property holds for only a vanishingly small fraction of all functions.

Prospects for Progress

- These observations are simple, but …
- they have forever changed the way that we look at quadratic (and smaller) lower bounds.
- We are not claiming to have found a way around the obstacles identified by [Razborov & Rudich]. (Such a claim will have to wait until someone proves that NC1≠ TC0.) But we do believe that this avenue deserves further exploration.

Conclusion

- There are good reasons to develop and explore the connections between Kolmogorov complexity and circuit complexity.
- There are many open problems in this area that I will be delighted to discuss with you in more detail.
- There are two bad typos in the proceedings version of the paper. (“P” should be “NP”.) A corrected version is available at my home page.

Speculation

Connections between Kolmogorov Complexity and Circuit Complexity might be relevant to the question of whether NEXP is contained in (non-uniform) TC0 (depth 3).

Speculation

- [IKW] showed that NEXP is in P/poly iff NEXP = MA iff MA cannot be derandomized
- The proof shows that NEXP is in P/poly iff every set in P contains strings of KT-complexity O(log n) iff NEXP = IP[P/poly].

Speculation

Similar techniques show:

- NEXP is in nonuniform NC1 iff every set in P contains strings of KF-complexity O(log n) iff NEXP = MIPNC1 iff MIPNC1 cannot be derandomized.
- NEXP is in nonuniform TC0 iff every set in P contains strings of small complexity iff NEXP = MIPTC0 iff MIPTC0 cannot be derandomized.

Speculation

What else happens in such a collapse?

- If NP = uniform TC0, then #P is not contained in non-uniform TC0 (so NEXP is not in non-uniform TC0).
- So let’s consider NEXP = MIPTC0 and NP ≠ uniform TC0. If this “hardness assumption” were sufficient to “derandomize” MIPTC0 then this would give the desired lower bound on NEXP…
- [Fortnow, Klivans], [van Melkebeek, Santhanam]

Download Presentation

Connecting to Server..