Loading in 5 sec....

Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower BoundsPowerPoint Presentation

Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds

Download Presentation

Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds

Loading in 2 Seconds...

- 335 Views
- Updated On :
- Presentation posted in: Sports / GamesEducation / CareerFashion / BeautyGraphics / DesignNews / Politics

Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds. DCFS 2008. Today’s Goal:. To raise awareness of the tight connection between circuit complexity and Kolmogorov complexity. And to show that this is useful.

Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds

DCFS 2008

- To raise awareness of the tight connection between circuit complexity and Kolmogorov complexity.
- And to show that this is useful.
- To plant seeds of optimism, regarding the prospects of proving lower bounds in circuit complexity.

- C(x) = min{|d| : U(d) = x}
- Important property
- Invariance: The choice of the universal Turing machine U is unimportant.

- x is random if C(x) ≥ |x|.
- CA(x) = min{|d| : UA(d) = x}

- Let D be a circuit of AND and OR gates (with negations at the inputs). Size(D) = # of wires in D.
- Size(f) = min{Size(D) : D computes f}
- We may allow oracle gates for a set A, along with AND and OR gates.
- SizeA(f) = min{Size(D) : DA computes f}

- There are some obvious similarities in the definitions. What are some differences?
- A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings.
- Given any string x, let fx be the function whose truth table is the string of length 2log|x|, padded out with 0’s, and define Size(x) to be Size(fx).

- There are some obvious similarities in the definitions. What are some differences?
- A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings.
- A more fundamental difference:
- C(x) is not computable; Size(x) is.

- The Minimum Circuit Size Problem (MCSP): {(x,i) : Size(x) ≤ i}.

- MCSP is in NP, but is not known to be NP-complete.
- MCSP is not believed to be in P.
- Factoring is in BPPMCSP.
- Every cryptographically-secure one-way function can be inverted in PMCSP/poly.

- C(x) ≈ SizeH(x), where H is the halting problem.
- For one direction, let U(d) = x. We need a small circuit (with oracle gates for H) for fx, where fx(i) is the i-th bit of x. This is easy, since {(d,i,b) : U(d) outputs a string whose i-th bit is b} is computably-enumerable.
- For the other direction, let SizeH(fx) = m. No oracle gate has more than m wires coming into it. Given a description of D (size not much bigger than m) and the m-bit number giving the size of {y in H : |y| ≤ m}, U can simulate DH and produce fx

- C(x) ≈ SizeH(x), where H is the halting problem.
- So there is a connection between C(x) and Size(x) …
- …but is it useful?
- First, let’s look at decidable versions of Kolmogorov complexity.

- The usual definition:
- Ct(x) = min{|d| : U(d) = x in time t(|d|)}.
- Problems with this definition
- No invariance! If U and U’ are different universal Turing machines, CtU and CtU’ have no clear relationship.
- (One can bound CtU by Ct’U’ for t’ slightly larger than t – but nothing can be done for t’=t.)

- No nice connection to circuit complexity!

- Levin’s definition:
- Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}.
- Invariance holds! If U and U’ are different universal Turing machines, KtU(x) and KtU’(x) are within log |x| of each other.
- Let A be complete for E = Dtime(2O(n)). Then Kt(x) ≈ SizeA(x).

- Levin’s definition:
- Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}.
- Why log t?
- This gives an optimal search order for NP search problems.
- Adding t instead of log t would give every string complexity ≥ |x|.

- …So let’s look at how to make the run-time be much smaller.

- C(x) = min{|d| : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x} (where bit # i+1 of x is *).
- This is identical to the original definition.

- Kt(x) = min{|d|+log t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}.
- The new and old definitions are within O(log |x|) of each other.

- Define KT(x) = min{|d|+t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}.

- C(x) ≈ SizeH(x).
- Kt(x) ≈ SizeE(x).
- KT(x) ≈ Size(x).
- Other measures of complexity can be captured in this way, too:
- Branching Program Size ≈ KB(x) = min{|d|+2s : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in space s(|d|)}.

- C(x) ≈ SizeH(x).
- Kt(x) ≈ SizeE(x).
- KT(x) ≈ Size(x).
- Other measures of complexity can be captured in this way, too:
- Formula Size ≈ KF(x) = min{|d|+2t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.

- The result that Factoring is in BPPMCSP was first proved by observing that, in PMCSP, one can accept a large set of strings having large KT complexity (and by making use of many important results in the theory of pseudorandom generators and derandomization).
- (Basic Idea): There is a pseudorandom generator based on factoring, such that factoring is in BPPT for any test T that distinguishes truly random strings from pseudorandom strings. MCSP is such a test.

- Consider RKT, RKt, and RC.
- RKT is in coNP, and not known to be coNP hard.
- RC is not hard for NP under poly-time many-one reductions, unless P=NP.
- How about more powerful reductions?
- Is there anything interesting that we could compute quickly if C were computable, that we can’t already compute quickly?
- Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in

- But RC is undecidable! Perhaps H is in P relative to RC?

- Consider RKT, RKt, and RC.
- RKT is in coNP, and not known to be coNP hard.
- RC, is not hard for NP under poly-time many-one reductions, unless P=NP.
- How about more powerful reductions?
- PSPACE is in P relative to RC.
- NEXP is in NP relative to RC.
- Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in poly time, relative to RC [BFNV].

- But RC is undecidable! Perhaps H is in P relative to RC?

- Perhaps H is in P relative to RC?
- This is still open. It is known that there is a computable time bound t such that H is in DTime(t) relative to RC [Kummer].
- …but the bound t depends on the choice of U in the definition of C(x).
- We also know that H is in P/poly relative to RC.

- Consider RKT, RKt, and RC.
- What about RKt?
- RKt, is not hard for NP under poly-time many-one reductions, unless E=NE.
- How about more powerful reductions?
- EXP = NP(RKt).
- RKt is complete for EXP under P/poly reductions.
- Open if RKt is in P!

- C(x) ≈ SizeH(x).
- Kt(x) ≈ SizeE(x).
- KT(x) ≈ Size(x).
- Other measures of complexity can be captured in this way, too:
- Formula Size ≈ KF(x) = min{|d|+2t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.

- C(x) ≈ SizeH(x).
- Kt(x) ≈ SizeE(x).
- KT(x) ≈ Size(x).
- Other measures of complexity can be captured in this way, too:
- A similar definition captures depth k threshold circuit size.

- [This is the clever transition to start the discussion of circuit lower bounds…]

- NP
- P
- .
- .
- NC
- L (Deterministic Logspace)

- TC0O(1)-Depth Circuits of MAJ gates
- AC0 [6]
- NC1 Log-Depth Circuits
- AC0 can’t compute Mod 2 [FSS,A]
- AC0 O(1)-Depth Circuits of AND/OR gates

- TC0O(1)-Depth Circuits of MAJ gates
- AC0 [6]
- NC1 Log-Depth Circuits
- AC0 can’t compute Mod 2 [FSS,A]
- AC0 O(1)-Depth Circuits of AND/OR gates

- TC0O(1)-Depth Circuits of MAJ gates
- NC1 Log-Depth Circuits
- AC0 [2]can’t compute Mod 3 [R,S]
- AC0 [2]
- AC0 O(1)-Depth Circuits of AND/OR gates

- NC1 Log-Depth Circuits
- TC0O(1)-Depth Circuits of MAJ gates
- AC0 [6]
- AC0 [2]
- AC0 O(1)-Depth Circuits of AND/OR gates

- NC1 poly-size formulae
- TC0O(1)-Depth Circuits of MAJ gates
- AC0 [6]
- AC0 [2]
- AC0 O(1)-Depth Circuits of AND/OR gates

- NP has complete sets (under polynomial time reducibility ≤P)
- These small classes have complete sets, too (under ≤AC°)

- A ≤AC°B means that there is a constant-depth circuit computing A that has the usual AND and OR gates, and also has ‘oracle gates’ for B.

B

- NC1
- TC0
- AC0 [6]
- AC0 [2]
- AC0

- sorting, multiplication, division
- [Naor,Reingold] Pseudorandom Generator

- NC1
- TC0
- AC0 [6]
- AC0 [2]
- AC0

- BFE: Balanced Boolean Formula Evaluation (AND,OR,XOR)
- Word problem over S5

- A regular set complete for NC1

=

- NP (SAT, Clique, TSP,…)
- P (Linear Programming, CVP, …)
- NL (Connectivity, Shortest Paths, 2SAT, …)
- L (Undirected Connectivity, Acyclicity, …)
- NC1 (BFE, Regular Sets)
- TC0 (Sorting, Multiplication, Division)

We’re interested in NC1 (for instance) not because we want to build formulae for these functions…

- NP (SAT, Clique, TSP,…)
- P (Linear Programming, CVP, …)
- NL (Connectivity, Shortest Paths, 2SAT, …)
- L (Undirected Connectivity, Acyclicity, …)
- NC1 (BFE, Regular Sets)
- TC0 (Sorting, Multiplication, Division)

… but because we want to know if the blocks of this partition are distinct.

- NP (SAT, Clique, TSP,…)
- P (Linear Programming, CVP, …)
- NL (Connectivity, Shortest Paths, 2SAT, …)
- L (Undirected Connectivity, Acyclicity, …)
- NC1 (BFE, Regular Sets)
- TC0 (Sorting, Multiplication, Division)

These classes are real.

They’re important.

- Is P = NP?
- Is AC0[6] = NP?
- Is depth 3 AC0[6] = NP?

We’ll focus on questions such as:

Is BFE in TC0?

Is BFE in AC0[6]?

- Conventional Wisdom: Not Close At All!
- No new superpolynomial size lower bounds in over two decades.
- Razborov and Rudich: Any “natural” argument proving a lower bound against a circuit class C yields a proof that C can’t compute a pseudorandom function generator.
- Since the [Naor, Reingold] generator is computable in TC0, this is bad news.

- Problems requiring formulae of size n3 [Håstad]
- Problems requiring branching programs of size nearly n loglog n [Beame, Saks, Sun, Vee]
- Problems requiring depth d TC0 circuits of size n1+c [Impagliazzo, Paturi, Saks]
- Time-Space Tradeoffs [Fortnow, Lipton, Van Melkebeek, Viglas]
- There is little feeling that these results bring us any closer to separating complexity classes.

- How close are the following two statements?
- TC0 Circuits for BFE must be of size n1+Ω(1)
- For some c>0, TC0 Circuits for BFE must be of size n1+c.

- How close are the following two statements?
- TC0 Circuits for BFE must be of size n1+Ω(1)
- For some c>0, FTC0 Circuits for BFE must be of size n1+c

This is known [IPS’97]

This implies TC0≠ NC1 [A, Koucky]

- A set B is said to be “self-reducible” if B≤PB

- A set B is said to be “self-reducible” if B≤PB via a reduction that, on input x, does not ask about whether x is in B.
- Very well-studied notion.
- For example, φ is in SAT if and only if (φ0 is in SAT) or(φ1 is in SAT)

- Many of the important problems in (or near) NC1 have a special self-reducibility property:

- Many of the important problems in (or near) NC1 have a special self-reducibility property: Instances of length n are AC0-Turing (or TC0-Turing) reducible to instances of length n½ via reductions of linear size.
- Examples:
- BFE
- the word problem over S5
- MAJORITY
- Iterated Product of 3-by-3 Integer Matrices

- BFE

A subformula near the root

Subformulae near inputs

- S5

- The self-reduction of S5, on inputs of size n, uses (n½ + 1) oracle gates of size n½.
- Thus if S5 has TC0 circuits of size nk, it also has circuits of size (n½ + 1)nk/2= O(n(k+1)/2).
- Similar arguments hold for other classes (such as AC0[6] and NC1).
- More complicated self-reductions can be presented for MAJORITY and Iterated Product of 3-by-3 matrices.

- If BFE has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.
- If S5 has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.
- If MAJ has AC0[6] circuits, then it has such circuits of nearly linear size. (Etc.)
- Thus, e.g., to separate NC1 from TC0, it suffices to show that BFE requires TC0 circuits of size n1.0000001.

- If BFE has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.
- If S5 has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.
- If MAJ has AC0[6] circuits, then it has such circuits of nearly linear size. (Etc.)
- How widespread is this phenomenon? Is it true for SAT? (I.e., can we show NP ≠ TC0 by proving that SAT requires TC0 circuits of size n1.0000001?)

- Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC.
- Thus there is no obvious way to apply these techniques to SAT or to problems complete for P.
- …but perhaps, rather than showing directly that SAT has this strong form of self-reducibility, one can argue that if SAT is in TC0 then it has TC0 circuits of nearly-linear size.

- Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC.

- Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC.

d levels of oracle gates

d2 levels of oracle gates

After log log rounds,

the depth is logO(1)n

d3 levels of oracle gates

- We have seen that existing techniques prove bounds that are “nearly” good enough to separate NC1 and TC0. Some of these proofs are “natural”.
- Don’t the results of [Razborov & Rudich] indicate that further progress will require very different approaches?
- Not necessarily!

- The [Razborov & Rudich] framework of natural proofs assumes that a “natural” proof of a lower bound will make use of a combinatorial property that (among other things) is shared by a large fraction of the functions on n bits.
- In contrast, we are making use of a self-reducibility property that allows us to boost a n1+ε lower bound to a superpolynomial lower bound. This self-reducibility property holds for only a vanishingly small fraction of all functions.

- These observations are simple, but …
- they have forever changed the way that we look at quadratic (and smaller) lower bounds.
- We are not claiming to have found a way around the obstacles identified by [Razborov & Rudich]. (Such a claim will have to wait until someone proves that NC1≠ TC0.) But we do believe that this avenue deserves further exploration.

- There are good reasons to develop and explore the connections between Kolmogorov complexity and circuit complexity.
- There are many open problems in this area that I will be delighted to discuss with you in more detail.
- There are two bad typos in the proceedings version of the paper. (“P” should be “NP”.) A corrected version is available at my home page.

Connections between Kolmogorov Complexity and Circuit Complexity might be relevant to the question of whether NEXP is contained in (non-uniform) TC0 (depth 3).

- [IKW] showed that NEXP is in P/poly iff NEXP = MA iff MA cannot be derandomized
- The proof shows that NEXP is in P/poly iff every set in P contains strings of KT-complexity O(log n) iff NEXP = IP[P/poly].

Similar techniques show:

- NEXP is in nonuniform NC1 iff every set in P contains strings of KF-complexity O(log n) iff NEXP = MIPNC1 iff MIPNC1 cannot be derandomized.
- NEXP is in nonuniform TC0 iff every set in P contains strings of small complexity iff NEXP = MIPTC0 iff MIPTC0 cannot be derandomized.

What else happens in such a collapse?

- If NP = uniform TC0, then #P is not contained in non-uniform TC0 (so NEXP is not in non-uniform TC0).
- So let’s consider NEXP = MIPTC0 and NP ≠ uniform TC0. If this “hardness assumption” were sufficient to “derandomize” MIPTC0 then this would give the desired lower bound on NEXP…
- [Fortnow, Klivans], [van Melkebeek, Santhanam]