Circuit complexity kolmogorov complexity and prospects for lower bounds
Download
1 / 63

Circuit Complexity - PowerPoint PPT Presentation


  • 350 Views
  • Updated On :

Circuit Complexity, Kolmogorov Complexity, and Prospects for Lower Bounds. DCFS 2008. Today’s Goal:. To raise awareness of the tight connection between circuit complexity and Kolmogorov complexity. And to show that this is useful.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Circuit Complexity' - issac


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Today s goal
Today’s Goal: Lower Bounds

  • To raise awareness of the tight connection between circuit complexity and Kolmogorov complexity.

  • And to show that this is useful.

  • To plant seeds of optimism, regarding the prospects of proving lower bounds in circuit complexity.


Kolmogorov complexity
Kolmogorov Complexity Lower Bounds

  • C(x) = min{|d| : U(d) = x}

  • Important property

    • Invariance: The choice of the universal Turing machine U is unimportant.

  • x is random if C(x) ≥ |x|.

  • CA(x) = min{|d| : UA(d) = x}


Circuit complexity
Circuit Complexity Lower Bounds

  • Let D be a circuit of AND and OR gates (with negations at the inputs). Size(D) = # of wires in D.

  • Size(f) = min{Size(D) : D computes f}

  • We may allow oracle gates for a set A, along with AND and OR gates.

  • SizeA(f) = min{Size(D) : DA computes f}


K complexity circuit complexity
K-complexity Lower Bounds≈ Circuit Complexity

  • There are some obvious similarities in the definitions. What are some differences?

  • A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings.

    • Given any string x, let fx be the function whose truth table is the string of length 2log|x|, padded out with 0’s, and define Size(x) to be Size(fx).


K complexity circuit complexity1
K-complexity Lower Bounds≈ Circuit Complexity

  • There are some obvious similarities in the definitions. What are some differences?

  • A minor difference: Size gives a measure of the complexity of functions, C gives a measure of the complexity of strings.

  • A more fundamental difference:

    • C(x) is not computable; Size(x) is.

  • The Minimum Circuit Size Problem (MCSP): {(x,i) : Size(x) ≤ i}.


MCSP Lower Bounds

  • MCSP is in NP, but is not known to be NP-complete.

  • MCSP is not believed to be in P.

    • Factoring is in BPPMCSP.

    • Every cryptographically-secure one-way function can be inverted in PMCSP/poly.


So how can k complexity and circuit complexity be the same
So how can K-complexity and Circuit complexity be the same? Lower Bounds

  • C(x) ≈ SizeH(x), where H is the halting problem.

  • For one direction, let U(d) = x. We need a small circuit (with oracle gates for H) for fx, where fx(i) is the i-th bit of x. This is easy, since {(d,i,b) : U(d) outputs a string whose i-th bit is b} is computably-enumerable.

  • For the other direction, let SizeH(fx) = m. No oracle gate has more than m wires coming into it. Given a description of D (size not much bigger than m) and the m-bit number giving the size of {y in H : |y| ≤ m}, U can simulate DH and produce fx


So how can k complexity and circuit complexity be the same1
So how can K-complexity and Circuit complexity be the same? Lower Bounds

  • C(x) ≈ SizeH(x), where H is the halting problem.

  • So there is a connection between C(x) and Size(x) …

  • …but is it useful?

  • First, let’s look at decidable versions of Kolmogorov complexity.


Time bounded kolmogorov complexity
Time-Bounded Kolmogorov Complexity Lower Bounds

  • The usual definition:

  • Ct(x) = min{|d| : U(d) = x in time t(|d|)}.

  • Problems with this definition

    • No invariance! If U and U’ are different universal Turing machines, CtU and CtU’ have no clear relationship.

    • (One can bound CtU by Ct’U’ for t’ slightly larger than t – but nothing can be done for t’=t.)

  • No nice connection to circuit complexity!


Time bounded kolmogorov complexity1
Time-Bounded Kolmogorov Complexity Lower Bounds

  • Levin’s definition:

  • Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}.

  • Invariance holds! If U and U’ are different universal Turing machines, KtU(x) and KtU’(x) are within log |x| of each other.

  • Let A be complete for E = Dtime(2O(n)). Then Kt(x) ≈ SizeA(x).


Time bounded kolmogorov complexity2
Time-Bounded Kolmogorov Complexity Lower Bounds

  • Levin’s definition:

  • Kt(x) = min{|d|+log t : U(d) = x in time t(|d|)}.

  • Why log t?

    • This gives an optimal search order for NP search problems.

    • Adding t instead of log t would give every string complexity ≥ |x|.

  • …So let’s look at how to make the run-time be much smaller.


Revised kolmogorov complexity
Revised Kolmogorov Complexity Lower Bounds

  • C(x) = min{|d| : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x} (where bit # i+1 of x is *).

    • This is identical to the original definition.

  • Kt(x) = min{|d|+log t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}.

    • The new and old definitions are within O(log |x|) of each other.

  • Define KT(x) = min{|d|+t : for all i ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}.


Kolmogorov complexity is circuit complexity
Kolmogorov Complexity is Circuit Complexity Lower Bounds

  • C(x) ≈ SizeH(x).

  • Kt(x) ≈ SizeE(x).

  • KT(x) ≈ Size(x).

  • Other measures of complexity can be captured in this way, too:

    • Branching Program Size ≈ KB(x) = min{|d|+2s : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in space s(|d|)}.


Kolmogorov complexity is circuit complexity1
Kolmogorov Complexity is Circuit Complexity Lower Bounds

  • C(x) ≈ SizeH(x).

  • Kt(x) ≈ SizeE(x).

  • KT(x) ≈ Size(x).

  • Other measures of complexity can be captured in this way, too:

    • Formula Size ≈ KF(x) = min{|d|+2t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.


But is this interesting
…but is this interesting? Lower Bounds

  • The result that Factoring is in BPPMCSP was first proved by observing that, in PMCSP, one can accept a large set of strings having large KT complexity (and by making use of many important results in the theory of pseudorandom generators and derandomization).

  • (Basic Idea): There is a pseudorandom generator based on factoring, such that factoring is in BPPT for any test T that distinguishes truly random strings from pseudorandom strings. MCSP is such a test.


This idea has many variants
This idea has many variants. Lower Bounds

  • Consider RKT, RKt, and RC.

  • RKT is in coNP, and not known to be coNP hard.

  • RC is not hard for NP under poly-time many-one reductions, unless P=NP.

    • How about more powerful reductions?

    • Is there anything interesting that we could compute quickly if C were computable, that we can’t already compute quickly?

    • Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in

  • But RC is undecidable! Perhaps H is in P relative to RC?


This idea has many variants1
This idea has many variants. Lower Bounds

  • Consider RKT, RKt, and RC.

  • RKT is in coNP, and not known to be coNP hard.

  • RC, is not hard for NP under poly-time many-one reductions, unless P=NP.

    • How about more powerful reductions?

    • PSPACE is in P relative to RC.

    • NEXP is in NP relative to RC.

    • Proof uses PRGs, Interactive Proofs, and the fact that an element of RC of length n can be found in poly time, relative to RC [BFNV].

  • But RC is undecidable! Perhaps H is in P relative to RC?


Relationship between h and r c
Relationship between H and R Lower BoundsC

  • Perhaps H is in P relative to RC?

  • This is still open. It is known that there is a computable time bound t such that H is in DTime(t) relative to RC [Kummer].

  • …but the bound t depends on the choice of U in the definition of C(x).

  • We also know that H is in P/poly relative to RC.


This idea has many variants2
This idea has many variants. Lower Bounds

  • Consider RKT, RKt, and RC.

  • What about RKt?

  • RKt, is not hard for NP under poly-time many-one reductions, unless E=NE.

    • How about more powerful reductions?

    • EXP = NP(RKt).

    • RKt is complete for EXP under P/poly reductions.

    • Open if RKt is in P!


Kolmogorov complexity is circuit complexity2
Kolmogorov Complexity is Circuit Complexity Lower Bounds

  • C(x) ≈ SizeH(x).

  • Kt(x) ≈ SizeE(x).

  • KT(x) ≈ Size(x).

  • Other measures of complexity can be captured in this way, too:

    • Formula Size ≈ KF(x) = min{|d|+2t : for all I ≤ |x| + 1, U(d,i,b) = 1 iff b is the i-th bit of x, in time t(|d|)}, for an alternating Turing machine U.


Kolmogorov complexity is circuit complexity3
Kolmogorov Complexity is Circuit Complexity Lower Bounds

  • C(x) ≈ SizeH(x).

  • Kt(x) ≈ SizeE(x).

  • KT(x) ≈ Size(x).

  • Other measures of complexity can be captured in this way, too:

    • A similar definition captures depth k threshold circuit size.

  • [This is the clever transition to start the discussion of circuit lower bounds…]


Big complexity classes
Big Complexity Classes Lower Bounds

  • NP

  • P

  • .

  • .

  • NC

  • L (Deterministic Logspace)


The main objects of interest small complexity classes
The Main Objects of Interest: Lower BoundsSmall Complexity Classes

  • TC0O(1)-Depth Circuits of MAJ gates

  • AC0 [6]

  • NC1 Log-Depth Circuits

  • AC0 can’t compute Mod 2 [FSS,A]

  • AC0 O(1)-Depth Circuits of AND/OR gates


The main objects of interest small complexity classes1
The Main Objects of Interest: Lower BoundsSmall Complexity Classes

  • TC0O(1)-Depth Circuits of MAJ gates

  • AC0 [6]

  • NC1 Log-Depth Circuits

  • AC0 can’t compute Mod 2 [FSS,A]

  • AC0 O(1)-Depth Circuits of AND/OR gates


The main objects of interest small complexity classes2
The Main Objects of Interest: Lower BoundsSmall Complexity Classes

  • TC0O(1)-Depth Circuits of MAJ gates

  • NC1 Log-Depth Circuits

  • AC0 [2]can’t compute Mod 3 [R,S]

  • AC0 [2]

  • AC0 O(1)-Depth Circuits of AND/OR gates


The main objects of interest small complexity classes3
The Main Objects of Interest: Lower BoundsSmall Complexity Classes

  • NC1 Log-Depth Circuits

  • TC0O(1)-Depth Circuits of MAJ gates

  • AC0 [6]

  • AC0 [2]

  • AC0 O(1)-Depth Circuits of AND/OR gates


The main objects of interest small complexity classes4
The Main Objects of Interest: Lower BoundsSmall Complexity Classes

  • NC1 poly-size formulae

  • TC0O(1)-Depth Circuits of MAJ gates

  • AC0 [6]

  • AC0 [2]

  • AC0 O(1)-Depth Circuits of AND/OR gates


Complete problems
Complete Problems Lower Bounds

  • NP has complete sets (under polynomial time reducibility ≤P)

  • These small classes have complete sets, too (under ≤AC°)


Reductions
Reductions Lower Bounds

  • A ≤AC°B means that there is a constant-depth circuit computing A that has the usual AND and OR gates, and also has ‘oracle gates’ for B.

B


Complete problems1
Complete Problems Lower Bounds

  • NC1

  • TC0

  • AC0 [6]

  • AC0 [2]

  • AC0

  • sorting, multiplication, division

  • [Naor,Reingold] Pseudorandom Generator


Complete problems2
Complete Problems Lower Bounds

  • NC1

  • TC0

  • AC0 [6]

  • AC0 [2]

  • AC0

  • BFE: Balanced Boolean Formula Evaluation (AND,OR,XOR)

  • Word problem over S5


The word problem over s 5
The Word Problem Over S Lower Bounds5

  • A regular set complete for NC1

=


Complexity classes are not invented they re discovered
Complexity Classes are not Invented – They’re Discovered Lower Bounds

  • NP (SAT, Clique, TSP,…)

  • P (Linear Programming, CVP, …)

  • NL (Connectivity, Shortest Paths, 2SAT, …)

  • L (Undirected Connectivity, Acyclicity, …)

  • NC1 (BFE, Regular Sets)

  • TC0 (Sorting, Multiplication, Division)

We’re interested in NC1 (for instance) not because we want to build formulae for these functions…


Complexity classes are not invented they re discovered1
Complexity Classes are not Invented – They’re Discovered Lower Bounds

  • NP (SAT, Clique, TSP,…)

  • P (Linear Programming, CVP, …)

  • NL (Connectivity, Shortest Paths, 2SAT, …)

  • L (Undirected Connectivity, Acyclicity, …)

  • NC1 (BFE, Regular Sets)

  • TC0 (Sorting, Multiplication, Division)

… but because we want to know if the blocks of this partition are distinct.


Complexity classes are not invented they re discovered2
Complexity Classes are not Invented – They’re Discovered Lower Bounds

  • NP (SAT, Clique, TSP,…)

  • P (Linear Programming, CVP, …)

  • NL (Connectivity, Shortest Paths, 2SAT, …)

  • L (Undirected Connectivity, Acyclicity, …)

  • NC1 (BFE, Regular Sets)

  • TC0 (Sorting, Multiplication, Division)

These classes are real.

They’re important.


Longstanding open problems
Longstanding Open Problems Lower Bounds

  • Is P = NP?

  • Is AC0[6] = NP?

  • Is depth 3 AC0[6] = NP?

We’ll focus on questions such as:

Is BFE in TC0?

Is BFE in AC0[6]?


How close are we to proving circuit lower bounds
How Close Are We to Proving Circuit Lower Bounds? Lower Bounds

  • Conventional Wisdom: Not Close At All!

  • No new superpolynomial size lower bounds in over two decades.

  • Razborov and Rudich: Any “natural” argument proving a lower bound against a circuit class C yields a proof that C can’t compute a pseudorandom function generator.

  • Since the [Naor, Reingold] generator is computable in TC0, this is bad news.


More modest goals
More Modest Goals Lower Bounds

  • Problems requiring formulae of size n3 [Håstad]

  • Problems requiring branching programs of size nearly n loglog n [Beame, Saks, Sun, Vee]

  • Problems requiring depth d TC0 circuits of size n1+c [Impagliazzo, Paturi, Saks]

  • Time-Space Tradeoffs [Fortnow, Lipton, Van Melkebeek, Viglas]

  • There is little feeling that these results bring us any closer to separating complexity classes.


How close are we to proving circuit lower bounds1
How Close Are We to Proving Circuit Lower Bounds? Lower Bounds

  • How close are the following two statements?

  • TC0 Circuits for BFE must be of size n1+Ω(1)

  • For some c>0, TC0 Circuits for BFE must be of size n1+c.


How close are we to proving circuit lower bounds2
How Close Are We to Proving Circuit Lower Bounds? Lower Bounds

  • How close are the following two statements?

  • TC0 Circuits for BFE must be of size n1+Ω(1)

  • For some c>0, FTC0 Circuits for BFE must be of size n1+c

This is known [IPS’97]

This implies TC0≠ NC1 [A, Koucky]


Self reducibility
Self-Reducibility Lower Bounds

  • A set B is said to be “self-reducible” if B≤PB


Self reducibility1
Self-Reducibility Lower Bounds

  • A set B is said to be “self-reducible” if B≤PB via a reduction that, on input x, does not ask about whether x is in B.

  • Very well-studied notion.

  • For example, φ is in SAT if and only if (φ0 is in SAT) or(φ1 is in SAT)


Self reducibility2
Self-Reducibility Lower Bounds

  • Many of the important problems in (or near) NC1 have a special self-reducibility property:


Self reducibility3
Self-Reducibility Lower Bounds

  • Many of the important problems in (or near) NC1 have a special self-reducibility property: Instances of length n are AC0-Turing (or TC0-Turing) reducible to instances of length n½ via reductions of linear size.

  • Examples:

    • BFE

    • the word problem over S5

    • MAJORITY

    • Iterated Product of 3-by-3 Integer Matrices


Self reducibility4
Self Reducibility Lower Bounds

  • BFE

A subformula near the root

Subformulae near inputs


Self reducibility5
Self Reducibility Lower Bounds

  • S5


Self reducibility6
Self Reducibility Lower Bounds

  • The self-reduction of S5, on inputs of size n, uses (n½ + 1) oracle gates of size n½.

  • Thus if S5 has TC0 circuits of size nk, it also has circuits of size (n½ + 1)nk/2= O(n(k+1)/2).

  • Similar arguments hold for other classes (such as AC0[6] and NC1).

  • More complicated self-reductions can be presented for MAJORITY and Iterated Product of 3-by-3 matrices.


A corollary
A Corollary Lower Bounds

  • If BFE has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.

  • If S5 has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.

  • If MAJ has AC0[6] circuits, then it has such circuits of nearly linear size. (Etc.)

  • Thus, e.g., to separate NC1 from TC0, it suffices to show that BFE requires TC0 circuits of size n1.0000001.


A corollary1
A Corollary Lower Bounds

  • If BFE has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.

  • If S5 has TC0 or AC0[6] circuits, then it has such circuits of nearly linear size.

  • If MAJ has AC0[6] circuits, then it has such circuits of nearly linear size. (Etc.)

  • How widespread is this phenomenon? Is it true for SAT? (I.e., can we show NP ≠ TC0 by proving that SAT requires TC0 circuits of size n1.0000001?)


Limitations of self reducibility
Limitations of Self-Reducibility Lower Bounds

  • Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC.

  • Thus there is no obvious way to apply these techniques to SAT or to problems complete for P.

  • …but perhaps, rather than showing directly that SAT has this strong form of self-reducibility, one can argue that if SAT is in TC0 then it has TC0 circuits of nearly-linear size.


Limitations of self reducibility1
Limitations of Self-Reducibility Lower Bounds

  • Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC.


Limitations of self reducibility2
Limitations of Self-Reducibility Lower Bounds

  • Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC.

d levels of oracle gates


Limitations of self reducibility3
Limitations of Self-Reducibility Lower Bounds

  • Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC.

d2 levels of oracle gates


Limitations of self reducibility4
Limitations of Self-Reducibility Lower Bounds

  • Any problem for which instances of length n are TC0-Turing reducible to instances of length n½ via poly-size reductions lies in NC.

After log log rounds,

the depth is logO(1)n

d3 levels of oracle gates


Prospects for progress
Prospects for Progress Lower Bounds

  • We have seen that existing techniques prove bounds that are “nearly” good enough to separate NC1 and TC0. Some of these proofs are “natural”.

  • Don’t the results of [Razborov & Rudich] indicate that further progress will require very different approaches?

  • Not necessarily!


Prospects for progress1
Prospects for Progress Lower Bounds

  • The [Razborov & Rudich] framework of natural proofs assumes that a “natural” proof of a lower bound will make use of a combinatorial property that (among other things) is shared by a large fraction of the functions on n bits.

  • In contrast, we are making use of a self-reducibility property that allows us to boost a n1+ε lower bound to a superpolynomial lower bound. This self-reducibility property holds for only a vanishingly small fraction of all functions.


Prospects for progress2
Prospects for Progress Lower Bounds

  • These observations are simple, but …

  • they have forever changed the way that we look at quadratic (and smaller) lower bounds.

  • We are not claiming to have found a way around the obstacles identified by [Razborov & Rudich]. (Such a claim will have to wait until someone proves that NC1≠ TC0.) But we do believe that this avenue deserves further exploration.


Conclusion
Conclusion Lower Bounds

  • There are good reasons to develop and explore the connections between Kolmogorov complexity and circuit complexity.

  • There are many open problems in this area that I will be delighted to discuss with you in more detail.

  • There are two bad typos in the proceedings version of the paper. (“P” should be “NP”.) A corrected version is available at my home page.


Speculation
Speculation Lower Bounds

Connections between Kolmogorov Complexity and Circuit Complexity might be relevant to the question of whether NEXP is contained in (non-uniform) TC0 (depth 3).


Speculation1
Speculation Lower Bounds

  • [IKW] showed that NEXP is in P/poly iff NEXP = MA iff MA cannot be derandomized

  • The proof shows that NEXP is in P/poly iff every set in P contains strings of KT-complexity O(log n) iff NEXP = IP[P/poly].


Speculation2
Speculation Lower Bounds

Similar techniques show:

  • NEXP is in nonuniform NC1 iff every set in P contains strings of KF-complexity O(log n) iff NEXP = MIPNC1 iff MIPNC1 cannot be derandomized.

  • NEXP is in nonuniform TC0 iff every set in P contains strings of small complexity iff NEXP = MIPTC0 iff MIPTC0 cannot be derandomized.


Speculation3
Speculation Lower Bounds

What else happens in such a collapse?

  • If NP = uniform TC0, then #P is not contained in non-uniform TC0 (so NEXP is not in non-uniform TC0).

  • So let’s consider NEXP = MIPTC0 and NP ≠ uniform TC0. If this “hardness assumption” were sufficient to “derandomize” MIPTC0 then this would give the desired lower bound on NEXP…

  • [Fortnow, Klivans], [van Melkebeek, Santhanam]


ad