curiouser and curiouser the link between incompressibility and complexity n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Curiouser and Curiouser : The Link between Incompressibility and Complexity PowerPoint Presentation
Download Presentation
Curiouser and Curiouser : The Link between Incompressibility and Complexity

Loading in 2 Seconds...

play fullscreen
1 / 50

Curiouser and Curiouser : The Link between Incompressibility and Complexity - PowerPoint PPT Presentation


  • 61 Views
  • Uploaded on

Curiouser and Curiouser : The Link between Incompressibility and Complexity. CiE Special Session, June 19, 2012. Today’s Goal:. To present new developments in a line of research dating back to 2002, presenting some unexpected connections between

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Curiouser and Curiouser : The Link between Incompressibility and Complexity' - tilden


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
curiouser and curiouser the link between incompressibility and complexity

Curiouser and Curiouser: The Link between Incompressibility and Complexity

CiE Special Session, June 19, 2012

today s goal
Today’s Goal:
  • To present new developments in a line of research dating back to 2002, presenting some unexpected connections between
    • Kolmogorov Complexity (the theory of randomness), and
    • Computational Complexity Theory
  • Which ought to have nothing to do with each other!
complexity classes
Complexity Classes

EXPSPACE

NEXP

PSPACE

NP

P/poly

BPP

P

a jewel of derandomization
A Jewel of Derandomization
  • [Impagliazzo, Wigderson, 1997]: If there is a problem computable in time 2n that requires circuits of size 2εn, then P = BPP.
kolmogorov complexity
Kolmogorov Complexity

C(x) = min{|d| : U(d) = x}

U is a “universal” Turing machine

K(x) = min{|d| : U(d) = x}

U is a “universal” prefix-free Turing machine

Important property

Invariance: The choice of the universal Turing machine U is unimportant (up to an additive constant).

x is random if C(x) ≥ |x|.

kolmogorov complexity1
Kolmogorov Complexity

C(x) = min{|d| : U(d) = x}

U is a “universal” Turing machine

K(x) = min{|d| : U(d) = x}

U is a “universal” prefix-free Turing machine

Important property

Invariance: The choice of the universal Turing machine U is unimportant (up to an additive constant).

x is random if C(x) ≥ |x|, or K(x) ≥ |x|.

k c and randomness
K, C, and Randomness
  • K(x) and C(x) are “close”:
    • C(x) ≤ K(x) ≤ C(x) + 2 log |x|
  • Two notions of randomness:
    • RC = {x : C(x) ≥ |x|}
    • RK = {x : K(x) ≥ |x|}
  • …actually, infinitely many notions of randomness:
    • RCU = {x : CU(x) ≥ |x|}
k c and randomness1
K, C, and Randomness
  • K(x) and C(x) are “close”:
    • C(x) ≤ K(x) ≤ C(x) + 2 log |x|
  • Two notions of randomness:
    • RC = {x : C(x) ≥ |x|}
    • RK = {x : K(x) ≥ |x|}
  • …actually, infinitely many notions of randomness:
    • RCU = {x : CU(x) ≥ |x|}, RKU = {x : KU(x) ≥ |x|}
k c and randomness2
K, C, and Randomness
  • When it makes no difference, we’ll write “R” instead of RC or RK.
  • Basic facts:
    • R is undecidable
    • …but it is not “easy” to use it as an oracle.
    • R is not NP-hard under poly-time ≤m reductions, unless P=NP.
    • Things get more interesting when we consider more powerful types of reducibility.
three bizarre inclusions
Three Bizarre Inclusions
  • NEXP is contained in NPR. [ABK06]
  • PSPACE is contained in PR. [ABKMR06]
  • BPP is contained in {A : A is poly-time ≤tt R}. [BFKL10]
    • A ≤tt reduction is a “non-adaptive” reduction.
    • On input x, a list of queries is formulated before receiving any answer from the oracle.
three bizarre inclusions1
Three Bizarre Inclusions
  • NEXP is contained in NPR. [ABK06]
  • PSPACE is contained in PR. [ABKMR06]
  • BPP is contained in PttR. [BFKL10]

“Bizarre”, because a non-computable “upper bound” is presented on complexity classes!

We have been unable to squeeze larger complexity classes inside. Are these containments optimal?

three bizarre inclusions2
Three Bizarre Inclusions
  • NEXP is contained in NPR. [ABK06]
  • PSPACE is contained in PR. [ABKMR06]
  • BPP is contained in PttR. [BFKL10]

“Bizarre”, because a non-computable “upper bound” is presented on complexity classes!

If we restrict attention to RK, then we can do better…

three bizarre inclusions3
Three Bizarre Inclusions
  • NEXP is contained in NPRK.
    • The decidable sets that are in NPRK for every U are in EXPSPACE. [AFG11]
  • PSPACE is contained in PRK.
  • BPP is contained in PttRK.
    • The decidable sets that are in PttRK for every U are in PSPACE. [AFG11]
three bizarre inclusions4
Three Bizarre Inclusions
  • NEXP is contained in NPRK (for every U).
    • The decidable sets that are in NPRK for every U are in EXPSPACE. [AFG11]
  • PSPACE is contained in PRK (for every U).
  • BPP is contained in PttRK (for every U).
    • The decidable sets that are in PttRK for every U are in PSPACE. [AFG11]
    • [CELM] The sets that are in PttRK for every U are decidable.
three bizarre inclusions5
Three Bizarre Inclusions
  • NEXP is contained in NPRK (for every U).
    • The decidable sets that are in NPRK for every U are in EXPSPACE. [AFG11]
  • PSPACE is contained in PRK (for every U).
  • BPP is contained in PttRK (for every U).
    • The sets that are in PttRK for every U are in PSPACE. [AFG11]
three bizarre inclusions6
Three Bizarre Inclusions
  • NEXP is contained in NPRK (for every U).
    • The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
  • PSPACE is contained in PRK (for every U).
  • BPP is contained in PttRK (for every U).
    • The sets that are in PttRK for every U are in PSPACE. [AFG11]
three bizarre inclusions7
Three Bizarre Inclusions
  • NEXP is contained in NPRK (for every U).
    • The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
  • Conjecture: This should hold for RC, too.
  • BPP is contained in PttRK (for every U).
    • The sets that are in PttRK for every U are in PSPACE. [AFG11]
three bizarre inclusions8
Three Bizarre Inclusions
  • NEXP is contained in NPRK (for every U).
    • The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
  • This holds even for sets in EXPttRK for all U!
  • BPP is contained in PttRK (for every U).
    • The sets that are in PttRK for every U are in PSPACE. [AFG11]
three bizarre inclusions9
Three Bizarre Inclusions
  • NEXP is contained in NPRK (for every U).
    • The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
  • Conjecture: This class is exactly NEXP.
  • BPP is contained in PttRK (for every U).
    • The sets that are in PttRK for every U are in PSPACE. [AFG11]

Conjecture: This class is exactly BPP.

three bizarre inclusions10
Three Bizarre Inclusions
  • NEXP is contained in NPRK (for every U).
    • The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
  • Conjecture: This class is exactly NEXP.
  • BPP is contained in PttRK (for every U).
    • The sets that are in PttRK for every U are in PSPACE. [AFG11]

Conjecture: This class is exactly BPP P.

k complexity and bpp vs p
K-Complexity and BPP vs P
  • BPP is contained in PttRK (for every U).
    • The sets that are in PttRK for every U are in PSPACE.

Conjecture: This class is exactly P.

  • Some support for this conjecture [ABK06]:
    • The decidable sets that are in PdttRC for every U are in P.
    • The decidable sets that are in Pparity-ttRC for every U are in P.
k complexity and bpp vs p1
K-Complexity and BPP vs P
  • BPP is contained in PttRK (for every U).
    • The sets that are in PttRK for every U are in PSPACE.

Conjecture: This class is exactly P.

  • New results support a weaker conjecture:
  • Conjecture: This class is contained in PSPACE ∩ P/poly.
  • More strongly: Every decidable set in PttR is in P/poly.
k complexity and bpp vs p2
K-Complexity and BPP vs P
  • BPP is contained in PttRK (for every U).
    • The sets that are in PttRK for every U are in PSPACE.

Conjecture: This class is exactly P.

  • New results support a weaker conjecture :
  • Conjecture: This class is contained in PSPACE ∩ P/poly.
  • More strongly: Every decidable set in PttR is in P/poly (i.e., for every U, and for both C and K).
the central conjecture
The Central Conjecture
  • Conjecture: Every decidable set in PttR is in P/poly.
  • What can we show?
the central conjecture1
The Central Conjecture
  • Conjecture: Every decidable set in PttR is in P/poly.
  • What can we show?
  • We show that a similar statement holds in the context of time-bounded K-complexity.
time bounded k complexity
Time-Bounded K-complexity
  • Let t be a time bound. (Think of t as being large, such as Ackermann’s function.)
  • Define Kt(x) to be min{|d| : U(d) = x in at most t(|x|) steps}.
  • Define RKt to be {x : Kt(x) ≥ |x|}.
  • Define TTRT = {A : A is in PttRKt for all large enough time bounds t}.
  • Vague intuition: Poly-time reductions should not be able to distinguish between RKt and RK, for large t.
the central conjecture2
The Central Conjecture
  • Conjecture: Every decidable set in PttR is in P/poly.
  • We show that a similar statement holds in the context of time-bounded K-complexity:
    • TTRT is contained in P/poly [ABFL12].
  • If t(n) = 22n, then RKt is NOT in P/poly.
  • …which supports our “vague intuition”, because this set is not reducible to the time-t’-random strings for t’ >> t.
the central conjecture3
The Central Conjecture
  • Conjecture: Every decidable set in PttR is in P/poly.
  • We show that a similar statement holds in the context of time-bounded K-complexity:
    • TTRT is contained in P/poly [ABFL12].
  • BUT – The same P/poly bound holds, even if we consider PRKt instead of PttRKt.
  • …and recall PSPACE is contained in PR.
  • So the “vague intuition” is wrong!
slide29

The Central Conjecture: An Earlier Approach

  • Conjecture: Every decidable set in PttR is in P/poly.
  • We give a proof of a statement of the form:

A

A

jΨ(n,j)

n

such that: if for each n and j

there is a proof in PA of Ψ(n,j)

then the conjecture holds.

basic proof theory
Basic Proof Theory
  • Recall that Peano Arithmetic cannot prove the statement “PA is consistent”.
  • Let PA1 be PA + “PA is consistent”.
  • Similarly, one can define PA2, PA3, …
  • “PA is consistent” can be formulated as “for all j, there is no length j proof of 0=1”.
  • For each j, PA can prove “there is no length j proof of 0=1”.
the central conjecture an earlier approach
The Central Conjecture: An Earlier Approach
  • Conjecture: Every decidable set in PttR is in P/poly.
  • We give a proof (in PA1) of a statement of the form:

A

A

jΨ(n,j)

n

such that: if for each n and j

there is a proof in PA of Ψ(n,j)

then the conjecture holds.

the central conjecture the earlier approach fails
The Central Conjecture: The Earlier Approach Fails
  • The connections to proof theory were unexpected and intriguing, and seemed promising…
  • But unfortunately, it turns out that many of the statements Ψ(n,j) are independent of PA (and a related approach yields statements Ψ(n,j,k) that are independent of each system PAr).
a high level view of the earlier approach
A High-Level View of the “Earlier Approach”
  • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then there is a d such that for all x,

there is a V containing only strings of length at most d+log f(|x|), such that

MV(x) = A(x).

Note: V says “long queries are non-random”.

a warm up
A Warm-Up
  • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then there is a d such that for all x,

there is a V containing only strings of length at most d+log f(|x|), such that

MV(x) = A(x).

Note: If some V works for all x of length n, then A is in P/poly.

proof
Proof
  • Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x).
  • Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x).
  • This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1)
proof1
Proof
  • Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x).
  • Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x).
  • This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|).
  • Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x)
proof2
Proof
  • Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x).
  • Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x).
  • This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|).
  • Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x) = A(x). Contradiction!
cleaning things up
Cleaning Things Up
  • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then there is a d such that for all x,

there is a V containing only strings of length at most d+log f(|x|) gA(|x|), such that

MV(x) = A(x).

cleaning things up1
Cleaning Things Up
  • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then there is a d such that for all x,

there is a V containing only strings of length at most d+log f(|x|) gA(|x|), such that

MV(x) = A(x).

cleaning things up2
Cleaning Things Up
  • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then for all x,

there is a V containing only strings in R of length at most gA(|x|) such that

MV(x) = A(x).

a refinement
A Refinement
  • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then for all x,

there is a V containing only strings in R of length at most gA(|x|) such that

MV(x) = A(x).

approximating r
Approximating R
  • We can obtain a series of approximations to R (up to length gA(n)) as follows:
  • Rn,0 = all strings of length at most gA(n).
  • Rn,i+1 = Rn,i minus the i+1st string of length at most gA(n) that is found, in an enumeration of non-random strings.
  • Rn,0, Rn,1, Rn,2, … Rn,i* = R∩{0,1}gA(n)
a refinement1
A Refinement
  • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then for all xє{0,1}n, for all i,

there is a V containing only strings in Rn,i such that

MV(x) = A(x).

proof3
Proof
  • Assume that for each d there is some x,i such that, for all V containing strings inRn,iof length at most d+log f(|x|), MV(x)≠A(x).
  • Consider the machine that takes input (d,r) and finds x,i (as above) and outputs the rth element of Q(x).
  • This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|).
  • Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x) = A(x). Contradiction!
where does pa come in
Where does PA come in??
  • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then for all xє{0,1}n, for all i,

there is a V containing only strings in Rn,i such that

MV(x) = A(x)

and there is not a length-k proof that “for all i, V is not equal to Rn,i”.

what went wrong with the earlier approach
What went wrong with the earlier approach.
  • We have shown: For all xє{0,1}n, for all i, there is a V containing only short strings in Rn,i such that MV(x) = A(x).
  • We were aiming at showing that one can swap the quantifiers, so that for all n, there is a V containing only short strings in Rn,i such that, for all x of length n, MV(x) = A(x).
  • But there is a (useless) reduction M for which this is false. (M already knows the outcome of its queries, assuming that the oracle is R.)
open questions
Open Questions:
  • Decrease the gap (NEXP vs EXPSPACE) between the lower and upper bounds on the complexity of the problems that are in NPRK for every U.
  • Some of our proofs rely on using RK. Do similar results hold also for RC?
    • Disprove: The halting problem is in PttRC.
  • Can the PSPACE ∩ P/poly bound (in the time-bounded setting) be improved to BPP?
  • Is this approach relevant at all to the P=BPP question?
p vs bpp
P vs BPP
  • Our main intuition for P=BPP comes from [Impagliazzo, Wigderson]. Circuit lower bounds imply derandomization.
  • Note that this provides much more than “merely” P=BPP; it gives a recipe for simulating any probabilistic algorithm.
  • Goldreich has argued that any proof of P=BPP actually yields pseudorandom generators (and hence a “recipe” as above)…
    • …but this has only been proved for the “promise problem” formulation of P=BPP.
p vs bpp1
P vs BPP
  • Recall that TTRT sits between BPP and PSPACE ∩ P/poly.
  • A proof that TTRT = P would show that BPP = P – but it is not clear that this would yield any sort of recipe for constructing useful pseudorandom generators.
  • Although it would be a less “useful” approach, perhaps it might be an easier approach?