Curiouser and Curiouser : The Link between Incompressibility and Complexity

1 / 50

# Curiouser and Curiouser : The Link between Incompressibility and Complexity - PowerPoint PPT Presentation

Curiouser and Curiouser : The Link between Incompressibility and Complexity. CiE Special Session, June 19, 2012. Today’s Goal:. To present new developments in a line of research dating back to 2002, presenting some unexpected connections between

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Curiouser and Curiouser : The Link between Incompressibility and Complexity' - tilden

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Curiouser and Curiouser: The Link between Incompressibility and Complexity

CiE Special Session, June 19, 2012

Today’s Goal:
• To present new developments in a line of research dating back to 2002, presenting some unexpected connections between
• Kolmogorov Complexity (the theory of randomness), and
• Computational Complexity Theory
• Which ought to have nothing to do with each other!
Complexity Classes

EXPSPACE

NEXP

PSPACE

NP

P/poly

BPP

P

A Jewel of Derandomization
• [Impagliazzo, Wigderson, 1997]: If there is a problem computable in time 2n that requires circuits of size 2εn, then P = BPP.
Kolmogorov Complexity

C(x) = min{|d| : U(d) = x}

U is a “universal” Turing machine

K(x) = min{|d| : U(d) = x}

U is a “universal” prefix-free Turing machine

Important property

Invariance: The choice of the universal Turing machine U is unimportant (up to an additive constant).

x is random if C(x) ≥ |x|.

Kolmogorov Complexity

C(x) = min{|d| : U(d) = x}

U is a “universal” Turing machine

K(x) = min{|d| : U(d) = x}

U is a “universal” prefix-free Turing machine

Important property

Invariance: The choice of the universal Turing machine U is unimportant (up to an additive constant).

x is random if C(x) ≥ |x|, or K(x) ≥ |x|.

K, C, and Randomness
• K(x) and C(x) are “close”:
• C(x) ≤ K(x) ≤ C(x) + 2 log |x|
• Two notions of randomness:
• RC = {x : C(x) ≥ |x|}
• RK = {x : K(x) ≥ |x|}
• …actually, infinitely many notions of randomness:
• RCU = {x : CU(x) ≥ |x|}
K, C, and Randomness
• K(x) and C(x) are “close”:
• C(x) ≤ K(x) ≤ C(x) + 2 log |x|
• Two notions of randomness:
• RC = {x : C(x) ≥ |x|}
• RK = {x : K(x) ≥ |x|}
• …actually, infinitely many notions of randomness:
• RCU = {x : CU(x) ≥ |x|}, RKU = {x : KU(x) ≥ |x|}
K, C, and Randomness
• When it makes no difference, we’ll write “R” instead of RC or RK.
• Basic facts:
• R is undecidable
• …but it is not “easy” to use it as an oracle.
• R is not NP-hard under poly-time ≤m reductions, unless P=NP.
• Things get more interesting when we consider more powerful types of reducibility.
Three Bizarre Inclusions
• NEXP is contained in NPR. [ABK06]
• PSPACE is contained in PR. [ABKMR06]
• BPP is contained in {A : A is poly-time ≤tt R}. [BFKL10]
• A ≤tt reduction is a “non-adaptive” reduction.
• On input x, a list of queries is formulated before receiving any answer from the oracle.
Three Bizarre Inclusions
• NEXP is contained in NPR. [ABK06]
• PSPACE is contained in PR. [ABKMR06]
• BPP is contained in PttR. [BFKL10]

“Bizarre”, because a non-computable “upper bound” is presented on complexity classes!

We have been unable to squeeze larger complexity classes inside. Are these containments optimal?

Three Bizarre Inclusions
• NEXP is contained in NPR. [ABK06]
• PSPACE is contained in PR. [ABKMR06]
• BPP is contained in PttR. [BFKL10]

“Bizarre”, because a non-computable “upper bound” is presented on complexity classes!

If we restrict attention to RK, then we can do better…

Three Bizarre Inclusions
• NEXP is contained in NPRK.
• The decidable sets that are in NPRK for every U are in EXPSPACE. [AFG11]
• PSPACE is contained in PRK.
• BPP is contained in PttRK.
• The decidable sets that are in PttRK for every U are in PSPACE. [AFG11]
Three Bizarre Inclusions
• NEXP is contained in NPRK (for every U).
• The decidable sets that are in NPRK for every U are in EXPSPACE. [AFG11]
• PSPACE is contained in PRK (for every U).
• BPP is contained in PttRK (for every U).
• The decidable sets that are in PttRK for every U are in PSPACE. [AFG11]
• [CELM] The sets that are in PttRK for every U are decidable.
Three Bizarre Inclusions
• NEXP is contained in NPRK (for every U).
• The decidable sets that are in NPRK for every U are in EXPSPACE. [AFG11]
• PSPACE is contained in PRK (for every U).
• BPP is contained in PttRK (for every U).
• The sets that are in PttRK for every U are in PSPACE. [AFG11]
Three Bizarre Inclusions
• NEXP is contained in NPRK (for every U).
• The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
• PSPACE is contained in PRK (for every U).
• BPP is contained in PttRK (for every U).
• The sets that are in PttRK for every U are in PSPACE. [AFG11]
Three Bizarre Inclusions
• NEXP is contained in NPRK (for every U).
• The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
• Conjecture: This should hold for RC, too.
• BPP is contained in PttRK (for every U).
• The sets that are in PttRK for every U are in PSPACE. [AFG11]
Three Bizarre Inclusions
• NEXP is contained in NPRK (for every U).
• The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
• This holds even for sets in EXPttRK for all U!
• BPP is contained in PttRK (for every U).
• The sets that are in PttRK for every U are in PSPACE. [AFG11]
Three Bizarre Inclusions
• NEXP is contained in NPRK (for every U).
• The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
• Conjecture: This class is exactly NEXP.
• BPP is contained in PttRK (for every U).
• The sets that are in PttRK for every U are in PSPACE. [AFG11]

Conjecture: This class is exactly BPP.

Three Bizarre Inclusions
• NEXP is contained in NPRK (for every U).
• The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
• Conjecture: This class is exactly NEXP.
• BPP is contained in PttRK (for every U).
• The sets that are in PttRK for every U are in PSPACE. [AFG11]

Conjecture: This class is exactly BPP P.

K-Complexity and BPP vs P
• BPP is contained in PttRK (for every U).
• The sets that are in PttRK for every U are in PSPACE.

Conjecture: This class is exactly P.

• Some support for this conjecture [ABK06]:
• The decidable sets that are in PdttRC for every U are in P.
• The decidable sets that are in Pparity-ttRC for every U are in P.
K-Complexity and BPP vs P
• BPP is contained in PttRK (for every U).
• The sets that are in PttRK for every U are in PSPACE.

Conjecture: This class is exactly P.

• New results support a weaker conjecture:
• Conjecture: This class is contained in PSPACE ∩ P/poly.
• More strongly: Every decidable set in PttR is in P/poly.
K-Complexity and BPP vs P
• BPP is contained in PttRK (for every U).
• The sets that are in PttRK for every U are in PSPACE.

Conjecture: This class is exactly P.

• New results support a weaker conjecture :
• Conjecture: This class is contained in PSPACE ∩ P/poly.
• More strongly: Every decidable set in PttR is in P/poly (i.e., for every U, and for both C and K).
The Central Conjecture
• Conjecture: Every decidable set in PttR is in P/poly.
• What can we show?
The Central Conjecture
• Conjecture: Every decidable set in PttR is in P/poly.
• What can we show?
• We show that a similar statement holds in the context of time-bounded K-complexity.
Time-Bounded K-complexity
• Let t be a time bound. (Think of t as being large, such as Ackermann’s function.)
• Define Kt(x) to be min{|d| : U(d) = x in at most t(|x|) steps}.
• Define RKt to be {x : Kt(x) ≥ |x|}.
• Define TTRT = {A : A is in PttRKt for all large enough time bounds t}.
• Vague intuition: Poly-time reductions should not be able to distinguish between RKt and RK, for large t.
The Central Conjecture
• Conjecture: Every decidable set in PttR is in P/poly.
• We show that a similar statement holds in the context of time-bounded K-complexity:
• TTRT is contained in P/poly [ABFL12].
• If t(n) = 22n, then RKt is NOT in P/poly.
• …which supports our “vague intuition”, because this set is not reducible to the time-t’-random strings for t’ >> t.
The Central Conjecture
• Conjecture: Every decidable set in PttR is in P/poly.
• We show that a similar statement holds in the context of time-bounded K-complexity:
• TTRT is contained in P/poly [ABFL12].
• BUT – The same P/poly bound holds, even if we consider PRKt instead of PttRKt.
• …and recall PSPACE is contained in PR.
• So the “vague intuition” is wrong!
• Conjecture: Every decidable set in PttR is in P/poly.
• We give a proof of a statement of the form:

A

A

jΨ(n,j)

n

such that: if for each n and j

there is a proof in PA of Ψ(n,j)

then the conjecture holds.

Basic Proof Theory
• Recall that Peano Arithmetic cannot prove the statement “PA is consistent”.
• Let PA1 be PA + “PA is consistent”.
• Similarly, one can define PA2, PA3, …
• “PA is consistent” can be formulated as “for all j, there is no length j proof of 0=1”.
• For each j, PA can prove “there is no length j proof of 0=1”.
The Central Conjecture: An Earlier Approach
• Conjecture: Every decidable set in PttR is in P/poly.
• We give a proof (in PA1) of a statement of the form:

A

A

jΨ(n,j)

n

such that: if for each n and j

there is a proof in PA of Ψ(n,j)

then the conjecture holds.

The Central Conjecture: The Earlier Approach Fails
• The connections to proof theory were unexpected and intriguing, and seemed promising…
• But unfortunately, it turns out that many of the statements Ψ(n,j) are independent of PA (and a related approach yields statements Ψ(n,j,k) that are independent of each system PAr).
A High-Level View of the “Earlier Approach”
• Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then there is a d such that for all x,

there is a V containing only strings of length at most d+log f(|x|), such that

MV(x) = A(x).

Note: V says “long queries are non-random”.

A Warm-Up
• Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then there is a d such that for all x,

there is a V containing only strings of length at most d+log f(|x|), such that

MV(x) = A(x).

Note: If some V works for all x of length n, then A is in P/poly.

Proof
• Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x).
• Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x).
• This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1)
Proof
• Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x).
• Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x).
• This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|).
• Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x)
Proof
• Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x).
• Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x).
• This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|).
• Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x) = A(x). Contradiction!
Cleaning Things Up
• Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then there is a d such that for all x,

there is a V containing only strings of length at most d+log f(|x|) gA(|x|), such that

MV(x) = A(x).

Cleaning Things Up
• Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then there is a d such that for all x,

there is a V containing only strings of length at most d+log f(|x|) gA(|x|), such that

MV(x) = A(x).

Cleaning Things Up
• Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then for all x,

there is a V containing only strings in R of length at most gA(|x|) such that

MV(x) = A(x).

A Refinement
• Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then for all x,

there is a V containing only strings in R of length at most gA(|x|) such that

MV(x) = A(x).

Approximating R
• We can obtain a series of approximations to R (up to length gA(n)) as follows:
• Rn,0 = all strings of length at most gA(n).
• Rn,i+1 = Rn,i minus the i+1st string of length at most gA(n) that is found, in an enumeration of non-random strings.
• Rn,0, Rn,1, Rn,2, … Rn,i* = R∩{0,1}gA(n)
A Refinement
• Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then for all xє{0,1}n, for all i,

there is a V containing only strings in Rn,i such that

MV(x) = A(x).

Proof
• Assume that for each d there is some x,i such that, for all V containing strings inRn,iof length at most d+log f(|x|), MV(x)≠A(x).
• Consider the machine that takes input (d,r) and finds x,i (as above) and outputs the rth element of Q(x).
• This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|).
• Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x) = A(x). Contradiction!
Where does PA come in??
• Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then for all xє{0,1}n, for all i,

there is a V containing only strings in Rn,i such that

MV(x) = A(x)

and there is not a length-k proof that “for all i, V is not equal to Rn,i”.

What went wrong with the earlier approach.
• We have shown: For all xє{0,1}n, for all i, there is a V containing only short strings in Rn,i such that MV(x) = A(x).
• We were aiming at showing that one can swap the quantifiers, so that for all n, there is a V containing only short strings in Rn,i such that, for all x of length n, MV(x) = A(x).
• But there is a (useless) reduction M for which this is false. (M already knows the outcome of its queries, assuming that the oracle is R.)
Open Questions:
• Decrease the gap (NEXP vs EXPSPACE) between the lower and upper bounds on the complexity of the problems that are in NPRK for every U.
• Some of our proofs rely on using RK. Do similar results hold also for RC?
• Disprove: The halting problem is in PttRC.
• Can the PSPACE ∩ P/poly bound (in the time-bounded setting) be improved to BPP?
• Is this approach relevant at all to the P=BPP question?
P vs BPP
• Our main intuition for P=BPP comes from [Impagliazzo, Wigderson]. Circuit lower bounds imply derandomization.
• Note that this provides much more than “merely” P=BPP; it gives a recipe for simulating any probabilistic algorithm.
• Goldreich has argued that any proof of P=BPP actually yields pseudorandom generators (and hence a “recipe” as above)…
• …but this has only been proved for the “promise problem” formulation of P=BPP.
P vs BPP
• Recall that TTRT sits between BPP and PSPACE ∩ P/poly.
• A proof that TTRT = P would show that BPP = P – but it is not clear that this would yield any sort of recipe for constructing useful pseudorandom generators.
• Although it would be a less “useful” approach, perhaps it might be an easier approach?