Loading in 2 Seconds...

Curiouser and Curiouser : The Link between Incompressibility and Complexity

Loading in 2 Seconds...

- 61 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about 'Curiouser and Curiouser : The Link between Incompressibility and Complexity' - tilden

Download Now**An Image/Link below is provided (as is) to download presentation**

Download Now

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### Curiouser and Curiouser: The Link between Incompressibility and Complexity

CiE Special Session, June 19, 2012

Today’s Goal:

- To present new developments in a line of research dating back to 2002, presenting some unexpected connections between
- Kolmogorov Complexity (the theory of randomness), and
- Computational Complexity Theory
- Which ought to have nothing to do with each other!

A Jewel of Derandomization

- [Impagliazzo, Wigderson, 1997]: If there is a problem computable in time 2n that requires circuits of size 2εn, then P = BPP.

Kolmogorov Complexity

C(x) = min{|d| : U(d) = x}

U is a “universal” Turing machine

K(x) = min{|d| : U(d) = x}

U is a “universal” prefix-free Turing machine

Important property

Invariance: The choice of the universal Turing machine U is unimportant (up to an additive constant).

x is random if C(x) ≥ |x|.

Kolmogorov Complexity

C(x) = min{|d| : U(d) = x}

U is a “universal” Turing machine

K(x) = min{|d| : U(d) = x}

U is a “universal” prefix-free Turing machine

Important property

Invariance: The choice of the universal Turing machine U is unimportant (up to an additive constant).

x is random if C(x) ≥ |x|, or K(x) ≥ |x|.

K, C, and Randomness

- K(x) and C(x) are “close”:
- C(x) ≤ K(x) ≤ C(x) + 2 log |x|
- Two notions of randomness:
- RC = {x : C(x) ≥ |x|}
- RK = {x : K(x) ≥ |x|}
- …actually, infinitely many notions of randomness:
- RCU = {x : CU(x) ≥ |x|}

K, C, and Randomness

- K(x) and C(x) are “close”:
- C(x) ≤ K(x) ≤ C(x) + 2 log |x|
- Two notions of randomness:
- RC = {x : C(x) ≥ |x|}
- RK = {x : K(x) ≥ |x|}
- …actually, infinitely many notions of randomness:
- RCU = {x : CU(x) ≥ |x|}, RKU = {x : KU(x) ≥ |x|}

K, C, and Randomness

- When it makes no difference, we’ll write “R” instead of RC or RK.
- Basic facts:
- R is undecidable
- …but it is not “easy” to use it as an oracle.
- R is not NP-hard under poly-time ≤m reductions, unless P=NP.
- Things get more interesting when we consider more powerful types of reducibility.

Three Bizarre Inclusions

- NEXP is contained in NPR. [ABK06]
- PSPACE is contained in PR. [ABKMR06]
- BPP is contained in {A : A is poly-time ≤tt R}. [BFKL10]
- A ≤tt reduction is a “non-adaptive” reduction.
- On input x, a list of queries is formulated before receiving any answer from the oracle.

Three Bizarre Inclusions

- NEXP is contained in NPR. [ABK06]
- PSPACE is contained in PR. [ABKMR06]
- BPP is contained in PttR. [BFKL10]

“Bizarre”, because a non-computable “upper bound” is presented on complexity classes!

We have been unable to squeeze larger complexity classes inside. Are these containments optimal?

Three Bizarre Inclusions

- NEXP is contained in NPR. [ABK06]
- PSPACE is contained in PR. [ABKMR06]
- BPP is contained in PttR. [BFKL10]

“Bizarre”, because a non-computable “upper bound” is presented on complexity classes!

If we restrict attention to RK, then we can do better…

Three Bizarre Inclusions

- NEXP is contained in NPRK.
- The decidable sets that are in NPRK for every U are in EXPSPACE. [AFG11]
- PSPACE is contained in PRK.
- BPP is contained in PttRK.
- The decidable sets that are in PttRK for every U are in PSPACE. [AFG11]

Three Bizarre Inclusions

- NEXP is contained in NPRK (for every U).
- The decidable sets that are in NPRK for every U are in EXPSPACE. [AFG11]
- PSPACE is contained in PRK (for every U).
- BPP is contained in PttRK (for every U).
- The decidable sets that are in PttRK for every U are in PSPACE. [AFG11]
- [CELM] The sets that are in PttRK for every U are decidable.

Three Bizarre Inclusions

- NEXP is contained in NPRK (for every U).
- The decidable sets that are in NPRK for every U are in EXPSPACE. [AFG11]
- PSPACE is contained in PRK (for every U).
- BPP is contained in PttRK (for every U).
- The sets that are in PttRK for every U are in PSPACE. [AFG11]

Three Bizarre Inclusions

- NEXP is contained in NPRK (for every U).
- The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
- PSPACE is contained in PRK (for every U).
- BPP is contained in PttRK (for every U).
- The sets that are in PttRK for every U are in PSPACE. [AFG11]

Three Bizarre Inclusions

- NEXP is contained in NPRK (for every U).
- The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
- Conjecture: This should hold for RC, too.
- BPP is contained in PttRK (for every U).
- The sets that are in PttRK for every U are in PSPACE. [AFG11]

Three Bizarre Inclusions

- NEXP is contained in NPRK (for every U).
- The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
- This holds even for sets in EXPttRK for all U!
- BPP is contained in PttRK (for every U).
- The sets that are in PttRK for every U are in PSPACE. [AFG11]

Three Bizarre Inclusions

- NEXP is contained in NPRK (for every U).
- The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
- Conjecture: This class is exactly NEXP.
- BPP is contained in PttRK (for every U).
- The sets that are in PttRK for every U are in PSPACE. [AFG11]

Conjecture: This class is exactly BPP.

Three Bizarre Inclusions

- NEXP is contained in NPRK (for every U).
- The sets that are in NPRK for every U are in EXPSPACE. [AFG11]
- Conjecture: This class is exactly NEXP.
- BPP is contained in PttRK (for every U).
- The sets that are in PttRK for every U are in PSPACE. [AFG11]

Conjecture: This class is exactly BPP P.

K-Complexity and BPP vs P

- BPP is contained in PttRK (for every U).
- The sets that are in PttRK for every U are in PSPACE.

Conjecture: This class is exactly P.

- Some support for this conjecture [ABK06]:
- The decidable sets that are in PdttRC for every U are in P.
- The decidable sets that are in Pparity-ttRC for every U are in P.

K-Complexity and BPP vs P

- BPP is contained in PttRK (for every U).
- The sets that are in PttRK for every U are in PSPACE.

Conjecture: This class is exactly P.

- New results support a weaker conjecture:
- Conjecture: This class is contained in PSPACE ∩ P/poly.
- More strongly: Every decidable set in PttR is in P/poly.

K-Complexity and BPP vs P

- BPP is contained in PttRK (for every U).
- The sets that are in PttRK for every U are in PSPACE.

Conjecture: This class is exactly P.

- New results support a weaker conjecture :
- Conjecture: This class is contained in PSPACE ∩ P/poly.
- More strongly: Every decidable set in PttR is in P/poly (i.e., for every U, and for both C and K).

The Central Conjecture

- Conjecture: Every decidable set in PttR is in P/poly.
- What can we show?

The Central Conjecture

- Conjecture: Every decidable set in PttR is in P/poly.
- What can we show?
- We show that a similar statement holds in the context of time-bounded K-complexity.

Time-Bounded K-complexity

- Let t be a time bound. (Think of t as being large, such as Ackermann’s function.)
- Define Kt(x) to be min{|d| : U(d) = x in at most t(|x|) steps}.
- Define RKt to be {x : Kt(x) ≥ |x|}.
- Define TTRT = {A : A is in PttRKt for all large enough time bounds t}.
- Vague intuition: Poly-time reductions should not be able to distinguish between RKt and RK, for large t.

The Central Conjecture

- Conjecture: Every decidable set in PttR is in P/poly.
- We show that a similar statement holds in the context of time-bounded K-complexity:
- TTRT is contained in P/poly [ABFL12].
- If t(n) = 22n, then RKt is NOT in P/poly.
- …which supports our “vague intuition”, because this set is not reducible to the time-t’-random strings for t’ >> t.

The Central Conjecture

- Conjecture: Every decidable set in PttR is in P/poly.
- We show that a similar statement holds in the context of time-bounded K-complexity:
- TTRT is contained in P/poly [ABFL12].
- BUT – The same P/poly bound holds, even if we consider PRKt instead of PttRKt.
- …and recall PSPACE is contained in PR.
- So the “vague intuition” is wrong!

The Central Conjecture: An Earlier Approach

- Conjecture: Every decidable set in PttR is in P/poly.
- We give a proof of a statement of the form:

A

A

jΨ(n,j)

n

such that: if for each n and j

there is a proof in PA of Ψ(n,j)

then the conjecture holds.

Basic Proof Theory

- Recall that Peano Arithmetic cannot prove the statement “PA is consistent”.
- Let PA1 be PA + “PA is consistent”.
- Similarly, one can define PA2, PA3, …
- “PA is consistent” can be formulated as “for all j, there is no length j proof of 0=1”.
- For each j, PA can prove “there is no length j proof of 0=1”.

The Central Conjecture: An Earlier Approach

- Conjecture: Every decidable set in PttR is in P/poly.
- We give a proof (in PA1) of a statement of the form:

A

A

jΨ(n,j)

n

such that: if for each n and j

there is a proof in PA of Ψ(n,j)

then the conjecture holds.

The Central Conjecture: The Earlier Approach Fails

- The connections to proof theory were unexpected and intriguing, and seemed promising…
- But unfortunately, it turns out that many of the statements Ψ(n,j) are independent of PA (and a related approach yields statements Ψ(n,j,k) that are independent of each system PAr).

A High-Level View of the “Earlier Approach”

- Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then there is a d such that for all x,

there is a V containing only strings of length at most d+log f(|x|), such that

MV(x) = A(x).

Note: V says “long queries are non-random”.

A Warm-Up

- Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then there is a d such that for all x,

there is a V containing only strings of length at most d+log f(|x|), such that

MV(x) = A(x).

Note: If some V works for all x of length n, then A is in P/poly.

Proof

- Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x).
- Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x).
- This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1)

Proof

- Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x).
- Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x).
- This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|).
- Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x)

Proof

- Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x).
- Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x).
- This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|).
- Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x) = A(x). Contradiction!

Cleaning Things Up

- Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|).

Then there is a d such that for all x,

there is a V containing only strings of length at most d+log f(|x|) gA(|x|), such that

MV(x) = A(x).

Cleaning Things Up

Then there is a d such that for all x,

there is a V containing only strings of length at most d+log f(|x|) gA(|x|), such that

MV(x) = A(x).

Cleaning Things Up

Then for all x,

there is a V containing only strings in R of length at most gA(|x|) such that

MV(x) = A(x).

A Refinement

Then for all x,

there is a V containing only strings in R of length at most gA(|x|) such that

MV(x) = A(x).

Approximating R

- We can obtain a series of approximations to R (up to length gA(n)) as follows:
- Rn,0 = all strings of length at most gA(n).
- Rn,i+1 = Rn,i minus the i+1st string of length at most gA(n) that is found, in an enumeration of non-random strings.
- Rn,0, Rn,1, Rn,2, … Rn,i* = R∩{0,1}gA(n)

A Refinement

Then for all xє{0,1}n, for all i,

there is a V containing only strings in Rn,i such that

MV(x) = A(x).

Proof

- Assume that for each d there is some x,i such that, for all V containing strings inRn,iof length at most d+log f(|x|), MV(x)≠A(x).
- Consider the machine that takes input (d,r) and finds x,i (as above) and outputs the rth element of Q(x).
- This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|).
- Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x) = A(x). Contradiction!

Where does PA come in??

Then for all xє{0,1}n, for all i,

there is a V containing only strings in Rn,i such that

MV(x) = A(x)

and there is not a length-k proof that “for all i, V is not equal to Rn,i”.

What went wrong with the earlier approach.

- We have shown: For all xє{0,1}n, for all i, there is a V containing only short strings in Rn,i such that MV(x) = A(x).
- We were aiming at showing that one can swap the quantifiers, so that for all n, there is a V containing only short strings in Rn,i such that, for all x of length n, MV(x) = A(x).
- But there is a (useless) reduction M for which this is false. (M already knows the outcome of its queries, assuming that the oracle is R.)

Open Questions:

- Decrease the gap (NEXP vs EXPSPACE) between the lower and upper bounds on the complexity of the problems that are in NPRK for every U.
- Some of our proofs rely on using RK. Do similar results hold also for RC?
- Disprove: The halting problem is in PttRC.
- Can the PSPACE ∩ P/poly bound (in the time-bounded setting) be improved to BPP?
- Is this approach relevant at all to the P=BPP question?

P vs BPP

- Our main intuition for P=BPP comes from [Impagliazzo, Wigderson]. Circuit lower bounds imply derandomization.
- Note that this provides much more than “merely” P=BPP; it gives a recipe for simulating any probabilistic algorithm.
- Goldreich has argued that any proof of P=BPP actually yields pseudorandom generators (and hence a “recipe” as above)…
- …but this has only been proved for the “promise problem” formulation of P=BPP.

P vs BPP

- Recall that TTRT sits between BPP and PSPACE ∩ P/poly.
- A proof that TTRT = P would show that BPP = P – but it is not clear that this would yield any sort of recipe for constructing useful pseudorandom generators.
- Although it would be a less “useful” approach, perhaps it might be an easier approach?

Download Presentation

Connecting to Server..