1 / 77

The Strange Link between Incompressibility and Complexity

The Strange Link between Incompressibility and Complexity. China Theory Week, Aarhus August 13, 2012. Today’s Goal:. To present new developments in a line of research dating back to 2002, presenting some unexpected connections between Kolmogorov Complexity (the theory of randomness), and

tuvya
Download Presentation

The Strange Link between Incompressibility and Complexity

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. The Strange Link between Incompressibility and Complexity China Theory Week, Aarhus August 13, 2012

  2. Today’s Goal: • To present new developments in a line of research dating back to 2002, presenting some unexpected connections between • Kolmogorov Complexity (the theory of randomness), and • Computational Complexity Theory • Which ought to have nothing to do with each other!

  3. Complexity Classes EXPSPACE NEXP PSPACE NP P/poly BPP P

  4. A Jewel of Derandomization • [Impagliazzo, Wigderson, 1997]: If there is a problem computable in time 2n that requires circuits of size 2εn, then P = BPP.

  5. Randomness • Which of these sequences did I obtain by flipping a coin? • 0101010101010101010101010101010 • 0110100111111100111000100101100 • 1101010001011100111111011001010 • 1/π=0.3183098861837906715377675267450 • Each of these sequences is equally likely, in the sense of probability theory – which does not provide us with a way to talk about “randomness”.

  6. Randomness • Which of these sequences did I obtain by flipping a coin? • 0101010101010101010101010101010 • 0110100111111100111000100101100 • 1101010001011100111111011001010 • 1/π=0.3183098861837906715377675267450 • How many bits of information does each of these sequences contain?

  7. Kolmogorov Complexity C(x) = min{|d| : U(d) = x} U is a “universal” Turing machine K(x) = min{|d| : U(d) = x} U is a “universal” prefix-free Turing machine Important property Invariance: The choice of the universal Turing machine U is unimportant (up to an additive constant). x is random if C(x) ≥ |x|.

  8. Kolmogorov Complexity C(x) = min{|d| : U(d) = x} U is a “universal” Turing machine K(x) = min{|d| : U(d) = x} U is a “universal” prefix-free Turing machine Important property Invariance: The choice of the universal Turing machine U is unimportant (up to an additive constant). x is random if C(x) ≥ |x|, or K(x) ≥ |x|.

  9. K, C, and Randomness • K(x) and C(x) are “close”: • C(x) ≤ K(x) ≤ C(x) + 2 log |x| • Two notions of randomness: • RC = {x : C(x) ≥ |x|} • RK = {x : K(x) ≥ |x|} • …actually, infinitely many notions of randomness: • RCU = {x : CU(x) ≥ |x|}

  10. K, C, and Randomness • K(x) and C(x) are “close”: • C(x) ≤ K(x) ≤ C(x) + 2 log |x| • Two notions of randomness: • RC = {x : C(x) ≥ |x|} • RK = {x : K(x) ≥ |x|} • …actually, infinitely many notions of randomness: • RCU = {x : CU(x) ≥ |x|}, RKU = {x : KU(x) ≥ |x|}

  11. K, C, and Randomness • When it makes no difference, we’ll write “R” instead of RC or RK. • Basic facts: • R is undecidable • …but it is not “easy” to use it as an oracle. • R is not NP-hard under poly-time ≤m reductions, unless P=NP. • Things get more interesting when we consider more powerful types of reducibility.

  12. Three Bizarre Inclusions • NEXP is contained in NPR. [ABK06] • PSPACE is contained in PR. [ABKMR06] • BPP is contained in {A : A is poly-time ≤tt R}. [BFKL10] • A ≤tt reduction is a “non-adaptive” reduction. • On input x, a list of queries is formulated before receiving any answer from the oracle.

  13. Three Bizarre Inclusions • NEXP is contained in NPR. [ABK06] • PSPACE is contained in PR. [ABKMR06] • BPP is contained in PttR. [BFKL10] “Bizarre”, because a non-computable “upper bound” is presented on complexity classes! We have been unable to squeeze larger complexity classes inside. Are these containments optimal?

  14. Three Bizarre Inclusions • NEXP is contained in NPR. [ABK06] • PSPACE is contained in PR. [ABKMR06] • BPP is contained in PttR. [BFKL10] “Bizarre”, because a non-computable “upper bound” is presented on complexity classes! If we restrict attention to RK, then we can do better…

  15. Three Bizarre Inclusions • NEXP is contained in NPRK. • The decidable sets that are in NPRK for every U are in EXPSPACE. [AFG11] • PSPACE is contained in PRK. • BPP is contained in PttRK. • The decidable sets that are in PttRK for every U are in PSPACE. [AFG11]

  16. Three Bizarre Inclusions • NEXP is contained in NPRK (for every U). • The decidable sets that are in NPRK for every U are in EXPSPACE. [AFG11] • PSPACE is contained in PRK (for every U). • BPP is contained in PttRK (for every U). • The decidable sets that are in PttRK for every U are in PSPACE. [AFG11] • [CELM] The sets that are in PttRK for every U are decidable.

  17. Three Bizarre Inclusions • NEXP is contained in NPRK (for every U). • The decidable sets that are in NPRK for every U are in EXPSPACE. [AFG11] • PSPACE is contained in PRK (for every U). • BPP is contained in PttRK (for every U). • The sets that are in PttRK for every U are in PSPACE. [AFG11]

  18. Three Bizarre Inclusions • NEXP is contained in NPRK (for every U). • The sets that are in NPRK for every U are in EXPSPACE. [AFG11] • PSPACE is contained in PRK (for every U). • BPP is contained in PttRK (for every U). • The sets that are in PttRK for every U are in PSPACE. [AFG11]

  19. Three Bizarre Inclusions • NEXP is contained in NPRK (for every U). • The sets that are in NPRK for every U are in EXPSPACE. [AFG11] • Conjecture: This should hold for RC, too. • BPP is contained in PttRK (for every U). • The sets that are in PttRK for every U are in PSPACE. [AFG11]

  20. Three Bizarre Inclusions • NEXP is contained in NPRK (for every U). • The sets that are in NPRK for every U are in EXPSPACE. [AFG11] • This holds even for sets in EXPttRK for all U! • BPP is contained in PttRK (for every U). • The sets that are in PttRK for every U are in PSPACE. [AFG11]

  21. Three Bizarre Inclusions • NEXP is contained in NPRK (for every U). • The sets that are in NPRK for every U are in EXPSPACE. [AFG11] • Conjecture: This class is exactly NEXP. • BPP is contained in PttRK (for every U). • The sets that are in PttRK for every U are in PSPACE. [AFG11] Conjecture: This class is exactly BPP.

  22. Three Bizarre Inclusions • NEXP is contained in NPRK (for every U). • The sets that are in NPRK for every U are in EXPSPACE. [AFG11] • Conjecture: This class is exactly NEXP. • BPP is contained in PttRK (for every U). • The sets that are in PttRK for every U are in PSPACE. [AFG11] Conjecture: This class is exactly BPP P.

  23. K-Complexity and BPP vs P • BPP is contained in PttRK (for every U). • The sets that are in PttRK for every U are in PSPACE. Conjecture: This class is exactly P. • Some support for this conjecture [ABK06]: • The decidable sets that are in PdttRK for every U are in P. • The decidable sets that are in Pparity-ttRK for every U are in P.

  24. K-Complexity and BPP vs P • BPP is contained in PttRK (for every U). • The sets that are in PttRK for every U are in PSPACE. Conjecture: This class is exactly P. • New results support a weaker conjecture: • Conjecture: This class is contained in PSPACE ∩ P/poly. • More strongly: Every decidable set in PttR is in P/poly.

  25. K-Complexity and BPP vs P • BPP is contained in PttRK (for every U). • The sets that are in PttRK for every U are in PSPACE. Conjecture: This class is exactly P. • New results support a weaker conjecture : • Conjecture: This class is contained in PSPACE ∩ P/poly. • More strongly: Every decidable set in PttR is in P/poly (i.e., for every U, and for both C and K).

  26. The Central Conjecture • Conjecture: Every decidable set in PttR is in P/poly. • What can we show?

  27. The Central Conjecture • Conjecture: Every decidable set in PttR is in P/poly. • What can we show? • We show that a similar statement holds in the context of time-bounded K-complexity.

  28. Time-Bounded K-complexity • Let t be a time bound. (Think of t as being large, such as Ackermann’s function.) • Define Kt(x) to be min{|d| : U(d) = x in at most t(|x|) steps}. • Define RKt to be {x : Kt(x) ≥ |x|}. • Define TTRT = {A : A is in PttRKt for all large enough time bounds t}. • Vague intuition: Poly-time reductions should not be able to distinguish between RKt and RK, for large t.

  29. The Central Conjecture • Conjecture: Every decidable set in PttR is in P/poly. • We show that a similar statement holds in the context of time-bounded K-complexity: • TTRT is contained in P/poly [ABFL12]. • If t(n) = 22n, then RKt is NOT in P/poly. • …which supports our “vague intuition”, because this set is not reducible to the time-t’-random strings for t’ >> t.

  30. The Central Conjecture • Conjecture: Every decidable set in PttR is in P/poly. • We show that a similar statement holds in the context of time-bounded K-complexity: • TTRT is contained in P/poly [ABFL12]. • BUT – The same P/poly bound holds, even if we consider PRKt instead of PttRKt. • …and recall PSPACE is contained in PR. • So the “vague intuition” is wrong!

  31. The Central Conjecture: An Earlier Approach • Conjecture: Every decidable set in PttR is in P/poly. • We give a proof of a statement of the form: A A jΨ(n,j) n such that: if for each n and j there is a proof in PA of Ψ(n,j) then the conjecture holds.

  32. Basic Proof Theory • Recall that Peano Arithmetic cannot prove the statement “PA is consistent”. • Let PA1 be PA + “PA is consistent”. • Similarly, one can define PA2, PA3, … • “PA is consistent” can be formulated as “for all j, there is no length j proof of 0=1”. • For each j, PA can prove “there is no length j proof of 0=1”.

  33. The Central Conjecture: An Earlier Approach • Conjecture: Every decidable set in PttR is in P/poly. • We give a proof (in PA1) of a statement of the form: A A jΨ(n,j) n such that: if for each n and j there is a proof in PA of Ψ(n,j) then the conjecture holds.

  34. The Central Conjecture: The Earlier Approach Fails • The connections to proof theory were unexpected and intriguing, and seemed promising… • But unfortunately, it turns out that many of the statements Ψ(n,j) are independent of PA (and a related approach yields statements Ψ(n,j,k) that are independent of each system PAr).

  35. A High-Level View of the “Earlier Approach” • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then there is a d such that for all x, there is a V containing only strings of length at most d+log f(|x|), such that MV(x) = A(x). Note: V says “long queries are non-random”.

  36. A Warm-Up • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then there is a d such that for all x, there is a V containing only strings of length at most d+log f(|x|), such that MV(x) = A(x). Note: If some V works for all x of length n, then A is in P/poly.

  37. Proof • Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x). • Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x). • This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1)

  38. Proof • Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x). • Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x). • This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|). • Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x)

  39. Proof • Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x). • Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x). • This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|). • Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x) = A(x). Contradiction!

  40. Cleaning Things Up • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then there is a d such that for all x, there is a V containing only strings of length at most d+log f(|x|) gA(|x|), such that MV(x) = A(x).

  41. Cleaning Things Up • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then there is a d such that for all x, there is a V containing only strings of length at most d+log f(|x|) gA(|x|), such that MV(x) = A(x).

  42. Cleaning Things Up • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then for all x, there is a V containing only strings in R of length at most gA(|x|) such that MV(x) = A(x).

  43. A Refinement • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then for all x, there is a V containing only strings in R of length at most gA(|x|) such that MV(x) = A(x).

  44. Approximating R • We can obtain a series of approximations to R (up to length gA(n)) as follows: • Rn,0 = all strings of length at most gA(n). • Rn,i+1 = Rn,i minus the i+1st string of length at most gA(n) that is found, in an enumeration of non-random strings. • Rn,0, Rn,1, Rn,2, … Rn,i* = R∩{0,1}gA(n)

  45. A Refinement • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then for all xє{0,1}n, for all i, there is a V containing only strings in Rn,i such that MV(x) = A(x).

  46. Proof • Assume that for each d there is some x,i such that, for all V containing strings inRn,iof length at most d+log f(|x|), MV(x)≠A(x). • Consider the machine that takes input (d,r) and finds x,i (as above) and outputs the rth element of Q(x). • This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|). • Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x) = A(x). Contradiction!

  47. Where does PA come in?? • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then for all xє{0,1}n, for all i, there is a V containing only strings in Rn,i such that MV(x) = A(x) And for all k, there is not a length-k proof that “for all i, V is not equal to Rn,i”.

  48. What went wrong with the earlier approach. • We have shown: For all xє{0,1}n, for all i, there is a V containing only short strings in Rn,i such that MV(x) = A(x). • We were aiming at showing that one can swap the quantifiers, so that for all n, there is a V containing only short strings in Rn,i such that, for all x of length n, MV(x) = A(x). • But there is a (useless) reduction M for which this is false. (M already knows the outcome of its queries, assuming that the oracle is R.)

  49. Open Questions: • Decrease the gap (NEXP vs EXPSPACE) between the lower and upper bounds on the complexity of the problems that are in NPRK for every U. • Some of our proofs rely on using RK. Do similar results hold also for RC? • Disprove: The halting problem is in PttRC. • Can the PSPACE ∩ P/poly bound (in the time-bounded setting) be improved to BPP? • Is this approach relevant at all to the P=BPP question?

  50. P vs BPP • Our main intuition for P=BPP comes from [Impagliazzo, Wigderson]. Circuit lower bounds imply derandomization. • Note that this provides much more than “merely” P=BPP; it gives a recipe for simulating any probabilistic algorithm. • Goldreich has argued that any proof of P=BPP actually yields pseudorandom generators (and hence a “recipe” as above)… • …but this has only been proved for the “promise problem” formulation of P=BPP.

More Related