The Strange Link between Incompressibility and Complexity

1. The Strange Link between Incompressibility and Complexity China Theory Week, Aarhus August 13, 2012

2. Today’s Goal: • To present new developments in a line of research dating back to 2002, presenting some unexpected connections between • Kolmogorov Complexity (the theory of randomness), and • Computational Complexity Theory • Which ought to have nothing to do with each other!

3. Complexity Classes EXPSPACE NEXP PSPACE NP P/poly BPP P

4. A Jewel of Derandomization • [Impagliazzo, Wigderson, 1997]: If there is a problem computable in time 2n that requires circuits of size 2εn, then P = BPP.

5. Randomness • Which of these sequences did I obtain by flipping a coin? • 0101010101010101010101010101010 • 0110100111111100111000100101100 • 1101010001011100111111011001010 • 1/π=0.3183098861837906715377675267450 • Each of these sequences is equally likely, in the sense of probability theory – which does not provide us with a way to talk about “randomness”.

6. Randomness • Which of these sequences did I obtain by flipping a coin? • 0101010101010101010101010101010 • 0110100111111100111000100101100 • 1101010001011100111111011001010 • 1/π=0.3183098861837906715377675267450 • How many bits of information does each of these sequences contain?

7. Kolmogorov Complexity C(x) = min{|d| : U(d) = x} U is a “universal” Turing machine K(x) = min{|d| : U(d) = x} U is a “universal” prefix-free Turing machine Important property Invariance: The choice of the universal Turing machine U is unimportant (up to an additive constant). x is random if C(x) ≥ |x|.

8. Kolmogorov Complexity C(x) = min{|d| : U(d) = x} U is a “universal” Turing machine K(x) = min{|d| : U(d) = x} U is a “universal” prefix-free Turing machine Important property Invariance: The choice of the universal Turing machine U is unimportant (up to an additive constant). x is random if C(x) ≥ |x|, or K(x) ≥ |x|.

9. K, C, and Randomness • K(x) and C(x) are “close”: • C(x) ≤ K(x) ≤ C(x) + 2 log |x| • Two notions of randomness: • RC = {x : C(x) ≥ |x|} • RK = {x : K(x) ≥ |x|} • …actually, infinitely many notions of randomness: • RCU = {x : CU(x) ≥ |x|}

10. K, C, and Randomness • K(x) and C(x) are “close”: • C(x) ≤ K(x) ≤ C(x) + 2 log |x| • Two notions of randomness: • RC = {x : C(x) ≥ |x|} • RK = {x : K(x) ≥ |x|} • …actually, infinitely many notions of randomness: • RCU = {x : CU(x) ≥ |x|}, RKU = {x : KU(x) ≥ |x|}

11. K, C, and Randomness • When it makes no difference, we’ll write “R” instead of RC or RK. • Basic facts: • R is undecidable • …but it is not “easy” to use it as an oracle. • R is not NP-hard under poly-time ≤m reductions, unless P=NP. • Things get more interesting when we consider more powerful types of reducibility.

12. Three Bizarre Inclusions • NEXP is contained in NPR. [ABK06] • PSPACE is contained in PR. [ABKMR06] • BPP is contained in {A : A is poly-time ≤tt R}. [BFKL10] • A ≤tt reduction is a “non-adaptive” reduction. • On input x, a list of queries is formulated before receiving any answer from the oracle.

13. Three Bizarre Inclusions • NEXP is contained in NPR. [ABK06] • PSPACE is contained in PR. [ABKMR06] • BPP is contained in PttR. [BFKL10] “Bizarre”, because a non-computable “upper bound” is presented on complexity classes! We have been unable to squeeze larger complexity classes inside. Are these containments optimal?

14. Three Bizarre Inclusions • NEXP is contained in NPR. [ABK06] • PSPACE is contained in PR. [ABKMR06] • BPP is contained in PttR. [BFKL10] “Bizarre”, because a non-computable “upper bound” is presented on complexity classes! If we restrict attention to RK, then we can do better…

15. Three Bizarre Inclusions • NEXP is contained in NPRK. • The decidable sets that are in NPRK for every U are in EXPSPACE. [AFG11] • PSPACE is contained in PRK. • BPP is contained in PttRK. • The decidable sets that are in PttRK for every U are in PSPACE. [AFG11]

16. Three Bizarre Inclusions • NEXP is contained in NPRK (for every U). • The decidable sets that are in NPRK for every U are in EXPSPACE. [AFG11] • PSPACE is contained in PRK (for every U). • BPP is contained in PttRK (for every U). • The decidable sets that are in PttRK for every U are in PSPACE. [AFG11] • [CELM] The sets that are in PttRK for every U are decidable.

17. Three Bizarre Inclusions • NEXP is contained in NPRK (for every U). • The decidable sets that are in NPRK for every U are in EXPSPACE. [AFG11] • PSPACE is contained in PRK (for every U). • BPP is contained in PttRK (for every U). • The sets that are in PttRK for every U are in PSPACE. [AFG11]

18. Three Bizarre Inclusions • NEXP is contained in NPRK (for every U). • The sets that are in NPRK for every U are in EXPSPACE. [AFG11] • PSPACE is contained in PRK (for every U). • BPP is contained in PttRK (for every U). • The sets that are in PttRK for every U are in PSPACE. [AFG11]

19. Three Bizarre Inclusions • NEXP is contained in NPRK (for every U). • The sets that are in NPRK for every U are in EXPSPACE. [AFG11] • Conjecture: This should hold for RC, too. • BPP is contained in PttRK (for every U). • The sets that are in PttRK for every U are in PSPACE. [AFG11]

20. Three Bizarre Inclusions • NEXP is contained in NPRK (for every U). • The sets that are in NPRK for every U are in EXPSPACE. [AFG11] • This holds even for sets in EXPttRK for all U! • BPP is contained in PttRK (for every U). • The sets that are in PttRK for every U are in PSPACE. [AFG11]

21. Three Bizarre Inclusions • NEXP is contained in NPRK (for every U). • The sets that are in NPRK for every U are in EXPSPACE. [AFG11] • Conjecture: This class is exactly NEXP. • BPP is contained in PttRK (for every U). • The sets that are in PttRK for every U are in PSPACE. [AFG11] Conjecture: This class is exactly BPP.

22. Three Bizarre Inclusions • NEXP is contained in NPRK (for every U). • The sets that are in NPRK for every U are in EXPSPACE. [AFG11] • Conjecture: This class is exactly NEXP. • BPP is contained in PttRK (for every U). • The sets that are in PttRK for every U are in PSPACE. [AFG11] Conjecture: This class is exactly BPP P.

23. K-Complexity and BPP vs P • BPP is contained in PttRK (for every U). • The sets that are in PttRK for every U are in PSPACE. Conjecture: This class is exactly P. • Some support for this conjecture [ABK06]: • The decidable sets that are in PdttRK for every U are in P. • The decidable sets that are in Pparity-ttRK for every U are in P.

24. K-Complexity and BPP vs P • BPP is contained in PttRK (for every U). • The sets that are in PttRK for every U are in PSPACE. Conjecture: This class is exactly P. • New results support a weaker conjecture: • Conjecture: This class is contained in PSPACE ∩ P/poly. • More strongly: Every decidable set in PttR is in P/poly.

25. K-Complexity and BPP vs P • BPP is contained in PttRK (for every U). • The sets that are in PttRK for every U are in PSPACE. Conjecture: This class is exactly P. • New results support a weaker conjecture : • Conjecture: This class is contained in PSPACE ∩ P/poly. • More strongly: Every decidable set in PttR is in P/poly (i.e., for every U, and for both C and K).

26. The Central Conjecture • Conjecture: Every decidable set in PttR is in P/poly. • What can we show?

27. The Central Conjecture • Conjecture: Every decidable set in PttR is in P/poly. • What can we show? • We show that a similar statement holds in the context of time-bounded K-complexity.

28. Time-Bounded K-complexity • Let t be a time bound. (Think of t as being large, such as Ackermann’s function.) • Define Kt(x) to be min{|d| : U(d) = x in at most t(|x|) steps}. • Define RKt to be {x : Kt(x) ≥ |x|}. • Define TTRT = {A : A is in PttRKt for all large enough time bounds t}. • Vague intuition: Poly-time reductions should not be able to distinguish between RKt and RK, for large t.

29. The Central Conjecture • Conjecture: Every decidable set in PttR is in P/poly. • We show that a similar statement holds in the context of time-bounded K-complexity: • TTRT is contained in P/poly [ABFL12]. • If t(n) = 22n, then RKt is NOT in P/poly. • …which supports our “vague intuition”, because this set is not reducible to the time-t’-random strings for t’ >> t.

30. The Central Conjecture • Conjecture: Every decidable set in PttR is in P/poly. • We show that a similar statement holds in the context of time-bounded K-complexity: • TTRT is contained in P/poly [ABFL12]. • BUT – The same P/poly bound holds, even if we consider PRKt instead of PttRKt. • …and recall PSPACE is contained in PR. • So the “vague intuition” is wrong!

31. The Central Conjecture: An Earlier Approach • Conjecture: Every decidable set in PttR is in P/poly. • We give a proof of a statement of the form: A A jΨ(n,j) n such that: if for each n and j there is a proof in PA of Ψ(n,j) then the conjecture holds.

32. Basic Proof Theory • Recall that Peano Arithmetic cannot prove the statement “PA is consistent”. • Let PA1 be PA + “PA is consistent”. • Similarly, one can define PA2, PA3, … • “PA is consistent” can be formulated as “for all j, there is no length j proof of 0=1”. • For each j, PA can prove “there is no length j proof of 0=1”.

33. The Central Conjecture: An Earlier Approach • Conjecture: Every decidable set in PttR is in P/poly. • We give a proof (in PA1) of a statement of the form: A A jΨ(n,j) n such that: if for each n and j there is a proof in PA of Ψ(n,j) then the conjecture holds.

34. The Central Conjecture: The Earlier Approach Fails • The connections to proof theory were unexpected and intriguing, and seemed promising… • But unfortunately, it turns out that many of the statements Ψ(n,j) are independent of PA (and a related approach yields statements Ψ(n,j,k) that are independent of each system PAr).

35. A High-Level View of the “Earlier Approach” • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then there is a d such that for all x, there is a V containing only strings of length at most d+log f(|x|), such that MV(x) = A(x). Note: V says “long queries are non-random”.

36. A Warm-Up • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then there is a d such that for all x, there is a V containing only strings of length at most d+log f(|x|), such that MV(x) = A(x). Note: If some V works for all x of length n, then A is in P/poly.

37. Proof • Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x). • Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x). • This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1)

38. Proof • Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x). • Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x). • This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|). • Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x)

39. Proof • Assume that for each d there is some x such that, for all V containing strings of length at most d+log f(|x|), MV(x)≠A(x). • Consider the machine that takes input (d,r) and finds x (as above) and outputs the rth element of Q(x). • This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|). • Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x) = A(x). Contradiction!

40. Cleaning Things Up • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then there is a d such that for all x, there is a V containing only strings of length at most d+log f(|x|) gA(|x|), such that MV(x) = A(x).

41. Cleaning Things Up • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then there is a d such that for all x, there is a V containing only strings of length at most d+log f(|x|) gA(|x|), such that MV(x) = A(x).

42. Cleaning Things Up • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then for all x, there is a V containing only strings in R of length at most gA(|x|) such that MV(x) = A(x).

43. A Refinement • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then for all x, there is a V containing only strings in R of length at most gA(|x|) such that MV(x) = A(x).

44. Approximating R • We can obtain a series of approximations to R (up to length gA(n)) as follows: • Rn,0 = all strings of length at most gA(n). • Rn,i+1 = Rn,i minus the i+1st string of length at most gA(n) that is found, in an enumeration of non-random strings. • Rn,0, Rn,1, Rn,2, … Rn,i* = R∩{0,1}gA(n)

45. A Refinement • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then for all xє{0,1}n, for all i, there is a V containing only strings in Rn,i such that MV(x) = A(x).

46. Proof • Assume that for each d there is some x,i such that, for all V containing strings inRn,iof length at most d+log f(|x|), MV(x)≠A(x). • Consider the machine that takes input (d,r) and finds x,i (as above) and outputs the rth element of Q(x). • This shows that each element y of Q(x) has C(y) ≤ log d + log f(|x|) + O(1) < d + log f(|x|). • Thus if we pick V* to be R∩{0,1}d+log f(|x|), we see that MV*(x) = MR(x) = A(x). Contradiction!

47. Where does PA come in?? • Let A be decidable, and let M be a poly-time machine computing a ≤tt-reduction from A to R. Let Q(x) be the set of queries that M asks on input x. Let the size of Q(x) be at most f(|x|). Then for all xє{0,1}n, for all i, there is a V containing only strings in Rn,i such that MV(x) = A(x) And for all k, there is not a length-k proof that “for all i, V is not equal to Rn,i”.

48. What went wrong with the earlier approach. • We have shown: For all xє{0,1}n, for all i, there is a V containing only short strings in Rn,i such that MV(x) = A(x). • We were aiming at showing that one can swap the quantifiers, so that for all n, there is a V containing only short strings in Rn,i such that, for all x of length n, MV(x) = A(x). • But there is a (useless) reduction M for which this is false. (M already knows the outcome of its queries, assuming that the oracle is R.)

49. Open Questions: • Decrease the gap (NEXP vs EXPSPACE) between the lower and upper bounds on the complexity of the problems that are in NPRK for every U. • Some of our proofs rely on using RK. Do similar results hold also for RC? • Disprove: The halting problem is in PttRC. • Can the PSPACE ∩ P/poly bound (in the time-bounded setting) be improved to BPP? • Is this approach relevant at all to the P=BPP question?

50. P vs BPP • Our main intuition for P=BPP comes from [Impagliazzo, Wigderson]. Circuit lower bounds imply derandomization. • Note that this provides much more than “merely” P=BPP; it gives a recipe for simulating any probabilistic algorithm. • Goldreich has argued that any proof of P=BPP actually yields pseudorandom generators (and hence a “recipe” as above)… • …but this has only been proved for the “promise problem” formulation of P=BPP.