Slides: Asaf Shapira & Oded Schwartz; Sonny Ben-Shimon & Yaniv Nahum.

1. Space Complexity Slides: Asaf Shapira & Oded Schwartz; Sonny Ben-Shimon & Yaniv Nahum. Notes: Leia Passoni, Reuben Sumner, Yoad Lustig & Tal Hassner. (from Oded Goldreich’s course lecture notes)

2. Introduction In this lecture we’ll cover: Space Complexity Non-Deterministic space

3. Complexity Functions 4.1 Definition: A function f is called constructible if it satisfies the following conditions: • Positive: f: N+  N+ • Monotone: f(n+1) f(n) for all n. • Constructible: there exists a Turing Machine Mf that, on input x, outputs a string of size f(|x|), in time O(|x|+f(|x|)), and in space O(f(|x|)).

4. Constructible functions 4.2 • Many “popular” complexity functions such as n, log(n), n2, n! satisfy the above criteria. • Odd situations may occur in regard to relations between complexity classes if we don't choose these functions properly. • Note: We will therefore use time constructible functions for time bound and space constructible functions for space bound.

5. Space Complexity - The model: 4.3 3-tape Turing machine: • Input Tape – Read Only. • Output tape – Write only. Omitted for decision problems. Usually unidirectional. • Work tape – Read & Write. Enables sub linear space Whose length corresponds to space usage

6. What kind of TM should we use? • Any multi-tape TM can be simulated by an ordinary TM with a polynomial loss of efficiency. • Due to the previous fact, from here on, a TM will refer to the 3 tape TM described above.

7. Dspace - Definition For every TM, M, and input x: WM(x) = The index of the rightmost cell on the worktape scanned by M on x. SM(n) = max|x|=n WM(x) cL(x) = 1 if xL, 0 otherwise. Dspace(S(n)) = {L| DTM M, x M(x)= cL(x) and n SM(n)  S(n) } Maximal amount of space used by M for input of length n

8. Below Logarithmic Space 4.4 • It is known that Dspace(O(1)) is equivalent to the set of regular languages. • Do we gain any strength by having sub-logarithmic sapce?? Or formally… • Dspace(o(log(n)))  Dspace(O(1)) ? or is it Dspace(o(log(n))) = Dspace(O(1)) ?

9. Dspace(o(log(n))) Dspace(O(1)) Theorem:Dspace(o(log(n))) is a proper superset of Dspace(O(1)). Proof: We will construct a language, L, such that LDspace(loglog(n)), but LDspace(O(1)), which will prove the theorem (log(log(n)o(log(n))) .

10. Proof (contd.) - Definition of L L = {xk|kN,xk=Bk’0$Bk’1$Bk’2$…$Bk’ (2k-1)$} where: Bk’i=Binary representation of i of length k. For example: x2 = 00$01$10$11$

11. Proof (contd.) Claim 1:LDspace(O(1))(Regular-Languages) Proof: By using the “Pumping Lemma” Claim 2:LDspace(loglog(n)) Proof: We will show an algorithm for deciding L that uses loglog(n) space.

12. The wrong way for prooving Claim 2 1) Check that the first block is all 0’s and that the last is all 1’s. 2) For any two consecutive blocks, check that the second is the binary increment of the first. (works if xL) this solution can use more then loglog(n) if xk  L (e.g. 0m$1m$) m=n/2 - 1 Clearly (1) can be done in constant space, and (2) in log(k) space which is loglog(n) space, as n=|xk|=(k+1)2k

13. The correct solution m = 1 while (true) { • check that the last m bits of the first block are all 0’s. • check that the last m bits of the Bk’i blocks form an increasing sequence mod 2m, and that each block has m bits. • check that the last m bits of the last block are all 1’s. • if you found an error, return false. • if m is the exact size of Bk’i return true. • m = m +1 }

14. The correct solution – An Example input:000$001$010$011$100$101$110$111$ m=2, 2 left bits increasing mod 22=4 m=1, 1 left bits increasing mod 21=2 000$001$010$011$100$101$110$111$ 000$001$010$011$100$101$110$111$ m=3, 3 left bits increasing mod 23=8 000$001$010$011$100$101$110$111$ The entire series is increasing  return true

15. Below Logarithm Conclusion: L  Space(O(loglog(n)))\Space(O(1))  Dspace(o(log(n)))  Dspace(O(1)) One can show that the above claim does not work for o(loglog(n)), that is: Dspace(o(loglog(n))=Dspace(O(1))

16. Configuration - Definition A configuration of M,M  Dspace(s(n)), is a complete description of its computational state on a fixed input, x (|x|=n), at a specific time. These include: • the state of M (|QM| possibilities) • contents of the worktape (2s(n)possibilities) • the head position on input tape (n possibilities) • the head position on the worktape (s(n) possibilities).

17. #Configuration – An upper bound Let C be the number of possible configuarations of a TM M. C|QM|*2s(n)* n* s(n) head position on worktape number of states contents of the worktape head position on input tape

18. Relation between time and space 4.5 Theorem:s(n) s.t log(n)  s(n)  Dspace(s(n))  Dtime(2O(s(n))) proof: For every L  Dspace(s(n)),There is a TMMthat uses no more thanO(s(n))space on input x.  the number of configurations of M  2O(s(n)). passing fromone configuration to another takes O(1) time. M will stop after 2O(s(n))steps. If it doesn’t, it must pass through the same configuration twice, which implies an infinite loop. log(n)  s(n) and previous slide

19. How to Make TMs halt???? 4.6 Theorem:TM, MDspace(s(n)),TM,M’Dspace(O(s(n)) :L(M’)=L(M),and M’ always halts. (s.t. log(n)  s(n)) proof: By simulation. Given x,M’ computes the maximal number of configurations C. That takes s(|x|) space. Now M’ simulates M. If it gets an answer in less than C steps, it returns the answer. Otherwise (M entered an infinite loop) it returns ‘no’. This stage also takes s(|x|), so the total isO(s|x|).

20. Space Hierarchy 4.7 Theorem:s1(n), s2(n). s1(n)  log(n), s2(n) is space-constructible and s1(n)=o(s2(n))  Dspace(s1(n))  Dspace(s2(n)) proof: By diagonalization. We will construct a language L, such that L  Dspace(s2(n)), but L can’t be recognized by a TM using s1(n) space. We define : Let c0 be a constant, 0 c0 1. L ={x|x=<M>01*,|<M>| c0*s2(|x|), M rejects x by using no more then c0*s2(|x|) space}

21. Space Hierarchy s2(n) is space-constructible Claim:L  Dspace(s2(n)) proof: By a straightforward algorithm. • check if x is of the right form. (O(1) space) • compute S:= c0*s2(|x|).(s2(|x|) space) • check that |<M>|  c0*s2(|x|). (log(S) space) • simulate M. if the bound S was exceeded – reject. if M rejects x – accept, else reject. We get a total of O(s2(|x|)) space as needed.

22. Space Hierarchy Claim:L  Dspace(s1(n)) proof:We will show that for every TM M of space complexity s1(n), L(M)  L. s1(n)=o(s2(n))n0. s1(n0)  c0*s2(n0). Let’s assume by contradiction that there is a TM M0 space complexity s1(n) that satisfies |<M0>|  c0*s2(n0) that accepts L. Let’s observe how M0operates on inputx := <M0>01n0-|<M0>|-1

23. L  Dspace(s1(n)) – Proof contd. • if M0 accepts x, then by definition x  L. • if M0 rejects x, then since |<M0>|  c0*s2(n0), and M0 onx uses at most s1(n0) c0*s2(n0) space, therfore x  L. In any case we find a contradiction to the fact that M0 accepts L.

24. Other space theorems 4.8 Borodin’s Gap Theorem: g(n), g(n) is recursive and g(n) n, f. Dspace(f(n))=Dspace(g(f(n)) Blum’s Speed-up Theorem: g(n), g(n) is recursive and g(n) n, LR,TM M, L(M) = L, MSpace(s(n)) M’, L(M’) = L, M’Space(g-1s(n))

25. Non Deterministic Space 5.1 • Defintion:Anon-deterministic Turing machine - NDTMis a TM with a non-Deterministic transition function, having a work tape, a read-only input tape, and a unidirectional write-only output tape. • The machine is said to accept input x if there exists a computation ending in an accepting state.

26. Defintions of On-line / Off-line TM • An offline(online) non-deterministic TM has a work tape, a read-only input tape, a unidirectional write-only output tape, and a two-way (one-way) read-only guess tape. • The contents of the guess tape is selected non-deterministically (and is the only non-deterministic part in this machine). • The machine is said to accept input x if there exists a content y to the guess tape such that the machine ends in an accepting state.

27. Nspaceon, Nspaceoff Definition:Nspaceon(S) = {L | there exist an online-NDTM MLaccepting x iff xL using at most S(|x|) space} Definition:Nspaceoff(S) = { L | there exist an offline-NDTMMLaccepting x iff xL using at most S(|x|) space }

28. NDTM = Online-NDTM 5.2 Claim: the NDTM model is equivalent to the online-NDTM model proof: We will show that a language L is decidable by a NDTM in time O(T) and space O(S) iff L is decidable by an online-NDTM in the same time and space bounds.  Use the guess string y to determine which transition function to take every step.  Guess every step the content of the next cell in the guess string. Remember last step’s guessed letter (using internal state) when simulated guess-string-head doesn’t move.

29. Nspaceon vs. Nspaceoff 5.3 Theorem:Nspaceon(S)  Nspaceoff(log(S)) • We’ll simulate an online-NDTMMon that uses space S, using an offline-NDTMMoff that uses space log(S). • Moff guess the sequence of configurations of Mon and then validates it.

30. Nspaceon vs. Nspaceoff The guess string will have blocks, each representing a configuration [of length (O(S))]. There will be no more then 2O(S) blocks (any valid series of configurations having more blocks, has a repeating configuration, therefore can be replaced by a shorter guess). guess string doesn’t count in the space of Moff

31. Nspaceon vs. Nspaceoff Moff will validate that: • The first block is a legal starting configuration. • Last block is a legal accepting configuration. • Every block is legally following it’s previous (will be done one by one).

32. Nspaceon vs. Nspaceoff • The (supposedly) configuration strings: ... $aaaabcaa$ aaaabchaa $ aaaabxaha $aaaabcaa$aaaab$... O(log(|Conf|)) 1. Check (almost) all char in strings are identical, and string length are identical. 2. Check that char marked with head position in 1st configuration transforms to a legal threesome on the 2nd. $aaaabchaa$ O(1) $aaaabxaha$

33. Nspaceon vs. Nspaceoff • The working string will hold a counter for the location in the configuration checked - log(O(S)), and O(1) more space for the validation. • A counter for the number of configuration checked - O(S) - is not needed.

34. Nspaceon vs. Nspaceoff • Note that this simulation can’t be done by the online machine, as it has to read forwards & backwards on the guess tape (block size being a function of n).

35. Nspaceon vs. Nspaceoff Theorem:Nspaceoff(S)  Nspaceon(2O(S)) proof: The proof of this theorem also uses simulation of one machine using the other.

36. Savitch’s Theorem 5.4 Theorem:NL = Nspace(log(n))  Dspace(log2(n)) We will later generalize this theorem and show that: S(n)log(n) Nspace(S)  Dspace(S2) Defintion:aConfiguration Graphis a graph that given a TMM which works in space S on an input x, we assign a node for every possible configuration of M’s computation on x, and an edge (u,v) if M can change from configuration u to v.

37. Savitch’s Theorem - Reducing Acceptance to Reachability • If there is more then one accepting configuration, another vertex t is added, with edges (u,t) for all accepting configurations’ vertices u. • The starting configuration’s vertex is named s. The question of M accepting x reduces to an s-t reachability problem on the configurations graph. We will next show an algorithm that solves reachability in Dspace(log2(n)).

38. Savitch’s Theorem The Trick: If there is a path from vertex u to v of size d>0, then there must be a vertex z, such that there is a path from u to z, shorter then d/2, and a path from z to u, shorter then d/2. Note: As we try to save space, we can afford trying ALL possible z’s. Time complexity does not matter.

39. The Algorithm boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if d=1 return FALSE for every vertex v(not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } Both use the same space

40. Why log2(n)? • The binary representation of all numbers used by the algorithm is at most of size of O(log(n)). • As the d parameter is reduced to half at each recursive call, the recursion tree is of depth O(log(n)). • Therefore at each step of the calculation, we use O(log(n)) numbers of size O(log(n)) resulting in O(log2(n)) total space.

41. Example of Savitch’s algorithm boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } 2 3 1 4 (a,b,c)=Is there a path from a to b, that takes no more than c steps. (1,4,3)(2,4,1) (1,4,3)(1,3,2)(1,2,1)TRUE (1,4,3)(1,3,2)(1,2,1) (1,4,3)(3,4,1)TRUE (1,4,3)(1,3,2) (1,4,3)(1,2,2)TRUE (1,4,3)(1,3,2)(2,3,1) (1,4,3)(1,2,2) (1,4,3)(2,4,1)FALSE (1,4,3) TRUE (1,4,3) (1,4,3)(1,3,2)(2,3,1)TRUE (1,4,3)(3,4,1) (1,4,3)(1,3,2)TRUE Log2(3)

42. Applying s-t-reachability to Savitch’s theorem. • Given a NDTMMn working in space log(n), we construct a DTMM working in log2(n) in the following way: • Given x, M solves the s-t reachability on the configuration graph of (Mn,x). • Note that the graph is generated “on demand” , by reusing space, therefore M never keeps the entire representation of the graph.

43. Appling s-t-reachability to Savitch’s thm. Since Mn works in log(n) space it has O(2log(n)) configurations, thus its configuration graph is of size O(2log(n)) and the reachability is solved in log2(O(2log(n)))= log2(n) space.

44. Savitch’s theorem - conclusion • NL  Dspace(log2(n)) • This is not just a special case of Savitch thm but equals it. • We’ll see this next

45. Generalization of the proof Note that in the last argument, We could have substituted the log(n) function by any function, and thus derive the general Savitch Theorem: S(n)log(n)Nspace(S)  Dspace(S2). We will next prove a lemma that will help us generalize any theorem proved for small functions to larger ones. Specifically, we will generalize the NL  Dspace(log2(n))theorem

46. Translation Lemma-(Padding argument) 5.5 Nspace(s1(n))  Dspace(s2(n))  Nspace(s1(f(n)))  Dspace(s2(f(n))) For space constructible functions s1(n), s2(n)  log(n), f(n)  n:

47. Padding argument • Let LNPspace(s1(f(n))) • There is a 3-Tape-NDTM MLwhich acceptsLin NPspace (s1(f(n))) |x| Input babba Work  O(s1(f(|x|)))

48. Padding argument • Define L’ = { x0f(|x|)-|x|| xL } • We’ll show a NDTM ML’which decides L’in the same space as ML. n’=f(|x|) Input babba00000000000000000000000000000000 Work  O(s1(n’)) = O(s1(f(|x|))

49. Padding argument – ML’ NSpace(log(n’)) 1. Count 0’s backwards, mark end of x and check f(|x|)-|x| = 0’s length 2. RunML on x. NSpace(s1(f(n))) = NSpace(s1(n’)) n' Input babba#0000000000000000000000000000000 Work  O(s1(n’))

50. Padding argument Total Nspace(O(s1(n’))) n' Input babba#0000000000000000000000000000000 Work  O(s1(n’))