Slides: Asaf Shapira & Oded Schwartz; Sonny Ben-Shimon & Yaniv Nahum. Notes: Leia Passoni, Reuben Sumner, Yoad

Space Complexity Slides: Asaf Shapira & Oded Schwartz; Sonny Ben-Shimon & Yaniv Nahum. Notes: Leia Passoni, Reuben Sumner, Yoad Lustig & Tal Hassner. (from Oded Goldreich’s course lecture notes)

Introduction This lecture covers: • Space Complexity • Non-Deterministic space

Complexity Functions 4.1 Def: A function f is called constructible if it satisfies the following conditions: • Positive: f: N+  N+ • Monotone: f(n+1)  f(n) for all n • Constructive:  a Turing Machine Mf that, on input x, outputs a string of size f(|x|), in time O(|x|+f(|x|)), and in space O(f(|x|))

Constructible functions 4.2 • Many “popular” complexity functions, e.g. n, log(n), n2, n! satisfy the above criteria • Odd things may occur, in regard to relations between complexity classes, if we don't choose these functions properly • Note: We therefore use time constructible functions for time bound and space constructible functions for space bound

Space Complexity - The model: 4.3 3-tape Turing machine: • Input Tape – Read Only • Output tape – Write only [Omitted for decision problems][Usually unidirectional] • Work tape – Read & Write Enables sub linear space Length of which corresponds to space usage

What kind of TM should we use? • Any multi-tape TM can be simulated by an ordinary TM with a polynomial loss of efficiency • Hence, from here on, a TM will refer to the 3 tape TM described above

DSPACE - Definition For every TM, M, and input x: WM(x) = The index of the rightmost cell on the work-tape scanned by M on x SM(n) = max|x|=n WM(x) cL(x) = 1 if xL, 0 otherwise DSPACE(S(n)) = {L| DTM M, x M(x)= cL(x) and n SM(n)  S(n) } Maximal amount of space used by M for input of length n

Sub-Logarithmic Space 4.4 • DSPACE(O(1)) is equivalent to the set of regular languages. • Do we gain any strength by having sub-logarithmic sapce?? Or formally… • DSAPCE(o(log(n)))  DSPACE(O(1)) ? or is it DSPACE(o(log(n))) = DSPACE(O(1)) ?

DSPACE(o(log(n))) DSPACE(O(1)) Thm:DSPACE(o(log(n))) is a proper superset of DSPACE(O(1)) Proof: We will construct a language L s.t. LDSPACE(loglog(n)), however, LDSPACE(O(1)), which will prove the theorem (log(log(n)) = o(log(n))) .

Proof (contd.) - Definition of L L = {xk|kN,xk=Bk’0$Bk’1$Bk’2$…$Bk’ (2k-1)$} where: Bk’i=Binary representation of i of length k. For example: x2 = 00$01$10$11$

Proof (contd.) Claim 1:LDSPACE(O(1))(Regular-Languages) Proof: By using the “Pumping Lemma” Claim 2:LDSPACE(loglog(n)) Proof: We will show an algorithm for deciding L that uses loglog(n) space.

The wrong way of proving Claim 2 works if xL 1) Check that the first block is all 0’s and that the last is all 1’s. 2) For any two consecutive blocks, check that the second is the binary increment of the first. Clearly (1) can be done in constant space, and (2) in log(k) space which is loglog(n) space, as n=|xk|=(k+1)2k this might use more then loglog(n) if xk  L (e.g. 0m$1m$) m=n/2 - 1

The correct solution m = 1 while (true) { • check that the last m bits of the first block are all 0’s • check that the last m bits of the Bk’i blocks form an increasing sequence mod 2m, and that each block has m bits. • check that the last m bits of the last block are all 1’s • if you found an error, return false • if m is the exact size of Bk’i return true. • m = m +1 }

The correct solution – An Example input:000$001$010$011$100$101$110$111$ m=1, 1 left bits increasing mod 21=2 m=2, 2 left bits increasing mod 22=4 000$001$010$011$100$101$110$111$ 000$001$010$011$100$101$110$111$ m=3, 3 left bits increasing mod 23=8 000$001$010$011$100$101$110$111$ The entire series is increasing  return true

Sub-Logarithm Conclusion: L  Space(O(loglog(n)))\Space(O(1))  DSPACE(o(log(n)))  DSPACE(O(1)) In fact, the above claim does not work for o(loglog(n)), that is: DSPACE(o(loglog(n))=DSPACE(O(1))

Configuration - Definition Def: A configuration of a TMM, is a complete description of its state at a computation stage, comprising: • the state of M (range |QM|) • contents of the worktape (range2s(n)) • the head position on the input tape (rangen) • the head position on the worktape (ranges(n)).

#Configuration – An upper bound Let C be the number of possible configuarations of a TM M. C|QM|*2s(n)* n* s(n) number of states head position on worktape contents of the worktape head position on input tape

Relation between time and space 4.5 Thm:s(n) ≥ log(n) DSPACE(s(n))  Dtime(2O(s(n))) Proof: For every L  DSPACE(s(n)),There is a TMMthat uses no more thanO(s(n))space on input x.  the number of configurations of M  2O(s(n)).  if M does not stop after 2O(s(n))steps, it must pass through the same configuration twice, which implies an infinite loop.

How to Make TMs halt? 4.6 Thm: For s(n)≤ log(n), for any TM, MDSPACE(s(n)), there is a TM,M’DSPACE(O(s(n)) s.t.L(M’)=L(M),and M’ always halts. Proof: By simulation. Given x,M’ computes the maximal number of configurations C -- that takes s(|x|) space. Now M’ simulates M. If it arrives at an answer in less than C steps, it returns it. Otherwise (M is in an infinite loop) M’ returns ‘no’. This stage also takes s(|x|), so the total isO(s|x|).

Space Hierarchy 4.7 Thm: for any s1(n), s2(n), if s1(n)  log(n), s2(n) is space-constructible and s1(n)=o(s2(n)) then DSPACE(s1(n)) ≠ DSPACE(s2(n)) Proof: By diagonalization. We construct a language L, such that L  DSPACE(s2(n)), but L can’t be recognized by any TM using s1(n) space: Let c0 be a constant, 0 c0 1. L ={ x | x = <M> 01*, |<M>|  c0s2(|x|), M rejects x using ≤c0s2(|x|) space }

Space Hierarchy Claim:L  DSPACE(s2(n)) Proof: By a straightforward algorithm. • check if x is of the right form. (O(1) space) • compute S:= c0s2(|x|).(s2(|x|) space) • check that |<M>|  c0s2(|x|). (log(S) space) • simulate M.if the bound S has been exceeded – reject.if M rejects x – accept, else reject. Altogether O(s2(|x|)) space as claimed.

Space Hierarchy Claim:L  DSPACE(s1(n)) Proof: We show that for every TMM of space complexity s1(n), L(M)  L. s1(n)=o(s2(n))n0. s1(n0)  c0*s2(n0). Assume, by way of contradiction, a TMM0 of space complexity s1(n), s.t. |<M0>|  c0*s2(n0), accepting L. Observe M0result on the inputstring

L  DSPACE(s1(n)) – Proof contd. • if M0 accepts x, then by definition x  L. • if M0 rejects x, then since |<M0>|  c0s2(n0), and M0 onx uses at most s1(n0) c0*s2(n0) space, it must be that x  L. In any case this is a contradiction to the fact that M0 accepts L.

5.1 Non Deterministic Space • Def:Anon-deterministic Turing machine - NDTMis aTM with a non-deterministic transition function, having a work tape, a read-only input tape, and a unidirectional write-only output tape. • The machine is said to accept input x if there exists a computation ending in an accepting state.

Def: On-line / Off-line TM • An offline(online) non-deterministic TM has a work tape, a read-only input tape, a unidirectional write-only output tape, and a two-way (one-way) read-only guess tape. • The machine is said to accept input x if there exists a content y to the guess tape that causes the machine’s computation to end in an accepting state.

Nspaceon, Nspaceoff Def:Nspaceon(S) ={L | there exist an online-NDTM ML, which uses ≤ S(|x|) space, that accepts x iff xL } Def:Nspaceoff(S) = { L | there exist an offline-NDTMML, whichuses ≤S(|x|) space, that accepts x iff xL}

NDTM = Online-NDTM 5.2 Claim: the NDTM model is equivalent to the online-NDTM model Proof: We show that a language L is decidable by a NDTM in time O(T) and space O(S) iff L is decidable by an online-NDTM with same time and space bounds.  Use the guess string y to determine which transition function to take every step.  Guess the content of the next cell in the guess string when read. Remember the last step’s guessed letter (using internal state) when guess-tape-head doesn’t move.

Nspaceon vs. Nspaceoff 5.3 Thm:Nspaceon(S)  Nspaceoff(log(S)) • We’ll simulate an online-NDTMMon that uses space S, using an offline-NDTMMoff that uses space log(S). • Moff guesses a sequence of configurations of Mon and then validates it to be an accepting run

Nspaceon vs. Nspaceoff The guess string has blocks, each representing a configuration [of length (O(S))] with ≤ 2O(S) blocks(any valid sequence of configurations with more blocks, must have the same configuration twice, therefore can be replaced by a shorter guess) guess string doesn’t count in the space of Moff

Nspaceon vs. Nspaceoff Moff will validate that: • The first block is a legal starting configuration. • Last block is a legal accepting configuration. • Every block can result by a legal move applied to previous block (carried out two consequetive blocks at a time)

Nspaceon vs. Nspaceoff • The (supposedly) configuration strings: ... $aaaabcaa$ aaaabchaa $ aaaabxaha $aaaabcaa$aaaab$... O(log(|Con|)) 1. Check (almost) all symbols in strings are identical, and string lengths identical. 2. Check that symbols marked with head position in 1st configuration transforms to a legal threesome on the 2nd. $aaaabchaa$ O(1) $aaaabxaha$

Nspaceon vs. Nspaceoff • The working tape holds a counter of the location in the configuration checked - log(O(S)), and O(1) additional space for the validation. • A counter for the number of configuration checked - O(S) - is not necessary!

Nspaceon vs. Nspaceoff • Note that this simulation can’t be done by the online machine, as it has to read forwards & backwards on the guess tape (block size being a function of n).

Nspaceon vs. Nspaceoff Thm:Nspaceoff(S)  Nspaceon(2O(S)) Proof: The proof of this theorem also uses simulation of one machine using the other.

Savitch’s Theorem 5.4 Thm:NL = Nspace(log(n))  DSPACE(log2(n)) We later generalize the theorem to read: S(n)log(n) Nspace(S)  DSPACE(S2) Def:aConfiguration Graphis a graph that, given a TMM which works in space S on an input x, has one vertex for every possible configuration of M’s computation on x, and an edge (u,v) if M can change move from u to v.

Savitch’s Thm - Reducing Acceptance to Reachability • If there is more then one accepting configuration, another vertex t is added, with edges (u,t) for all accepting configurations’ vertices u. • The starting configuration’s vertex is named s. The question of M accepting x reduces to an s-t reachability problem on the configuration graph. We next show reachability in DSPACE(log2(n)).

Savitch’s Theorem The Trick: If there is a path from vertex u to v of size d>0, then there must be a vertex z, s.t. there is a path from u to z, shorter then d/2, and a path from z to u, shorter then d/2. Note: As we try to save space, we can afford trying ALL possible z’s. Time complexity does not matter.

The Algorithm Both use the same space BooleanPATH(a,b,d) { if there is an edge from a to b then return TRUE else { if d=1 return FALSE for every vertex v (except a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } }

Why log2(n)? • The binary representation of all numbers used by the algorithm is at most of size of O(log(n)). • As the d parameter is reduced to half at each recursive call, the recursion tree is of depth O(log(n)). • Therefore at each step of the computation, we use at most O(log(n)) numbers of size O(log(n)) resulting in O(log2(n)) total space.

Example of Savitch’s algorithm boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } boolean PATH(a,b,d) { if there is an edge from a to b then return TRUE else { if (d=1) return FALSE for every vertex v (not a,b) { if PATH(a,v, d/2) and PATH(v,b, d/2) then return TRUE } return FALSE } } 2 3 1 4 (a,b,c)=Is there a path from a to b, that takes no more than c steps. (1,4,3)(2,4,1) (1,4,3)(1,3,2)(1,2,1)TRUE (1,4,3)(1,3,2)(1,2,1) (1,4,3)(3,4,1)TRUE (1,4,3)(1,3,2) (1,4,3)(1,2,2)TRUE (1,4,3)(1,3,2)(2,3,1) (1,4,3)(1,2,2) (1,4,3)(2,4,1)FALSE (1,4,3) TRUE (1,4,3) (1,4,3)(1,3,2)(2,3,1)TRUE (1,4,3)(3,4,1) (1,4,3)(1,3,2)TRUE Log2(3)

Applying s-t-reachability to Savitch’s theorem. • Given a NDTMMn working in space log(n), we construct a DTMM working in log2(n) in the following way: • Given x, M solves the s-t reachability on the configuration graph of (Mn,x). • Note: the graph is generated “on demand”, reusing space, therefore M never keeps the entire representation of the graph.

Appling s-t-reachability to Savitch’s thm. Since Mn works in log(n) space it has O(2log(n)) configurations, its configuration graph is of size O(2log(n)) and reachability is solved in log2(O(2log(n)))= log2(n) space.

Savitch’s theorem - conclusion • NL  DSPACE(log2(n)) • This is not just a special case of Savitch Thm but equivalent. • As we’ll see next

Generalization of the proof Note that in the last argument, We could have substituted the log(n) function by any function, and thus derive the general Savitch Theorem: S(n)log(n)Nspace(S)  DSPACE(S2). We will next prove a lemma that will help us generalize any theorem proved for small functions to larger ones. Specifically, we will generalize the NL  DSPACE(log2(n))theorem

Translation Lemma-(Padding argument) 5.5 For space constructible functions s1(n), s2(n)  log(n), f(n)  n: • Nspace(s1(n))  DSPACE(s2(n)) Nspace(s1(f(n)))  SPACE(s2(f(n)))

Padding argument • Let LNPspace(s1(f(n))) • There is a 3-Tape-NDTM MLwhich acceptsLin NPspace (s1(f(n))) |x| Input babba Work  O(s1(f(|x|)))

Padding argument • Define L’ = { x0f(|x|)-|x|| xL } • We’ll show a NDTM ML’which decides L’in the same space as ML. n’=f(|x|) Input babba00000000000000000000000000000000 Work  O(s1(n’)) = O(s1(f(|x|))

Padding argument – ML’ NSpace(log(n’)) NSpace(s1(f(n))) = NSpace(s1(n’)) 1. Count 0’s backwards, mark end of x and check f(|x|)-|x| = 0’s length 2. RunML on x. n' Input babba#0000000000000000000000000000000 Work  O(s1(n’))

Padding argument Total Nspace(O(s1(n’))) n' Input babba#0000000000000000000000000000000 Work  O(s1(n’))

Padding argument – M’L’ ML’ NPspace(s1(n)) using Nspace(s1(n))  DSPACE(s2(n)) : there is a M’L’, deterministic TM, which accept L’ in DSPACE(s2(n)) Given M’L’ , we will construct a DTM M*L that accept L in O(s2(f(n)) space.

Slides: Asaf Shapira & Oded Schwartz; Sonny Ben-Shimon & Yaniv Nahum. Notes: Leia Passoni, Reuben Sumner, Yoad