MATHEMATICAL FOUNDATIONS OF COMPUTER SCIENCE COMPSCI 350

1. MATHEMATICAL FOUNDATIONS OF COMPUTER SCIENCE COMPSCI 350

2. THE THEORY OF TURING MACHINES Standard One-tape Turing Machines Definition 1. A Turing machine (TM) is a 7-tuple (Q, , , , qs, qaccept, qreject), where Q, ,  are finite sets, and Q is the set of states of the machine, 2)  is the input alphabet ( U, the blank symbol, ), 3)  is the tape alphabet (U  , ), 4)  : Q x   Q x  x {L, R} is the machine’s transition function, 5) qs Q is the start state, usually taken to be q0 or q1, 6) qaccept Q is the accept state, and qreject Q is the reject state (qs ≠ qaccept ≠ qreject). This is the definition in Sipser’s text. Strictly speaking, Sipser takes  to be a function from Q \ { qaccept, qreject } x  to Q x  x {L, R}, since he assumes that  is not defined when the state is either qaccept or qreject .

3. The conventions for Turing machines are as follows. A Turing machine begins in the start state qs looking at the left-most symbol a of the input on a tape which is infinite to the right (apart from the input the tape is blank, ie contains only U’s). If the machine’s transition function  specifies that ( qs, a) = (qj, b, R), the machine head then writes symbol b in the square it is scanning, and moves Right into state qj. The machine then continues to follow the other instructions in the same way. • Suppose, for example, that the machine head now detects a symbol g. And suppose that the machine’s transition function  specifies that ( qj, g) = (qr, t, L).

4. In this case, the head writes t in the square (overwriting the previously written copy of g) and moves Left into state qr (here, of course, it meets the previously written b). (Note that we follow the convention that if the head is instructed to move left when the head is already on the leftmost square, it stays in the same place for that move and awaits the next instruction.) The machine halts when it enters either state qaccept or qreject . If it never enters those states, the computation continues for ever.

5. Definition 2. (a) Given a Turing machine M, a configuration (‘cf’, for short) for M is a (finite) string vqw, where q is a state of M, and where v and w are strings over the machine’s tape alphabet. [vw represents the current tape contents of the machine, in the sense that it gives the symbols on a left-most segment of the tape, with everything on the tape to the right of vw being blank; the machine is assumed to be in state q, scanning the first symbol of w.] We say that q is the state of the configuration, and the symbol to the right of q is the scanned symbol of the configuration [if w is the empty string, the scanned symbol is the blank symbol U]. Intuitively, one configuration yields another, given a machine M, if we get to the second by applying the transition function of M to the state and scanned symbol of the first. We now define this idea formally.

6. (b) If C and D are configurations of M, we say that C yieldsM D (C ├M D) if one of the following holds (where v, w are strings * and a, b, c ): 1) C = vaqi bw and (qi, b)= ( qk, c, L), then D = vqkacw, 2) C= vqi bw and (qi, b) = ( qk, c, R), then D = vcqk w, 3) C = qi bw and (qi, b) = ( qk, c, L), then D = qk cw [can’t go left, so leave head where it is], 4) C = vqi and (qi, U) = ( qk, c, R), then D = vcqk, 5i) C = vaqi and (qi, U) = ( qk, b, L) and b  U, then D = vqk ab, ii) C = vaqi and (qi, U) = ( qk, U, L), then D = vqk a .

7. (c) Let C be a configuration. 1) C is the start configuration of M on input w if C = qsw. 2) C is an accepting configuration if the state of the configuration is qaccept. 3) C is a rejecting configuration if the state of the configuration is qreject. 4) C is a halting configuration if C is an accepting or rejecting configuration.

8. Definition 3 (a) A Turing machine M with input alphabet accepts w, where w  *, if there is a sequence of configurations C1, ..., Ck such that C1 is the start configuration of M on w, each Ci yieldsM Ci+1 (0< i < k), and Ck is an accepting configuration. • (b) A Turing machine M with input alphabet rejects w, where w  *, if there is a sequence of configurations C1, ..., Ck such that C1 is the start configuration of M on w, each Ci yieldsM Ci+1 (0< i < k), and Ck is a rejecting configuration.

9. Note that our TMs are deterministic devices, so no machine ever accepts and rejects the same string w. (c) If M is a Turing machine with input alphabet , L(M) is the set of strings over  accepted by M. If L is a language over , we say that M recognises L if L = L(M). (d) A language L is Turing-recognisable if there is a Turing machine M such that L=L(M), ie such that M recognises L. Examples:Let A1={0, 1}*; A2 = Ø; A3 = {} be languages over {0, 1}. Show them to be Turing-recognisable.

10. Definition 4 (a) A Turing machine M with input alphabet  is a decider if, for all strings w over , M either accepts or rejects w. (b) M decides language L if M is a decider and L=L(M). (c) A language L is Turing-decidable if there is a Turing machine M such that M decides L. Note that it follows immediately that any Turing-decidable language is also Turing-recognisable.

11. Examples: A1= {0,1}*, A2=, A3= {}, • A4= {w  {0,1}* | w contains at least one 0}, • A5= {w  {0,1}* | w contains exactly one 0}, • A6= {wwR| w  {0,1}*}, • A7= {w  {0,1}*| w  A6}

12. Example 4: Consider the language A4 = { w | w  {0, 1}* and w contains at least one 0}. Show that A4 is Turing-decidable, and that the construction is correct, ie, that the machine M you construct is such that A4 = L(M). • Proof: Consider machine M = (Q, {0, 1}, {0, 1, U}, , q1, qaccept, qreject), where Q ={q1, q2, qaccept, qreject}, and transition function  is as follows. • [The accompanying notes help to explain the way the function behaves. Note that the transition function could also have been specified in a state diagram.]

13. (q1, U) = (qreject, U, R) (no 0 spotted before the end of string is reached; reject) (q1, 0) = (qaccept, 0, R) (M has seen its first 0; that is enough, so accept) (q1, 1) = (q1, 1, R) (keep going until a 0 is found)

14. We now show that the construction is correct by proving that A4 = L(M), ie that both A4L(M) and L(M)  A4. To show that A4L(M), let w=w1w2...wn and suppose that w  A4. So w contains at least one 0. Let wk be the first 0 (1 k  n). Consider the following cfs for M. Let C1 = q1w1w2...wn, C2 = w1q1w2...wn, … , Ck = w1w2...q1wk… wn. Let Ck+1 = w1w2...wkqaccept wk+1…wn. [Of course, if wk is wn then Ck+1 = w1w2...wkqaccept .]

15. It is obvious that Ci yieldsM Ci+1 for 1 i < k , since until the machine encounters the first 0, ie wk, it simply stays in q1 and moves right, and that is what is described in C1, C2, …, Ck. In addition, Ck yieldsM Ck+1 since (q1, wk) = (qaccept, wk, R) (wk is a 0). • Note also that C1 is the start configuration of M on w, and Ck+1 is an accepting configuration. It follows immediately from definition 3 that M accepts w, and hence that A4L(M).

16. ii) To show that L(M)  A4, consider w L(M). Let w = w1w2...wn; we allow this to cover the case of n = 0, ie w is the empty sequence ). Because w L(M), there is a sequence of cfs C1, C2, … , Ck+1 such that C1 is the start cf of M on w, each Ci yields Ci+1 (1  i ≤ k  n), and Ck+1 is an accepting cf. We thus have C1 = q1w1w2...wn, …, Ci = w1 … q1wi ...wn, … , Ck+1 = w1 ...wk qaccept … . Clearly, the application of  that yields Ck+1 from the preceding cf must have been (q1, 0) = (qaccept, 0, R) (since that is the only place qaccept occurs in ). Since Ck+1 = w1 ...wk qaccept … it follows that wk (the kth symbol of w) must be a 0. Hence w  A4, so that L(M)  A4. Combining i) and ii), it follows that A4 = L(M).

17. VARIANTS OF TURING MACHINES • Multi-tape Turing Machines and Non-deterministic Turing Machines • Definition 6. (a) A multitape Turing machine is a 7-tuple (Q, , , , qs, qaccept, qreject), where Q, , , qs, qaccept, qreject are as before, and where  : Q x k Q x k x {L, R}k is the machine’s transition function, where k is the number of tapes (1  k).

18. [The expression (qi, a1, …, ak) = (qj, b1, …, bk, L, R, …, L) means that if the machine is in state qi and the read-write heads on tapes 1 to k are reading symbols a1through ak , the machine goes to state qj, writes symbols b1 through bk, and moves each head to the Left or Right as specified.

19. Note that the definition given is that from the first edition of Sipser. The second edition gives a definition of multitape Turing machines that uses three moves: L, R and S, for stay-put. It is easy, of course, to show that having S makes no difference whatsoever, for a machine that only has L and R can always simulate the stay-put move S by first moving R, then straight away moving L. ]

20. (b) A nondeterministic Turing machine is a 7-tuple (Q, , , , qs, qaccept, qreject), where Q, , , qs, qaccept, qreject are as before, , and where : Q x  (Q x  x {L, R}) is the machine’s transition function. • [ (Q x  x {L, R}) is the set of all subsets of (Q x  x {L, R}). This means that the transition function for nondeterministic TMs can allow more than one choice: each of the members of  (Q x  x {L, R}) determines a possible next move.]

21. The notion of a configuration (cf) is the same as for single tape machines. The yield operation can be defined much as usual, except that a cf for a NTM can yield 0 or more cfs. We say that a NTM, N say, accepts w if there is a sequence C1, …, Ck of cfs for N such that i) C1 is the start cf on w, ii) Ci yields Ci+1 for all I, 0 < I < k, and iii) Ck is an accepting cf. We say that N rejects w if every sequence C1, C2, …, of cfs for N such that i) C1 is the start cf on w, and ii) Ci yields Ci+1 (for I > 0), is finite, and none terminates with an accepting cf.

22. Theorem 1. • Every multitape TM M has an equivalent standard one-tape TM S, one that accepts a string w iff M accepts it and rejects w iff M rejects it. • Corollary: (a) A language is Turing-recognisable iff some multitapeTM recognises it. • (b) A language is Turing-decidable iff some multitape TM decides it. • Proof: The left to right direction for both (a) and (b) is immediate since any standard TM is a multitape TM. The other direction follows from the theorem.

23. Theorem 2. • Every NTM (nondeterministic Turing machine) N has an equivalent standard one-tape TM D, one that accepts a string w iff N accepts it and rejects w iff N rejects it. • Note (as in the case of multitape machines) that this is stronger than just saying that N and D are equivalent, ie, that N and D recognise the same language, for it implies that if N decides a language then D decides it as well.

24. Proof of Theorem 2. • Most of the proof is in the textbook (proof of Theorem 3.16). We view N’s computation on an input w as a tree, each node of which is a configuration and each branch of which represents a sequence of possible configurations ordered by the ‘yields’ operation.

25. Let b = maxqQ, s  || q,s) || . We assign addresses to nodes of such a tree by means of strings over the aphabet {1, 2, …, b}. 132, for example, represents the node we arrive at by starting at the root of the tree, going to its first ‘child, then going to that node’s third child, then to that node’s second child. Each symbol in the string tells us which choice to make next when simulating a step in one branch in N’s nondeterministic computation.

26. Note that some of these addresses are invalid, since a symbol may not correspond to any choice (there may be far fewer than b choices available to go from one node to one of its children, and there may even be none, if the node is a halting configuration).

27. We describe a multitape Turing machine D that simulates N’s computation, accepting when N accepts, rejecting when N rejects, and not halting otherwise. Since every multitape Turing machine can be simulated by a standard single tape TM (by Theorem 1), the existence of this machine D would prove the theorem.

28. D has three tapes: tape 1 contains the input w and its contents are never changed, tape 2 is the working tape on which D simulates N’s computation on w, and tape 3, which contains an address over {1, 2, …, b}, keeps track of D’s location in N’s nondeterministic computation. Note that initially tape 1 contains input w, with tapes 2 and 3 empty. • D operates as follows:

29. 1. Copy tape 1 to tape 2. • 2. On tape 3, write the first string over alphabet {1, …, b} (using the lexicographic ordering of such strings). • 3. On tape 2, simulate N on input w, following the branch of its nondeterministic computation given by the address on tape 3. Before each step of N, consult the next symbol on tape 3 to determine which choice to make from those allowed by N’s transition function.

30. 3. (Cont.) Go to stage 4 if either no more symbols remain on tape 3 OR if this address is invalid OR if a rejecting configuration is encountered. If an accepting configuration is encountered, accept. • 4. Replace the string on tape 3 with the next string in the lexicographic ordering of strings over {1, …, b} [Sipser p. 14]. • 5. If each of the branches of nondeterministic computation terminates with a rejecting configuration, then reject.

31. (Re: stage 5. Note that if each of the branches terminates with a cf that is not an accepting cf, each branch has finitely many nodes; it then follows that the entire tree has finitely many nodes, so that D can search N’s entire computation tree to determine that each branch terminates with a rejecting configuration.)

32. By construction, D accepts a string w iff N accepts it, so that D recognises the same language that N recognises. In addition, D rejects a string iff N rejects it.  • Corollary: (a) A language is Turing-recognisable iff some NTM recognises it. • (b) A language is Turing-decidable iff some NTM decides it. • Proof: The left to right direction is immediate since any standard TM is an NTM. The other direction follows from the theorem. 

33. 3. CLOSURE THEOREMS FOR THE CLASS OF TURING-DECIDABLE LANGUAGES AND FOR THE CLASS OF TURING-RECOGNISABLE LANGUAGES.

34. Theorem 3 [First Closure Theorem] The class of Turing-decidable languages over a fixed alphabet is closed under the operations of i) union, ii) intersection, iii) complement, iv) concatenation, and v) star.

35. Proof of Theorem 3: i) Suppose A and B are Turing-decidable. Let M1 decide A and M2 decide B. Then M decides AB, where M is as follows: M = “On input w, 1. Make a second copy of w. Run M1 on the first copy of w. If M1 accepts, accept. If M1 rejects, run M2 on the second copy of w. If M2 accepts, accept. Otherwise, reject.” Clearly, M accepts w iff either M1 or M2 accepts w, ie, iff w is in AB.

36. ii) Suppose A and B are Turing-decidable. Let M1 decide A and M2 decide B. Then M decides AB, where M is as follows: M = “On input w, 1. Make a second copy of w. Run M1 on the first copy of w. If M1 accepts, go to 2. If M1 rejects, reject. 2. Run M2 on the second copy of w. If M2 accepts, accepts. Otherwise, reject.” Clearly, M accepts w iff both M1 and M2 accept w, ie, iff w is in AB.

37. iii) Suppose A is Turing-decidable. Let M decide A. Then MC decides the complement AC of A: MC = “On input w, 1. Run M on w. If M accepts, reject. If M rejects, accept.” Clearly, MC is a decider (since M is) and MC accepts w iff M rejects w iff w is in AC. Hence MC decides the complement AC of A.

38. iv) Suppose A and B are Turing-decidable. Let M1 decide A and M2 decide B. Then N, a NTM, decides AoB [the concatenation of A and B], where N is as follows: M = “On input w, 1. Nondeterministically split w into two parts, x and y. Go to 2. 2. Run M1 on the left part, x. If M1 accepts, go to 3. If M1 rejects, go to qreject. 3. Run M2 on the right part y. If M2 accepts, accept, ie, go to qaccept. If M2 rejects, go to qreject.”

39. Clearly N accepts w just so long as w is the concatenation of two strings x and y, the first of which is in A and the second in B, and N rejects w otherwise. Hence N decides AoB. According to the corollary to Theorem 2, a language is Turing-decidable iff some NTM decides it. Hence AoB is Turing decidable.

40. v) Suppose A and B are Turing-decidable. Let M decide A. Then N, a NTM, decides A*, where N is as follows: N = “On input w, 1. If w = , go to qaccept. Otherwise, nondeterministically split A into k parts (0<k≤|w|). Go to 2. 2. Run M on each of these parts. If M accepts each of them, go into qaccept. If M rejects any of them, go into qreject.”

41. It is not hard to see that N accepts an input w just so long as w is the concatenation of k strings, 0≤k≤|w|, all of which are in A. Furthermore, N is a decider, so that, by the corollary to Theorem 2, A* is Turing-decidable. 

42. Theorem 4 [Second Closure Theorem] The class of Turing-recognisable languages over a fixed alphabet is closed under the operations of i) union, ii) intersection, iii) concatenation, and iv) star.

43. Proof of Theorem 4: Suppose A and B are Turing-recogisable. Let M1 recognise A and M2 recognise B. Then M recognises AB, where M is as follows: M = “On input w, 1. Make a second copy of w, and run M1 and M2 alternatively on the two copies of w, one step at a time. If either accepts, accept. If both machines reject, reject.”

44. Note: M doesn’t spend all its time first simulating M1 on w. It might never finish! M = “On input w, 1. Make a second copy of w, and run M1 and M2alternatively on the two copies of w, one step at a time. If either accepts, accept. If both machines reject, reject.” Clearly, M accepts w iff either M1 or M2 accepts w, ie, iff w is in AB. Note that if neither M1 nor M2 accepts or rejects w, then M will loop.

45. ii) Suppose A and B are Turing- recognisable. Let M1 recognise A and M2 recognise B. Then M recognises AB, where M is as follows: M = “On input w, Make a second copy of w, and run M1 and M2 alternatively on the two copies of w, one step at a time. If both accept, accept. If either machine rejects, reject.” Note that neither M1 nor M2 are guaranteed to be deciders, so this machine M may never halt. In any case, it is clear that M accepts w just in case both M1 and M2 accept w, ie w is in AB.

46. iii) Suppose A and B are Turing- recognisable. Let M1 recognise A and M2 recognise B. Then N, a NTM, recognises AoB [the concatenation of A and B], where N is as follows: N = “On input w, 1. Nondeterministically split w into two parts, x and y. Go to 2. 2. Run M1 and M2 alternatively on the two parts x and y of w, one step at a time. If at any point both parts are accepted, accept, ie go into qaccept. If at any point one part is rejected, go into qreject.”

47. Once again, neither M1 nor M2 are guaranteed to be deciders, so this machine N may never halt. In any case, it is clear that N accepts w just in case w is the concatenation of two strings x and y such that x is in A and y is in B, ie such that w is in AoB.

48. iv) Suppose A is Turing-recognisable. Let M recognise A. Then N, a NTM, recognises A*, where N is as follows: N = “On input w, 1. If w = , go to qaccept. Otherwise, nondeterministically split A into k parts (0<k≤|w|). Go to 2. 2. Run M alternatively on each of these parts, one step at a time. If at some point M has accepted each of them, accept, ie, go into qaccept. If M rejects any of them, go into qreject.”

49. N = “On input w, 1. If w = , go to qaccept. Otherwise, nondeterministically split A into k parts (0<k≤|w|). Go to 2. 2. Run M alternatively on each of these parts, one step at a time. If at some point M has accepted each of them, accept, ie, go into qaccept. If M rejects any of them, go into qreject.” It is not hard to see that N accepts an input w just so long as w is the concatenation of k strings, 0 ≤ k ≤ |w|, each of which is accepted by M and hence are in A.

50. What happened to closure under complementation? Well, the proof of the closure of Turing-decidability can’t be adapted to the case of Turing-recognisability. (Our machines needn’t be deciders, so they might fail to accept certain inputs without rejecting them.) Indeed, it can be proved that the class of Turing-recognisable languages is not closed under complementation. This is a corollary of the immensely important fact that the Halting Problem for Turing machines is undecidable, to be proved later.