Computability Decidability

1. ComputabilityDecidability

2. Unsolvable problems • Computer scientists usually deal with problems which are solvable. • We are supposed to find an algorithm to solve a particular problem. • Why do we need to know about problems which are unsolvable ? • we will not waste time in finding an non-existed algorithm. • we will realize that the problem needs to be simplified or altered before we can find an algorithmic solution. • unsolvable problems can stimulate our imagination and help us gain an important perspective on computation.

3. Decidable Languages • We have seen in the last chapter an undecidable problem, the 10th problem of Hilbert. • Now we will show some other problems and consider whether they are decidable by algorithm or not.

4. Decidable Problems Concerning Regular Languages • Acceptance problem • Whether a finite automata accepts a string ? • ADFA = {<B,w>| B is a DFA that accepts input string w} • Whether a nondeterministic finite automata accepts a string ? • ANFA = {<B,w>| B is a NFA that accepts input string w} • Whether a regular expression generates a given string. • AREX = {<R,w>| R is a regular expression that generates string w}

5. Decidable Problems concerning Regular Lang. • Empty Language problem • Whether a language of a finite automata is empty ? • EDFA = {<A>| A is a DFA and L(A) = } • Equivalence problem • Whether two finite automata are equivalent ? • EQDFA = {<A,B>| A and B are DFAs and L(A) = L(B)}

6. Theorem 1 • ADFA is a decidable language • ADFA = {<B,w>| B is a DFA that accepts input string w} • Show that the problem of testing whether a given finite automata accepts a given string is decidable • Proof idea: Present a TM M that decides ADFA.

7. Proof • M = “on input <B,w>, where B is a DFA and w is a string: Simulate B on input w. If the simulation ends in an accept state, accept. If it ends in a non-accepting state, reject.” • Notes: • input of M is an encoded string <B,w> • B can be represented reasonably as a list of its five components, Q, , , q0 and F. • before M starts its computation, M first checks on whether it properly represents a DFA B and a string w. If not, M rejects.

8. Proof: Input Tape reject B M keeps track of B’s current state and B’s current position in the input w by writing this information down on its tape. accept Machine M

9. Theorem 2 • ANFA is a decidable language • ANFA = {<B,w>| B is a NFA that accepts input string w} • Proof idea: • Present a TM N that decides ANFA. • N has M as a subroutine. • Because M is designed to work with DFAs, N first converts the NFA B it receives as input to a DFA before passing it to M.

10. Proof • N = “on input <B,w>, where B is a NFA and w is a string: Convert NFA B to an equivalent DFA C Run TM M from Theorem 1 on input <C,w>. If M accepts, accept; otherwise reject.”

11. Proof: Input Tape Machine N reject C accept Machine M

12. Theorem 3 • AREX is a decidable language • AREX = {<R,w>| R is a regular expression that generates string w} • Proof idea: • Present a TM P that decides AREX.

13. Proof • P = “on input <R,w>, where R is a regular expression and w is a string: Convert regular expression R to an equivalent DFA A. Run TM M on input <A,w>. If M accepts, accept; if M rejects , reject.”

14. Theorem 4 • EDFA is a decidable language • EDFA = {<A>| A is a DFA and L(A) = } • Show that the problem of testing whether a given finite automata accepts any strings at all is decidable. • Proof idea: • A DFA accepts some string if and only if reaching an accept state from the start state by traveling alone the arrows of the DFA is possible. • Present a TM T that uses a marking algorithm.

15. A is empty Proof • T = “on input <A>, where A is a DFA: Mark the start state of A. Repeat until no new states get marked: Mark any state that has a transition coming into it from any state that is already marked. If no accept state is marked, accept; otherwise reject.”

16. Theorem 5 • EQDFA is a decidable language • EQDFA = {<A,B>| A and B are DFAs and L(A) = L(B)} • Show that the problem of testing whether 2 DFAs recognize the same language is decidable. • Proof idea: • Construct a new DFA C from A and B, where C accepts only those strings that are accepted by either A or B but not by both. • If A and B recognize the same language, C will accept nothing. • Test whether L(C) is empty by the theorem 4.4

17. L(A) L(B) L(C) = (L(A)  L(B))  (L(A)  L(B)) L(C) Symmetric difference of L(A) and L(B) C is also a DFA as the class of regular language is closed under complementation, union and intersection

18. Proof • F = “on input <A,B>, where A and B are DFAs: Construct DFA C as described. Run TM T from Theorem 4 on input <C> If T accepts, accept. If T rejects, reject.”

19. Proof Machine F Machine T C reject reject accept accept

20. Decidable Problems concerning to Context-Free Language • Acceptance problem • Whether a CFG accepts a string ? • ACFG = {<G,w>| G is a CFG that accepts input string w} • Emptiness problem • Whether a CFG generates any strings at all ? • ECFG = {<G>| G is a CFG and L(G) = } • Equivalent problem • Whether two CFGs generate the same language ? • EQCFG = {<G,H>| G and H are CFGs and L(G) = L(H)}

21. Theorem 6 • ACFG is a decidable language • ACFG = {<G,w>| G is a CFG that generates string w} • Proof idea: • Naïve idea : • Go through every derivations to determine whether any is a derivation of w. • But this algorithm will not halt if w is not a member of G. • Better approach: • We need to ensure that the algorithm tries only finitely many derivations, so use CNF form. • any derivation of w has 2n-1 steps, where n is the length of w.

22. Proof • S = “on input <G,w>, where G is a CFG and w is a string: Convert G to an equivalent grammar in Chomsky Normal Form. List all derivations with 2n-1 steps, where n is the length of w. If any of these derivations generate w, accept; if not, reject.”

23. CFLs and PDAs • As we can convert back and forth between CFLs and PDAs. • Hence, everything we say about the decidability of problems concerning CFLs applies well to PDAs.

24. Theorem 7 • ECFG is a decidable language • ECFG = {<G>| G is a CFG and L(G) = } • Proof idea: • Naïve idea : to determine whether L(G) =  • try going through all possible w’s, one by one. • But there are infinitely many w’s to try, so this algorithm could end up running forever. • Better approach: • We need to test whether the start variable can generate a string of terminals. • Determine for each variable whether that variable is capable of generating a string of terminals, the algorithm keeps track of this information by placing a mark on that variable.

25.  G is empty Proof • R = “on input <G>, where G is a CFG : Mark all terminal symbols in G. Repeat until no new variables get marked Mark any variable A where G has a rule AU1 U2… Uk and each symbol U1 , …, Ukhas already been marked. If the start symbol is not marked, accept; otherwise reject. ”

26. EQCFG • Consider the problem of testing whether 2 CFGs generate the same language. • ECFG = {<G,w>| G is a CFG that generates string w} • Proof idea: • Naïve idea : • use the same method as theorem 4.5 • But CFL is not closed under complementation or intersections. • In fact, ECFG is not decidable. We’ll prove it in the next chapter.

27. Theorem 8 • “Every context-free language is decidable.” • We show that every CFL is decidable. • Proof idea: • Naïve (bad) idea : • try to convert a PDA for A directly to a TM. • the PDA for A may be nondeterministic, so we get an NTM which we can convert into a deterministic TM. • however, PDA can have some branches go on forever, reading and writing the stack without coming to a halt. therefore our TM will not be a decider. • Better approach: • We prove this theorem with the TM S in Theorem 6 to decide ACFG.

28. Proof • Let G be a CFG for A and design a TM MG that decides A. • We build a copy of G into MG. • It works as follow. • MG = “on input w: Run TM S on input <G,w> If this machine accepts, accept; if it rejects, reject.”

29. Machine MG Machine S Convert G to CNF form. List all derivations with 2n-1 steps, where n is the length of w. If any of these derivations generate w, accept; if not, reject. accept accept reject reject Proof

30. Relationship among classes of Language Turing-recognizable Context-free Decidable Regular

31. Non Recursively Enumerator language • We want to find a language that is not Recursively Enumerable • This language is not accepted by any Turing Machine • It can’t be implemented on (contemporary) computer. • Example: • Verify that the program we wrote performs as specified (that it is correct). • Halting problem : Testing whether a TM accepts a given input string (ATM)

32. Theorem 9 • ATM = {<M,w>| M is a TM and M accepts w}. • ATM is undecidable. • Proof: observe that ATM is Turing-recognizable. • We try to find a TM (U) that recognizes ATM.

33. Halting Problem • U = “On input <M,w>, where M is a TM and w is a string: Simulate M on input w. If M ever enters its accept state, accept; if M ever enters its reject state, reject.” • Note: U loops on input <M,w> if M loops on w, that’s why U doesn’t decide ATM.

34. Universal Turing Machine • proposed by Turing. • a machine is called universal if it is capable to simulate any other Turing machine from the description of that machine. • played an important early role in stimulating the development of stored-program computers.

35. The Diagonalization method • discovered by Georg Cantor in 1873. • is the proof of the undecidability of the halting problem. • using countable and uncountable set to prove. • Idea is : we want to prove that • set of Turing-recognizable language is uncountable • while set of Turing machine is countable.

36. Cantor’s problem • Concerned with the problem of measuring the sizes of infinite sets. • If we have 2 infinite sets, how can we tell whether one is larger than the other or whether they are of the same size ? • Naïve: try to count the elements of the infinite set, we will never finish!

37. Cantor’s problem • Better approach: • Observe that 2 finite sets are of the same size if the elements of the one set can be paired with the element of the other set. • We can extend this idea to infinite sets.

38. Definition: Correspondence • Consider 2 sets, A and B, and a function f from A to B. • Say that f is one-to-one if it never maps 2 different elements to the same place that is, f(a)  f(b) whenever a  b. • Say that A and B are the same size if there is a one-to-one, onto function f: AB. • one-to-one and onto is called Correspondence

39. Example • Let • N be the set of Natural numbers {1,2,3,…} •  be the set of even natural numbers {2,4,6,…} • Using cantor’s method, we can see that N and  have the same size.

40. Definition : Countable • A set A is countable if either it is finite or it has the same size as N.

41. Example • Let • N be the set of Natural numbers {1,2,3,…} • Q be the set of positive rational numbers Q = {m/n |m,n N } • Q seems the be bigger than N but actually they have the same size. • Using cantor’s method, we list all the elements of Q and then pair each element with the element of N. • We have to ensure that every member of Q appears only once on the list.

42. 1 1 1 1 1 3 5 3 3 3 3 5 2 2 2 2 2 4 4 4 4 4 4 4 2 2 2 2 4 4 3 5 5 1 5 2 1 1 3 3 5 1 3 1 Correspondence of N and Q . . . . . .

43. Trick • We pair • the first element of Q on the list with the number 1 from N. • the second element of Q on the list with the number 2 from N. • … • After seeing the corresponding of N and Q, we can see that as each curve Q has finite number of elements which can be mapped with elements of N, thus both sets have same size.

44. Theorem 4.14 • R is uncountable • R is the set of real number. • Proof by the diagonalization method.

45. Proof • by contradiction : suppose that a correspondence f existed between N and R.  means that we must pair all the members of N with all the members of R. • show that f fails to work as it should. • How to show ?  finding an x in R that is not paired with anything in N.

46. Proof • Finding x by constructing it. • choose each digit of x to make x different from one of the real numbers that is paired with an element of N • Sure that x is different from any real number that is paired.

47. suppose we have a corresponding f: we construct x by let the first digit of x be anything different from the first fractional digit of f(1). let the second digit of x be anything different from the second fractional digit of f(2). … thus we can construct x which is different from f(n) for any n. Illustration of the idea

48. Corollary 4.15 • Some languages are not Turing-recognizable. • Proof idea: • set of Turing-recognizable language is uncountable while set of Turing machine is countable. • Because each Turing machine can recognize a single language and there are more languages than TMs, some languages are not recognized by any TM. • Thus, there exists some languages which are not Turing-recognizable.

49. set of all TMs is countable. • Proof idea: • first of all • we have to observe that the set of all * is countable, for any alphabet . • with only finitely many strings of each length, we may form a list of * by written down all strings of length 0, length 1, length 2 and so on.

50. set of all TMs is countable. • Secondly • We can encode each TM into a string <M> • as the set of all strings is countable therefore the set of all TMs is countable.