CSC 3130: Automata theory and formal languages

1. Fall 2008 The Chinese University of Hong Kong CSC 3130: Automata theory and formal languages Undecidable Languages Andrej Bogdanov http://www.cse.cuhk.edu.hk/~andrejb/csc3130

2. 0 1 1 1 1 q0 q1 q2 q3 0 0 0 Problems about automata • When we talked about DFA minimization, we asked questions like Is this DFA minimal? q1 1 q2 1 1 Answer: Yes, because e e e q3 q0 q1 q2

3. 0 1 1 1 1 q0 q1 q2 q3 0 0 0 Problems about automata • Can we make a TM that answers questions like: Is this DFA minimal? We can implement the DFA minimization algorithm By the C-T thesis, a Turing Machine can do this

4. 0 1 1 1 1 q0 q1 q2 q3 0 0 0 Problems about automata • What does a TM that answerslook like? Is this DFA minimal? A DFA is (Q, S, d, q0, F) M {q0,q1,q2,q3}{0,1}{(q0,0,q0)(q0,1,q1)(q1,0,q0)...

5. Taking an automaton as an input • This is a TM that takes the description 〈D〉 of a DFA D as an input and decides the language M {q0,q1,q2,q3},{0,1},{(q0,0,q0),(q0,1,q1),... MINDFA = {〈D〉: Dis a minimal DFA}

6. Acceptance problems about automata ADFA = {〈D, w〉: Dis a DFA that accepts input w} ANFA = {〈N, w〉: Nis an NFA that accepts w} AREX = {〈R, w〉: Ris a regular expression that generates w} Which of these is decidable?

7. Acceptance problems about automata • The following TM decides ADFA: decidable ADFA = {〈D, w〉: Dis a DFA that accepts input w} M := On input 〈D, w〉, where D is a DFA and w is a string Simulate D on input w If the simulation accepts, accept. If it doesn’t, reject.

8. Acceptance problems about automata • The following TM decides ANFA: decidable ANFA = {〈N, w〉: Nis an NFA that accepts input w} M := On input 〈N, w〉, where N is an NFA and w is a string Convert N to a DFA D using the subset construction Run the TM for ADFA on input 〈D, w〉

9. Other problems about automata decidable MINDFA = {〈D〉: Dis a minimal DFA} On input 〈D〉, where D is a DFA Run the DFA minimization algorithm on D to obtain a DFA D’ If D = D’accept, otherwise reject. decidable EDFA = {〈D〉: Daccepts no inputs} On input 〈D〉, where D is a DFA Run the DFA minimization algorithm on D to obtain a DFA D’ If D’ = accept, otherwise reject.

10. Problems about context-free grammars decidable ACFG = {〈G, w〉: Gis a CFG that generates w} On input 〈G, w〉, where G is a CFG and w is a string Convert G to Chomsky Normal Form Run the Cocke-Younger-Kasami algorithm If the top entry contains the start variable, accept Otherwise reject

11. Equivalence decidable EQREX = {〈R1, R2〉: R1 and R2are regular expressions that generate the same strings} On input 〈R1, R2〉, where R1, R2 are reg expressions Convert both R1 and R2 to DFAs D1 and D2 Run the DFA minimization algorithm on both to obtain minimized DFAs D1’ and D2’ If D1’ = D2’ accept, otherwise reject EQCFG = {〈G1, G2〉: G1 and G2are context-free grammars that generate the same strings} undecidable

12. Are Turing Machines computers? computer input data output data instructions (program) Can a Turing Machine take instructionsas part of its input?

13. The Universal Turing Machine U input x M on input x program 〈M〉 The universal TMU takes as inputs a program Mand a string x and simulates M on x The program M itself is specified as a TM!

14. Turing Machines as strings • This Turing Machine can be described by the string M ☐/☐R qa 0/0R • A Turing Machine is (Q, ,, , q0, qacc, qrej) q 1/1R qr 〈M〉 = {q,qa,qr},{0,1},{0,1,☐}, {(q,q,☐/☐R) (q,qa,0/0R) (q,qr,1/1R)}, q,qa,qr

15. The universal Turing Machine U program 〈M〉 {q,qa,qr}{0,1}{0,1,☐}{(q,q☐/☐R),( input x for M 001 state q U on input 〈M, x〉 will simulateM on input x

16. Acceptance of Turing Machines • Run the universal TM U on input 〈M, w〉 • Simulation is correct: • If M accepts w, we accept • If M rejects w, we reject • If M loops on w, we loop • This TM recognizes but does not decideATM ATM = {〈M, w〉: Mis a TM that accepts w}

17. Recognizing versus deciding qacc qrej accept reject loop halt The language recognized by a TM is the set ofall inputs that make it accept A TM decides language Lif it recognizes L andhalts (does not loop) on every input

18. Undecidability • Turing’s Theorem: • Before we prove this, let’s observe one thing: The language ATMis undecidable. A Turing Machine M can be given its own description〈M〉 as an input!

19. Idea of proof If some TM decides ATM, then this TM is so powerful that it will destroy itself.

20. Proof of undecidability • Proof by contradiction: Suppose ATM is decidable. Then there is a TM H that decides ATM: accept ifM accepts w 〈M, w〉 H reject ifM rejects wor M loops on w accept ifM accepts 〈M〉 〈M,〈M〉〉 What happens whenw = 〈M〉? reject ifM rejects 〈M〉or M loops on 〈M〉

21. Proof of undecidability accept ifM accepts 〈M〉 H 〈M,〈M〉〉 reject ifM rejects 〈M〉or M loops on 〈M〉 Let H’ be a TM that does the opposite of H H’ acc acc H rej rej

22. Proof of undecidability accept ifM accepts 〈M〉 H 〈M,〈M〉〉 reject ifM rejects 〈M〉or M loops on 〈M〉 ifM rejects 〈M〉 or M loops on 〈M〉 accept H’ 〈M,〈M〉〉 reject ifM accepts 〈M〉

23. Proof of undecidability ifM rejects 〈M〉 or M loops on 〈M〉 accept H’ 〈M,〈M〉〉 reject ifM accepts 〈M〉 Let D be the following TM: 〈M〉 H’ 〈M,〈M〉〉 copy

24. Proof of undecidability ifM rejects 〈M〉 or M loops on 〈M〉 accept D 〈M〉 reject ifM accepts 〈M〉 What happens when M = D? If D accepts 〈D〉 then D rejects 〈D〉 ifD rejects 〈D〉 or D loops on 〈D〉 H never loops, so H’ and D never loop either so D does not exist! 〈D〉 If D rejects 〈D〉 then D accepts 〈D〉 reject ifD accepts 〈D〉

25. Proof of undecidability: conclusion • Proof by contradiction • We assumed ATM was decidable • Then we had Turing Machines H, H’, D • But D does not exist! • Conclusion The language ATMis undecidable.

26. What happened? input w … 00 e 0 1 M1 acc rej rej acc M2 … rej acc loop rej Turing Machine M3 rej loop rej rej … … We can write an infinite table for every pair (M, w)

27. What happened? … 〈M4〉 〈M1〉 〈M2〉 〈M3〉 M1 acc loop rej acc M2 … rej acc rej acc M3 loop rej loop rej … … Now let’s look only at those wthat describe a TM M

28. What happened? … 〈M4〉 〈M1〉 〈M2〉 〈M3〉 M1 acc loop rej acc M2 … rej acc rej acc … … D rej rej acc rej … … If ATM is decidable, then TM D is in the table

29. What happened? … 〈M4〉 〈M1〉 〈M2〉 〈M3〉 M1 acc loop rej acc M2 … rej acc rej acc … … D rej rej acc rej D does the opposite of the diagonal entries 〈M〉 accept ifM rejects or loops on 〈M〉 D reject ifM accepts 〈M〉

30. What happened? … 〈M4〉 〈M1〉 〈M2〉 〈M3〉 〈D〉 M1 acc loop rej acc loop M2 … rej acc rej acc acc … … ? … D rej rej acc rej We run into trouble when we look at (D, 〈D〉)!

31. Unrecognizable languages • How about languages that are not recognizable? The language ATMis recognizable but not decidable. ATM = {〈M, w〉: Mis a TM that does not accept w} = {〈M, w〉: Mrejects or loops on input w} The language ATMis not recognizable.

32. Unrecognizable languages • Theorem • We know ATM is recognizable, so if ATM were also,then ATM would be decidable • Impossible by Turing’s Theorem If L and L are both recognizable, then L isdecidable.

33. Unrecognizable languages • Proof idea If L and L are both recognizable, then L isdecidable. accept ifw ∈ L M w accept rej/loop ifw ∉ L w accept ifw ∉ L M’ w reject rej/loop ifw ∈ L

34. Unrecognizable languages • Proof attempt If L and L are both recognizable, then L isdecidable. Problem: If M loops on w, we will never get to step 2! Let M = TM for L, M’ = TM for L On input w, Simulate M on input w. If it accepts, accept. ① Simulate M’ on input w. If it accepts, reject. ②

35. Bounded simulation • Decider for L: If L and L are both recognizable, then L isdecidable. Let M = TM for L, M’ = TM for L On input w, For k = 0 to infinity Simulate M on input w for ksteps. If it accepts, accept. Simulate M’ on input w for ksteps. If it accepts, reject.

36. Another undecidable language • To prove this, we will show that … so by Turing’s Theorem it is undecidable. HALTTM = {〈M, w〉: Mis a TM that halts on input w} HALTTMis an undecidable language If HALTTMcan be decided, so can ATM.

37. Undecidability of halting • Suppose R decides HALTTM If HALTTMcan be decided, so can ATM. accept ifM halts on w R 〈M, w〉 reject ifM loops on w • We want to use R to design D for ATM accept ifM accepts w ? 〈M, w〉 reject ifM rejects or loops on w

38. Undecidability of halting accept ifM halts on w R 〈M, w〉 reject ifM loops on w D R acc acc simulate M on w accept ifM accepts w U 〈M, w〉 rej rej reject ifM rejects or loops on w

39. More undecidable problems L1 = {〈M〉: Mis a TM that accepts input e} L2 = {〈M〉: Mis a TM that accepts some input} L3 = {〈M〉: Mis a TM that accepts all inputs} L4 = {〈M, M’〉: Mand M’ accept the same inputs} Next lecture...

40. Gödel’s incompleteness theorem • Turing’s proof of Gödel’s Theorem: Some mathematical statementsare true but not provable. If every true statement is provable, then ATM can be decided.

41. Proof sketch of Gödel’s theorem • Suppose every true statement has a proof • For every TM M and input w, one of the statementsis true, so it must have a proof M accepts w or M does not accept w

42. Proof sketch of Gödel’s theorem • Consider this TM D for ATM: On input 〈M, w〉: Fork = 0 to infinity For all possible proofs p of length k If p is a proof of “M accepts w”, accept. If p is a proof of “M does not accept w”, reject. One of them is true, so it has a proof p Eventually, D will find this proof