CSCI 3130: Formal languages and automata theory

1. Fall 2010 The Chinese University of Hong Kong CSCI 3130: Formal languages and automata theory Decidable andundecidable languages Andrej Bogdanov http://www.cse.cuhk.edu.hk/~andrejb/csc3130

2. Problems about automata • We can formulate this question as a language: b a a Does accept input abb? q0 q1 b ADFA = {〈D, w〉: Dis a DFA that accepts input w} Is ADFA decidable? ((q0,q1)(a,b)((q0,a,q0)(q0,b,q1)(q1,a,q0)(q1,b,q1)(q0)(q1))(abb) w D = (Q, S, d, q0, F)

3. Problems about automata ADFA = {〈D, w〉: Dis a DFA that accepts input w} pseudocode: TM description: On input 〈D, w〉,where D = (Q, S, d, q0, F): On input 〈D, w〉,where D is a DFA, w is a string Set q := q0 SimulateD on input w For i := 1 to length(w): If simulation ends in acc state, accept. Otherwise, reject. q := d(q, wi) If q ∈ F accept, else reject

4. Problems about automata ADFA = {〈D, w〉: Dis a DFA that accepts input w} Turing Machine details: Check input is in correct format. (Transition function is complete, no duplicate transitions) Perform simulation: state input symbol . . . . . . ((q0,q1)(a,b)((q0,a,q0)(q0,b,q1)(q1,a,q0)(q1,b,q1)(q0)(q1))(abb) qacc

5. Problems about automata decidable ADFA = {〈D, w〉: Dis a DFA that accepts input w} Turing Machine details: Check input is in correct format. (Transition function is complete, no duplicate transitions) Perform simulation: Put markers on start state of D and first symol of w Until w marker reaches last symbol: Update both markers If state marker is on accepting state, accept. Else reject.

6. Acceptance problems about automata decidable ADFA = {〈D, w〉: Dis a DFA that accepts input w} ANFA = {〈N, w〉: Nis an NFA that accepts w} AREX = {〈R, w〉: Ris a regular expression that generates w} Which of these is decidable?

7. Acceptance problems about automata • The following TM decides ADFA: decidable ADFA = {〈D, w〉: Dis a DFA that accepts input w} M := On input 〈D, w〉, where D is a DFA and w is a string Simulate D on input w If the simulation ends in acc state of D, accept. If it doesn’t, reject.

8. Acceptance problems about automata • The following TM decides ANFA: decidable ANFA = {〈N, w〉: Nis an NFA that accepts input w} N := On input 〈N, w〉, where N is an NFA and w is a string Convert N to a DFA D using the conversion procedure from Lecture 2 Run the TM M for ADFA on input 〈D, w〉 If M accepts, accept. Otherwise, reject.

9. Acceptance problems about automata • The following TM decides AREX: decidable AREX = {Ris a regular expression that generates w} P := On input 〈R, w〉, where R is a reg exp and w is a string Convert R to an NFA A using the conversion procedure from Lecture 4 Run the TM N for ANFA on input 〈A, w〉 If N accepts, accept. Otherwise, reject.

10. Other problems about automata • The following TM decides MINDFA: decidable MINDFA = {〈D〉: Dis a minimal DFA} R := On input 〈D〉, where D is a DFA Run the distinguishable states algorithm from Lecture 7 on D If every pair of states is distinguishable, accept. Otherwise, reject.

11. Other problems about automata • The following TM decides EQDFA: decidable EQDFA = {〈D1, D2〉: D1, D2are DFAs and L(D1) = L(D2)} S := On input 〈D1, D2〉, where D1 and D2 are DFAs Run the DFA minimization algorithm from Lecture 7 on D1 to obtain a DFA D1’ Run the DFA minimization algorithm from Lecture 7 on D2 to obtain a DFA D2’ If D1 = D2accept, otherwise reject.

12. Other problems about automata • The following TM decides EDFA: decidable EDFA = {〈D〉: D is a DFAs and L(D) is empty} T := On input 〈D〉, where D is a DFA Run the TM S for EQDFA on input 〈D,〉 If S accepts accept, otherwise reject.

13. Problems about context-free grammars decidable ACFG = {〈G, w〉: Gis a CFG that generates w} V := On input 〈G, w〉, where G is a CFG and w is a string Eliminate the nullable and unit productions from G Convert G to Chomsky Normal Form Run the Cocke-Younger-Kasami algorithm on 〈G, w〉 If the CYK algorithm produces a parse tree, accept. Otherwise, reject.

14. Decidability of context-free languages Every context-free language is decidable. Let L be a context-free language. There is a CFG G for L. The following TM decides L: MG := On input w, Run TM V on input 〈G, w〉. If Vaccepts accept, otherwise reject.

15. Are all languages about CFGs decidable? EQCFG = {〈G1, G2〉: G1 and G2are context-free grammars that generate the same strings} undecidable What’s the difference between EQDFA and EQCFG? To decide EQDFA, we minimized both DFAs But there is no method that, given a CFG or PDA, produces a unique equivalent minimal CFG or PDA

16. The universal Turing Machineand undecidability

17. Turing Machines versus computers computer program output input A computer is a machine that manipulates data according to a list of instructions. How does a Turing Machine take a program as part of its input?

18. The Universal Turing Machine U program 〈M〉 M on input x input x The universal TMU takes as inputs a program Mand a string x and simulates M on x The program M itself is specified as a TM!

19. Turing Machines as strings • This Turing Machine can be described by the string M ☐/☐R qa 0/0R • A Turing Machine is (Q, ,, , q0, qacc, qrej) q 1/1R qr 〈M〉 = (q,qa,qr)(0,1)(0,1,☐) ((q,q,☐/☐R) (q,qa,0/0R) (q,qr,1/1R)) (q)(qa)(qr)

20. The universal Turing Machine U (q,qa,qr)(0,1)(0,1, 001 input w for M program 〈M〉 U := On input 〈M, w〉, SimulateM on input w If M enters accept state, accept. If M enters reject state, reject.

21. Acceptance of Turing Machines ATM = {〈M, w〉: Mis a TM that accepts w} U := On input 〈M, w〉, SimulateM on input w M accepts w M rejects w M loops on w U accepts 〈M, w〉 U rejects 〈M, w〉 U loops on 〈M, w〉 TM Urecognizes but does not decideATM

22. Recognizing versus deciding qacc qrej accept reject loop halt The language recognized by a TM is the set ofall inputs that make it accept A TM decides language Lif it recognizes L andhalts (does not loop) on every input

23. Undecidability • Turing’s Theorem: • Before we show this, let’s observe one thing: The language ATMis undecidable. A Turing Machine M can be given its own description〈M〉 as an input!

24. Turing’s theorem A TM that decides ATMis so potent that it will destroy itself.

25. Proof of Turing’s Theorem • Proof by contradiction: Suppose ATM is decidable. Then there is a TM H that decides ATM: accept ifM accepts w 〈M, w〉 H reject ifM rejects wor M loops on w accept ifM accepts 〈M〉 〈M,〈M〉〉 What happens whenw = 〈M〉? reject ifM rejects 〈M〉or M loops on 〈M〉

26. Proof of undecidability accept ifM accepts 〈M〉 H 〈M,〈M〉〉 reject ifM rejects 〈M〉or M loops on 〈M〉 Let H’ be a TM that does the opposite of H acc acc H = (Q, ,, , q0, qacc, qrej) H H’ rej rej H’ = (Q, ,, , q0, qrej, qacc)

27. Proof of undecidability accept ifM accepts 〈M〉 H 〈M,〈M〉〉 reject ifM rejects 〈M〉or M loops on 〈M〉 ifM rejects 〈M〉 or M loops on 〈M〉 accept H’ 〈M,〈M〉〉 reject ifM accepts 〈M〉

28. Proof of undecidability ifM rejects 〈M〉 or M loops on 〈M〉 accept H’ 〈M,〈M〉〉 reject ifM accepts 〈M〉 Let D be the following TM: 〈M〉 H’ 〈M,〈M〉〉 copy

29. Proof of undecidability ifM rejects 〈M〉 or M loops on 〈M〉 accept D 〈M〉 reject ifM accepts 〈M〉 What happens when M = D? If D accepts 〈D〉 then D rejects 〈D〉 ifD rejects 〈D〉 or D loops on 〈D〉 H never loops, so H’ and D never loop either so D does not exist! 〈D〉 If D rejects 〈D〉 then D accepts 〈D〉 reject ifD accepts 〈D〉

30. Proof of undecidability: conclusion • Proof by contradiction • We assumed ATM was decidable • Then we had Turing Machines H, H’, D • But D does not exist! • Conclusion The language ATMis undecidable.

31. What happened? all possible inputs w … 00 e 0 1 M1 acc rej rej acc M2 … rej acc loop rej all possible Turing Machines M3 rej loop rej rej … … We can write an infinite table for every pair (M, w)

32. What happened? … 〈M4〉 〈M1〉 〈M2〉 〈M3〉 M1 acc loop rej acc M2 … rej acc rej acc M3 loop rej loop rej … … Now let’s look only at those wthat describe a TM M

33. What happened? … 〈M4〉 〈M1〉 〈M2〉 〈M3〉 M1 acc loop rej acc M2 … rej acc rej acc … … D rej rej acc rej … … If ATM is decidable, then TM D is in the table

34. What happened? … 〈M4〉 〈M1〉 〈M2〉 〈M3〉 M1 acc loop rej acc M2 … rej acc rej acc … … D rej rej acc rej D does the opposite of the diagonal entries 〈M〉 accept ifM rejects or loops on 〈M〉 D reject ifM accepts 〈M〉

35. What happened? … 〈M4〉 〈M1〉 〈M2〉 〈M3〉 〈D〉 M1 acc loop rej acc loop M2 … rej acc rej acc acc … … ? … D rej rej acc rej We run into trouble when we look at (D, 〈D〉)!

36. Unrecognizable languages • How about languages that are not recognizable? The language ATMis recognizable but not decidable. ATM = {〈M, w〉: Mis a TM that does not accept w} = {〈M, w〉: Mrejects or loops on input w} The language ATMis not recognizable.

37. Unrecognizable languages • Theorem • We know ATM is recognizable, so if ATM were also,then ATM would be decidable • Impossible by Turing’s Theorem If L and L are both recognizable, then L isdecidable.

38. Unrecognizable languages • Proof idea If L and L are both recognizable, then L isdecidable. accept ifw ∈ L M w accept rej/loop ifw ∉ L w accept ifw ∉ L M’ w reject rej/loop ifw ∈ L

39. Unrecognizable languages If L and L are both recognizable, then L isdecidable. Problem: If M loops on w, we will never get to step 2! Let M = TM for L, M’ = TM for L On input w, Simulate M on input w. If it accepts, accept. ① Simulate M’ on input w. If it accepts, reject. ②

40. Bounded simulation • Decider for L: If L and L are both recognizable, then L isdecidable. Let M = TM for L, M’ = TM for L On input w, For k = 0 to infinity Simulate M on input w for ksteps. If it accepts, accept. Simulate M’ on input w for ksteps. If it accepts, reject.