The Halting Problem

1. The Halting Problem Chuck Cusack Based on • M. Sipser, “Introduction to the Theory of Computation,” Second Edition, Thomson/Course Technology, 2006, Chapter 4.2.

2. An Important Problem • Have you even written a program that ran much longer than you expected? • Did you kill the program after a while, and still to this day do not know whether or not it would have finished? • Wouldn’t it be nice if there was an algorithm that would tell you whether or not your algorithm would eventually finish, or was going to run forever? • Well, maybe you can! • We will discuss this and a few other important problems. • We will also take a closer look at the relationships between the classes of languages we know about.

3. The Halting Problem I • Problem (The halting problem): Given a program A, and input to the program I, does A halt if given input I? • In other words, we want an algorithm H such that, given any program A and input I: • H(A,I) = true if A(I) halts (finishes), and • H(A,I) = false if A(I) will never halt (loops forever) • Is the halting problem solvable? • We can restate this problem in terms of languages and Turing machines: HALTTM = { M, w | M is a TM that halts on input w } • So the question we need to answer is Is HALTTM decidable?

4. Countable and Uncountable • A set is countable if it is finite or has the same size as N (the set of natural numbers {1, 2, 3, …}) • Examples: N (natural numbers), Z (integers), Q (rational numbers), E (even numbers), etc. • Every infinite countable set has the same number of elements. • In other words, you can define a one-to-one and onto mapping (or correspondence) from one set to the other. • e.g. f(n)=2n is a correspondence from N to E. • A set is uncountable if it has more elements than N . • Examples: R (real numbers)  [0,1]  Cantor set • There is, of course, no correspondence between countable and uncountable sets. • In addition, there is not necessarily a correspondence between two uncountable sets.

5. Not Turing Recognizable? • We begin with a simple but important result Corollary 4.18: Some languages are not Turing-recognizable. Proof: Most of the details are omitted, but the idea is • The set of Turing machines is countable • The set of languages is uncountable • Therefore there are some languages that cannot be recognized by a Turing machine. • In fact, most languages are not Turing-recognizable. See Section 4.2 of Sipser for a more detailed proof.

6. Turing- recognizable There are languages in here Are any languages in here? Decidable 0n1n2n Context-free 0n1n ATM Regular 0n What we know—and don’t • We have seen languages that are • Regular (e.g. 0n) • Context-free but not regular (e.g. 0n1n) • Decidable but not context-free (e.g. 0n1n2n) • Not Turing-recognizable (We just proved such languages exist). • Are there languages that are Turing-recognizable but not decidable? • We will show that ATM is Turing-recognizable but not decidable.

7. The Halting Problem II • Ultimately, we want to know: Is HALTTM = { M, w | M is a TM that halts on input w } decidable? • Before we can answer this question, we will consider a related question: Is ATM= { M, w | M is a TM that accepts w } decidable? • This is also sometimes called the halting problem. • Knowing the answer to this question will help us fill in our diagram as well as prove the main result of interest.

8. ATM is Turing-recognizable Thm 4.11a: ATM= { M, w | M is an TM that accepts w } is Turing-recognizable. Proof: Below is a TM U which recognizes ATM. U = “On input M, w , where M is a TM and w is the input string: • Simulate M on input w. • Accept if M enters an accept state, reject if M enters a reject state.” • Why doesn’t UdecideATM? • The TM U takes a description of M as input and simulates it with input w. Is it clear that this is possible? How?

9. Universal Turing Machines • The TM U we just used is an example of a universal Turing machine. • A universal Turing machine (UTM) is a TM which is capable of simulating any other Turing machine from the description of the machine. • The UTM concept helped lead to the stored-program concept which is central to modern computers. • If you think about it, modern PCs really are UTMs.

10. ATM is not decidable Thm 4.11: ATM= { M, w | M is an TM that accepts w } is not decidable. Proof: By contradiction • Suppose ATM is decidable. Let H be a decider for ATM. • Then given a TM M and input w, • H(M, w) = accept if M accepts w. • H(M, w) = reject if M does not accept w. • H never loops since it is a decider. • Construct a new TM D as follows: D = “On input M, where M is a TM: • Run H on input M, M. • If H accepts, reject. If H rejects, accept.”

11. Side Note: Running a TM on itself? • Notice that D does the following step: • Run H on input M, M. • In other words, H is asked to tell us what M does when run with input M (a description of itself). • Can a TM run with itself as input? • Sure, we do this all the time—what do you think a compiler does? It takes a program as input. • In fact, a compiler can be written in the language it is compiling. So, you can compile a compiler with itself! • If you like, just realize that a description of a TM is just a string, so a TM as the input is the same as passing any other string as input.

12. Proof Continued • We have TM D that operates as follows: D = “On input M, where M is a TM: • Run H on input M, M. • If H accepts, reject. If H rejects, accept.” • Notice that • D(M) = accept if M does not accept M. • D(M) = reject if M accepts M. • So what happens when we run D on itself? • D(D) = accept if D does not accept D. • D(D) = reject if D accepts D. • So what’s the problem?

13. Proof Completed: The Contradiction • Here is what happens when we run D on itself: • If D rejects D, then D(D) = accept. • In this case, Daccepts D if Drejects D. • If D accepts D, then D(D) = reject. • In this case, Drejects D if Daccepts D. • Both of these cases are problematic. • Whatever D is supposed to do, it has to do the opposite when it is given itself as input. • This is impossible, so TM D cannot exist. • But D just calls TM H, so TM H cannot exist. • But TM H exists because we assumed that ATM is decidable. • Therefore, it must be that case that ATM is not decidable.

14. ? Turing- recognizable ATM Decidable Context-free 0n1n2n Regular 0n1n 0n What We Know • We already know of languages in several of the classes in our diagram • We have shown that ATM is Turing-recognizable but not decidable. • We have already proven that there are languages that are not Turing-recognizable. • Can we find an example of a language that is not Turing-recognizable?

15. Co-Turing-recognizable • A language is called co-Turing-recognizable if it is the complement of a Turing-recognizable language. Thm 4.22: A language is decidable iff it is Turing-recognizable and co-Turing-recognizable. Proof: () • If language A is decidable, some TM M decides A. • M will accept any element of A, so A is Turing-recognizable • M reject any element not in A. Thus, we can recognize Āby accepting whenever M rejects. • Thus Ā is Turing-recognizable, so A is co-Turing-recognizable.

16. Proof Continued Proof: () • Let language A be Turing-recognizable and co-Turing-recognizable. • Then there exists • a TM M1 which recognizes A, and • a TM M2 which recognizes Ā • We construct a new TM M: M = “On input w: • Run both M1 and M2 on input w in parallel. • If M1 accepts, accept. If M2 accepts, reject.” • Since every string is in A or Ā, then either M1 or M2 will accept w. • Therefore M will always halt, so it is a decider. • Further, it accepts all strings in A and rejects all strings in Ā, so it decides A. • Therefore A is decidable.

17. ATM is not Turing-recognizable Corollary 4.23: ATM is not Turing-recognizable Proof: • According to Thm 4.11a, ATM is Turing-recognizable. • Let us assume that ATM is Turing-recognizable. • Then ATM is co-Turing-recognizable. • According to Thm 4.22, this implies that ATM is decidable. • However, according to Thm 4.11, ATM is not decidable. • This is a contradiction. • Therefore, ATM is not Turing-recognizable.

18. Turing- recognizable Decidable Context-free Regular Summary • Now we know of at least one language in each set. • Thus we know that these 5 sets of languages are all distinct. • We have also proven that the containment shown is correct. ATM ATM 0n1n2n 0n1n 0n

19. HALTTM is not Decidable Thm 5.1: HALTTM = { M, w | M is a TM that halts on input w } is not decidable. Proof: • Assume HALTTM is decidable. • Then some TM R decides HALTTM. • Using R, we construct a TM S that will decide ATM as follows: S = “On input M, w, where M is a TM and w a string, • Run TM R on input M, w. // Determine if M halts with input w. • Reject if R rejects. // If the machine doesn’t halt, M, w ATM. • If R accepts, simulate M on w until it halts. // If it halts, then just run it. • Accept if M accepts, reject if M rejects. // Return the answer • Clearly S decides ATM. • But ATM is not decidable, so we have a contradiction. • Therefore, HALTTM is not decidable.

20. Looking forward • We proved that the Halting Problem was undecidable using a contradiction proof: • We proved that HALTTM decidable implies that ATM is decidable • But we already proved that ATM is undecidable • Therefore we have a contradiction, and HALTTM must be undecidable • We will use this technique repeatedly as we move forward, so it is important to understand why this proof works and how to apply the same technique to other problems.