The Halting Problem and Decidability

1. The Halting Problem and Decidability How powerful is a TM? Any program in a high level language can be simulated by a TM. Any algorithmic procedure carried out by a human or a computer can be carried out by a TM. “Algorithm” is an informal notion – but we have identified it with a TM. Questions Are there problems that cannot be solved by an algorithm? (existence) What are some example problems? (witness) The plan: Universal Turing Machines The Halting Problem Undecidability/unsolvability

2. Universal Turing Machines Interpreting and running programs is algorithmic. It is is a difficult task, but nevertheless can be carried out by an algorithm (e.g. a compiler). SO We can design a (universal) TM which “compiles” or “runs” other TMs. This is the universal TM – it takes as input a TM and its input tape, and runs the TM on that input. e(M)*x UTM outputs fM(x) iff M computes fM(x) i.e. it halts input loops iff M loops on x If M is a TM, then encode it by e(M). x is the input for M, * is a separator symbol. There are lots of ways to encode a TM, i.e. to construct e(M). Here is one example. NB. A TM may be very large, and have many input symbols, but they are all finite, hence a finite encoding.

3. Encoding a Turing Machine Consider a TM M with states Q and alphabet S (the input alphabet). Let A = {a1,a2, …} both infinite sets St = {q1,q2,…} Such that Q C St and SC A. Encode every symbol ai in A by e(ai)= ai, where i is the binary representation of i. For example, a5 is encoded by a101. Similarly, qi encodes qi. Assume a1 is the marker <, a2 is a blank, and q1 is the start state. Represent every halting state by a blank. So the transition (q,a),( ,a) is a transition into a halting state (trust me). The TM is represented by a sequence of transitions of form ((qi,aj),(qk,t)) where t is either ar (a symbol), or -> or <-. Note, qk might be blank. For example, the transition ((q5,a4),(q6,->)) is represented by ((q101,a100),(q101,->)). Assume the sequence of transitions is ordered lexicographically. (Doesn’t matter which ordering, pick one). Encode the input string in the same way.

4. Encoding a Turing Machine Now, to simulate M, assume a 3-tape version for the UTM U. Tape 1 has e(M)*e(x) Copy e(M) on to tape 2 Shift e(x) to leftmost position of tape 1. Tape 1 will be used to simulate the I/O of M. Note that tape 1head does not look at a symbol from x, but an encoding of it, e.g. a101 instead of a5. So, before any simulation step assume head is a first symbol of encoding. E.g. a101. 4. Tape 3 is used to hold qi – the encoding of the current state qi in M. To simulate a step in M, assume U is in the state (corresponding to ) qj in M (qj on tape 3), and tape 1 head points at ai (tape 1). Look up transition for (qj,ai) on tape 2 and simulate that transition, leaving new state on tape 3 and move tape 1 head accordingly. This concludes simulation of M by U.

5. Undecidability/unsolvability Undecidability/unsolvability – shades of the same thing. A problem is unsovable if there exists no Turing machine to compute it Question1 . Are there problems that cannot be solved by an algorithm? (existence) Yes. Proof: Show all TMs can be enumerated Show all partial functions cannot be enumerated. (Diagonalisation) Encode every TM M by a unique number n(M) (Godel numbering) Encode M by e, as for UTMs. ie consider e(M). Convert every transition to binary, then concat all together, with inputs. Result is one long string which can be interpreted as a number. ii) Enumerate all the TMs as follows Tk is M iff n(M) = k is T otherwise, where T is a special non-halting machine

6. Undecidability/unsolvability 2. There exists a (partial) function f:N->N that is not (partial) Turing computable, i.e. there is a function without a corresponding machine. Proof: Diagonalisation. For every k, let fk be the function computed by Tk. Now construct a “diagonal” function f: f(x) = f(x) +1 if x is in dom(f(x)) = 0 otherwise e.g. if f(3) = 6, then f(3) = 7. 0 1 2 3 … f(0) f(0)<> f(0) f(1) f(1)<> f(1) f(2) f(2)<> f(2) . . . Clearly, f differs from every f at every single value. So f isn’t in the enumeration. Thus there exists a function which has no corresponding TM! Let’s consider a more concrete example.

7. The Halting Problem Problem: Design a machine H such that outputs 1 e(M)*e(x) H iff M halts given input x outputs 0 Input iff M does not halt given input x Theorem: The Halting problem is unsolvable Proof: By contradiction. Assume H exists and then derive a contradiction. Construct the following machine H’: e(M) copier e(M)*e(M) modified loops iff H M halts given input (e(M)) outputs 0 iff M does not halt H’ given input e(M)) Modified H – just like H, except instead of outputting 1, loops forever. Now, let e(H’) be the input to H’. If H’ loops => output is 0 (by defn of H’ when it outputs 0) If H’ outputs 0 => loops (by defn of H’ when it loops) CONTRADICTION! So H cannot exist.

8. Implications There is no algorithm which can take an arbitrary program and arbitrary input, and tell you whether or not the program will halt. You may be able to show this for a particular program, or even a sub-class of programs, but there is no general strategy for all programs. E.g. While b do …. For i=1 to x do …. The world of computing is littered with (interesting!) unsolvable problems!

9. Decision problems Unsolvable problems: yes/no solutions Examples Does z = x+y, (for integers x,y,z) Does TM M halt given x as input. Does TM M halt for very input x in S*. Does TM M halt for given input e. Are two TMs equivalent. Does a diaphantine equation have a solution. (Hilbert’s 10th problem) A DE has form p(x1,…xn) = 0 e.g. x2+y2 = z2 x2-2 = 0 7. Is a predicate calculus statement valid. (1936, Church)

10. Reduction Arguments by reduction: Defn A decision problem P reduces to P’, P->P’, if an algorithm to solve P’ can be used directly to solve P. P->P’ iff e(I) TM e’(I’) I is instance of P, I’ is instance of P’. ---------------------------------------------------------------------------------------------------- Theorem: If P->P’ and P’ is solvable, then so is P’ . Corollary: If P->P’ and P is unsolvable then so is P’. e(I) TM e’(I’) P’ 1 iff I’ in Y(P’) iff 1 in Y(P) P->P’ TM 0 iff I’ in N(P’) iff 1 in N(P)