Foundations of Software Design Fall 2002 Marti Hearst

1. Foundations of Software DesignFall 2002Marti Hearst Lecture 29: Computability, Turing Machines, Can Computers Think?  

2. Computability • Is there anything a computer cannot compute? • Linked to the notion of what is an algorithm.

3. Alan Turing • An amazing scientist • Helped solved the Enigma Machine (WWII) • Advances in Probability Theory • Invented the theory behind computers • Turing Machine • Turing Test

4. Turing Machines • Anything that can be computed by a finite set of rules can be computed by a Turing machine. • Turing Equivalence • We saw that finite automata are less powerful than TMs • Can’t compute • Thus not Turing-equivalent • Modern computers and programming languages are all Turing-equivalent

5. Languages accepted by Turing Machines Context-Free Languages Regular Languages Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

6. A Turing Machine Tape ...... ...... Read-Write head Control Unit Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

7. The Tape No boundaries -- infinite length ...... ...... Read-Write head The head moves Left or Right Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

8. ...... ...... Read-Write head The head at each time step: 1. Reads a symbol 2. Writes a symbol 3. Moves Left or Right Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

9. Example: Time 0 ...... ...... Time 1 ...... ...... 1. Reads 2. Writes 3. Moves Left Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

10. The Input String Input string Blank symbol ...... ...... head Head starts at the leftmost position of the input string Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

11. States & Transitions Write Read Move Left Move Right Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

12. Example: Time 1 ...... ...... current state Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

13. Time 1 ...... ...... Time 2 ...... ...... Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

14. Determinism Turing Machines are deterministic Not Allowed Allowed Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

15. Halting The machine haltsif there are no possible transitions to follow Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

16. Example: ...... ...... No possible transition HALT!!! Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

17. Final States Allowed Not Allowed • Final states have no outgoing transitions • In a final state the machine halts Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

18. Acceptance If machine halts in a final state Accept Input If machine halts in a non-final state or If machine enters an infinite loop Reject Input Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

19. Infinite Loop Example A Turing machine for language Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

20. Time 0 Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

21. Time 1 Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

22. Time 2 Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

23. Time 2 Time 3 Time 4 Time 5 ... Infinite Loop Adapted from slide by Costas Busch, http://www.cs.rpi.edu/courses/fall01/modcomp/

24. Church-Turing Thesis The Church-Turing thesis says that Turing machine algorithms are the same as our intuitive notion of algorithms. • Most people think the Church Turing thesis is correct. • It means, among other things that • You can simulate any computer program that runs on any hardware in any language with any other program on any other hardware using any other language. • Tasks that humans agree is an algorithm can always be executed on a computer. Adapted from slides by Lee Wee Sun, http://www.comp.nus.edu.sg/~cs1305/2000/

25. The Halting Problem • An example of something that is not computable. • Created by Turing in 1936 to define a problem which no algorithmic procedure can solve. • Can we write a program that will take in a user's program and inputs and decide whether • it will eventually stop, or • it will run infinitely in some infinite loop ? Adapted from slides by Lee Wee Sun, http://www.comp.nus.edu.sg/~cs1305/2000/

26. Proof (by contradiction) • Assume that it is possible to write a program to solve the Halting Problem. • Denote this program by HaltAnswerer(prog,inputs). • HaltAnswerer(prog,inputs) will • return yes if prog will halt on inputs and • no otherwise. • A program is just a string of characters • E.g. your Java program is just a long string of characters • An input can also be considered as just a string of characters • So HaltAnswerer is effectively just working on two strings Adapted from slides by Lee Wee Sun, http://www.comp.nus.edu.sg/~cs1305/2000/

27. Proof (cont.) • We can now write another program Loopy(prog) that uses HaltAnswerer • The program Loopy(prog) does the following: [1] If HaltAnswerer(prog,prog) returns yes, Loopy will go into an infinite loop. [2] If HaltAnswerer(prog,prog) returns no, Loopy will halt. Adapted from slides by Lee Wee Sun, http://www.comp.nus.edu.sg/~cs1305/2000/

28. Proof (cont.) [1] If HaltAnswerer(prog,prog) returns yes, Loopy will go into an infinite loop. [2] If HaltAnswerer(prog,prog) returns no, Loopy will halt. • Consider what happens when we run Loopy(Loopy). • If Loopyloops infinitely, • HaltAnswerer(Loopy,Loopy) return no which by [2] above means Loopy will halt. • If Loopyhalts, • HaltAnswerer(Loopy,Loopy) will return yes which by [1] above means Loopy will loop infinitely. • Conclusion: Our assumption that it is possible to write a program to solve the Halting Problem has resulted in a contradiction. Adapted from slides by Lee Wee Sun, http://www.comp.nus.edu.sg/~cs1305/2000/

29. Diagonalization • Not Diagon Alley • A proof by contradiction technique • Uses the notion of Infinity • The name comes from a proof that shows you can’t ever list all numbers. • Assume you list all possible binary numbers • Diagonalization shows you can always construct a new number that is not yet in the list • Have to assume numbers can have infinite length • Construct the new number by choosing the opposite of the number on the diagonal.

30. Diagonalization B1 0 0 0 0 0 … B2 0 0 1 0 1 … B3 1 0 1 1 0 … B4 0 1 1 1 1 … B5 1 1 0 0 0 … … D 1 1 0 0 1 …

31. The Universal Turing Machine • There are an infinite number of Turing Machines • There are an infinite number of calculations that can be done with a finite set of rules. • However, we can define a Universal Turing Machine which can simulate all possible TMs • Comes from the definition of TMs • You convert the description of the TM and its input into two tapes, and use these as the input to the UTM

32. The Halting Problem • Halting Problem: • There is no procedure for telling whether an arbitrary TM will halt on a given input. • Use Diagonalization to show this. • Again, proof by contradiction • Assume there is a rule for deciding if a TM will halt. • Construct a table as follows: • List all Turing machines down the side • List the possible inputs across the top • In position (j,i) put the result of executing Turing machine j on input i • If it halts, output H • If it doesn’t halt, output ?

33. Diagonalization 1 2 3 4 5 … T1 H H ? H H … T2 ? ? H H H … T3 H H H H H … T4 H H H ? H … T5 H H H H H … … D ? H ? H ? …

34. Diagonalization on the Halting Problem • Now define a new TM called D that will halt for all inputs. It outputs • H if TMi(i) does not halt • ? if TMi(i) does halt • We already said that the assumption is that we can always decide if a TM halts. • Also, we said this table lists all possible TMs. • So D must be in the table. • But this means that we are saying that D outputs halt if it doesn’t halt! • To see this, give D as input to itself. • This is a contradiction. Hence the premise does not hold: We cannot determine if an arbitrary program will halt.

35. “Going Meta” • This proof based in part on Gödel’s Theorem • If you are interested in these kinds of questions (and other things related to “going meta” like compiler compilers), see • Gödel, Escher, Bach: An Eternal Golden Braid, by Douglas Hofstadter.

36. The Turing Test • An observer • Interacts with a keyboard and monitor • Has to distinguish which of two respondents is a computer and which is human. • There is a contest with a $100,000 prize! • For the first computer whose responses are indistinguishable from a human's. • The Loebner Prize http://www.loebner.net/Prizef/loebner-prize.html • It’s actually pretty easy to fool people over the short term. • Chat room ’bots work quite well.

37. What is Intelligence? • Do androids dream … ? • What would it take for a computer’s thoughts to be indistinguishable from a human’s? • THIS is the deepest question of CS.