Chapter 12: Theory of Computation

1. Chapter 12:Theory of Computation Computer Science: An OverviewEleventh Edition by J. Glenn Brookshear

2. Alan Turing Alan Turing was one of the founding fathers of CS. • His computer model –the Turing Machine– was inspiration/premonition of the electronic computer that came two decades later • He was instrumental in cracking the Nazi Enigma cryptosystem in WWII • Invented the “Turing Test” used in AI • Legacy: The Turing Award. Pre-eminent award in Theoretical CS

3. A Thinking Machine First Goal of Turing’s Machine: A model that can compute anything that a human can compute. Before invention of electronic computers the term “computer” actually referred to a person who’s line of work is to calculate numerical quantities! Turing’s Thesis: Any “algorithm” can be carried out by one of his machines

4. A Thinking Machine • Imagine a super-organized, obsessive-compulsive human computer. • The computer wants to avoid mistakes so everything written down is completely specified one letter/number at a time. • The computer follows a finite set of rules which are referred to every time another symbol is written down. • Rules are such that at any given time, only one rule is active so no ambiguity can arise. • Each rule activates another rule depending on what letter/number is currently read

5. Turing Machine Components A Turing machine is made of three components: a tape, a controller and a read/write head

6. Tape • Although modern computers use a random-access storage device with finite capacity, we assume that the Turing machine’s memory is infinite. • Think of infinite as meaning whenever we need more tape we add another reel of tape • The tape, at any one time, holds a sequence of characters from the set of characters accepted by the machine.

7. Read/Write Head • The read/write head at any moment points to one symbol on the tape. We call this symbol the current symbol. • The read/write head reads and writes one symbol at a time from the tape. • After reading and writing, it moves to the left or to the right or stops. • Reading, writing and moving are all done under instructions from the controller.

8. Controller • The controller is the theoretical counterpart of the central processing unit (CPU) in modern computers. • Think of it as a machine that has a predetermined finite number of commands it can process states and moves from one state to another based on the input.

9. A Thinking Machine Sample Rules: If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read □, write 1, HALT! Let’s see how they are carried out on a piece of paper that contains the reverse binary representation of 47 (reading right to left):

10. A Thinking Machine If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read □, write 1, HALT!

11. A Thinking Machine If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read □, write 1, HALT!

12. A Thinking Machine If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read □, write 1, HALT!

13. A Thinking Machine If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read □, write 1, HALT!

14. A Thinking Machine If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read □, write 1, HALT!

15. A Thinking Machine If read 1, write 0, go right, repeat. If read 0, write 1, HALT! If read □, write 1, HALT!

16. A Thinking Machine So the successor’s output on 111101 was 000011 which is the reverse binary representation of 48.

17. How Hard Can a Problem Be? • The Travelling Salesman Problem (TSP) is a problem in combinatorial optimization studied in operations research and theoretical computer science. • Given a list of cities and their distances from each other, the task is to find a shortest possible tour that visits each city exactly once. • The problem was first formulated as a mathematical problem in 1930 and is one of the most intensively studied problems in optimization. It is used as a benchmark for many optimization methods. • Even though the problem is computationally difficult, a large number of heuristics and exact methods are known, so that some instances with tens of thousands of cities can be solved.

18. How Hard Can a Problem Be? • The TSP has several applications even in its purest formulation, such as planning, logistics, and the manufacture of microchips. Slightly modified, it appears as a sub-problem in many areas, such as DNA sequencing. • In these applications, the concept city represents, for example, customers, soldering points, or DNA fragments, and the concept distance represents travelling times or cost, or a similarity measure between DNA fragments.

19. How Hard Can a Problem Be? • An optimal TSP tour through Germany’s 15 largest cities. It is the shortest among 43,589,145,600possible tours visiting each city exactly once. http://valis.cs.uiuc.edu/~sariel/research/CG/applets/tsp/TspAlg.html

20. How Hard Can a Problem Be? • The knapsack problem or rucksack problem is a problem in combinatorial optimization: • Given a set of items, each with a weight and a value, determine the number of each item to include in a collection so that the total weight is less than a given limit and the total value is as large as possible. • It derives its name from the problem faced by someone who is constrained by a fixed-size knapsack and must fill it with the most useful items. • Example of a one-dimensional (constraint) knapsack problem: which boxes should be chosen to maximize the amount of money while still keeping the overall weight under or equal to 15 kg?

21. How Hard Can a Problem Be? • The time required to solve the problem using any currently known algorithm increases very quickly as the size of the problem grows. • As a result, the time required to solve even moderately large versions of many of these problems easily reaches into the billions or trillions of years, using any amount of computing power available today. • As a consequence, determining whether or not it is possible to solve these problems quickly is one of the principal unsolved problems in computer science today.

22. How Hard Can a Problem Be? • Class P • Problems that can be solved with one processor in polynomial time O(N2), O(N3), etc • Exponential/Factorial – • Have known solutions but they are intractable because for data of any size they take too long to execute • Only solution is to try all possible solutions and pick the best • Class NP problems – problems that can be solved in polynomial time with as many processors as desired

23. How Hard Can a Problem Be? • Class P problems are also in Class NP. An open problem in theoretical computing is whether Class NP are also in Class P. • Meaning, are there polynomial-time algorithms for these problems we haven’t discovered yet • NP-Complete Problems • A special class of problems within Class NP that has the property that if a polynomial solution can be found for any member of the class, such a solution exists for every member of the class

24. The Halting Problem • The Church-Turing thesis states that if some algorithm exists to carry out a calculation, then the same algorithm can be carried out on a Turing machine. • It follows, that if we can find a problem for which a Turing machine solution can be proven not to exist, then the problem must be unsolvable. • The Halting Problem is such a problem.

25. The Halting Problem • When a program is running, it is difficult to know whether it is caught in an infinite loop or whether it just needs more time to run and finish. • Thus, it would be very beneficial if we could predict that a program with specified input would not go into an infinite loop

26. The Halting Problem • The Halting Problem poses the question as: Given a program and an input to the program, determine if the program will eventually stop with that input • One could wait to see if the program stops. If it doesn’t stop after 10 minutes will it stop after a 30 minutes, an hour, a day, a year, 100 years… • Unfortunately, this problem isunsolvable • How can we know that it is unsolvable? Try every solution and fail? How do we know we have all the possible solution to test? • It will be shown by contradiction

27. Loops in Javascript • Using loops in Javascript – for loop <html> <head><title>Loops in Javascript</title></head> <body> <script language ="Javascript"> for (i=1; i<=5;i++){ document.write("<br/>loop number " + i); } </script> </body> </html> • file:///C:/Documents%20and%20Settings/student/Desktop/loop.html

28. Loops in Javascript • Using loops in Javascript – for loop and read in when to stop <html> <head><title>Loops in Javascript</title></head> <body> <script language ="Javascript"> document.write("<br/>Loop program begins"); loopEnd = prompt("Enter how many lines to print",""); loopEnd = parseInt(loopEnd); for (i=1;i<=loopEnd; i++){ document.write("<br/>loop number " + i); } document.write("<br/>Loop program ends"); </script> </body> </html> file:///C:/Documents%20and%20Settings/student/Desktop/loop2.html

29. Infinite Loops in Javascript • What if we make a typo and enter loopend instead of loopEnd in the for loop or we took out the stop test <html> <head><title>Loops in Javascript</title></head> <body> <script language ="Javascript"> document.write("<br/>Loop program begins"); loopEnd = prompt("Enter how many lines to print",""); loopEnd = parseInt(loopEnd); for (i=1;i<=loopend; i++){ document.write("<br/>loop number " + i); } document.write("<br/>Loop program ends"); </script> </body> </html> file:///C:/Documents%20and%20Settings/student/Desktop/loop3.html

30. Infinite Loops in Javascript • Is it possible to write a CheckForInfiniteLoop program to read our original loop program to tell us if the loop program will enter an infinite loop? • This is the Halting Problem and through a proof by contradiction it can be shown that no such checking program can be written • Thus, the Halting Problem is unsolvable. Here’s a proof

31. Testing a program for self-termination • Assume there exists a Turing machine program called SolvesHaltingProblem that determines for any program Example and Input SampleData whether program Example halts given input Sample Data. • The program prints Halts if it halts and Loops if the program contains an infinte loops

32. Testing a program for self-termination • Since both a program and data are all binary to a computer, let’s give SolvesHaltingProblem program Example as both the program to test and another copy of itself as the input data

33. Testing a program for self-termination • Now construct a new program called NewProgram that takes program Example as both program and data and uses the algorithm from SolvesHaltingProblem to write “Halts” if Example halts and “Loops” if it doesn not • If “Halts” is written, NewProgram creates and infinite loop; if “Loops” is written, New Program writes “Halts”. • Finally, apply program SolvesHaltingProblem to NewProgram using NewProgram as data • If SHP prints “Halts”, NewProgram goes into an infinite loop • If SHP prints “Loops”, NewProgram prints “Halts” and stops • Because SHP gives the wrong answer in at least one case, it doesn’t work on all cases

34. Testing a program for self-termination

35. Proving the unsolvability of the halting program