The Halting Problem

1. The Halting Problem • Can we design a program that, given any other program and its input, tells whether that program will halt when run on that input? • Output of this program, called HALT, when applied to program P and input I, is “halts” if P(I) eventually halts • Output of this program is “never halts” if P(I) never halts • Theorem (Alan Turing 1937):No computer program cansolve the halting problem.

2. Proof by Contradiction • Assume the hypothesis to be proven is false, i.e. there is a computer program that can solve the halting problem • Show that this assumption leads to a contradiction • Do it by creating a program and an input to it that generate this contradiction

3. Proof by Contradiction • Suppose there was a computer program,called HALT, which solved the halting problem. • We can write a new program that uses HALT and leads to a contradiction: Program NonConformist (Program P) If ( HALT(P) = “never halts” ) Then Halt Else Do While (1 > 0) Print “Hello!” End While EndIf End Program

4. Proof by Contradiction Program NonConformist (Program P) If ( HALT(P) = “never halts” ) Then Halt Else Do While (1 > 0) Print “Hello!” End While EndIf End Program • Does NonConformist(NonConformist) halt? • Note: It calls HALT(NonConformist) • Yes? Means HALT(NonConformist) = never halts • No? That means HALT(NonConformist) = halts • Contradiction: There exists a program (NonConformist) for which the HALT program gives the wrong answer

5. Undecidability • Halting Problem:Given a computer program P and input “input”: • Output “halts” if P(input) eventually halts • Output “never halts” if P(input) never halts • We’ve shown that the Halting Problem is Undecidable – no computer program can ever solve it, no matter how powerful the computer is.

6. Undecidability • Many other problems are undecidable, too. • Moreover, the same ideas that we used to prove undecidability can be used to prove a very disturbing statement.

7. GÖDEL’S INCOMPLETENESS THEOREM • Until Gödel came along, Mathematicians were searching for the “one true logical framework,” i.e. a framework in which every true statement can be proved. • Their faith was shattered by Gödel. • For more on this and its philosophical implications, see Gödel, Escher, Bach by Douglas Hofstadter.

8. GÖDEL’S INCOMPLETENESS THEOREM • In 1931, Gödel stunned the world by proving that in any logical framework in which you can express basic facts about numbers, there exists a true statement that cannot be proved in that framework. • “Not every true statement has a proof”

9. GÖDEL’S INCOMPLETENESS THEOREM • In 1931, Gödel stunned the world by proving that in any logical framework in which you can express basic facts about numbers, there exists a true statement that cannot be proved in that framework. • But wait, if it can’t be proved, how do we know it is true?

10. GÖDEL’S INCOMPLETENESS THEOREM • We can prove that it is true by “jumping” outside the original logical framework to a “larger” logical framework…

11. GÖDEL’S INCOMPLETENESS THEOREM • We can prove that it is true by “jumping” outside the original logical framework to a “larger” logical framework… • But that larger logical framework also has a true statement that cannot be proved!

12. GÖDEL’S INCOMPLETENESS THEOREM • We can prove that it is true by “jumping” outside the original logical framework to a “larger” logical framework… • But that larger logical framework also has a true statement that cannot be proved! • …and so on, and so on… • No matter when we stop, there will always be some true statement that doesn’t have a proof!

13. GÖDEL’S INCOMPLETENESS THEOREM • Until Gödel came along, Mathematicians were searching for the “one true logical framework,” i.e. a framework in which every true statement can be proved. • Their faith was shattered by Gödel. • For more on this and its philosophical implications, see Gödel, Escher, Bach by Douglas Hofstadter.

14. More Undecidable Problems? Post's Correspondence Problem (PCP) • An instance of PCP of size s is a finite set of pairs of strings (gi , hi) [for i = 1...s; s>=1] over some alphabet . • A solution is a sequence i1 i2 ... in of selections from each set of strings (gi , hi) such that the strings gi1gi2 ... gin and hi1hi2 ... hin formed by concatenation are identical.

15. Sample PCP • g1 = aba h1 = abaa • g2 = bbab h2 = abab • g3 = baaa h3 = a • g4 = a h4 = bb So, 1,3,1,2 would correspond to • aba baaa aba bbab from g’s • abaa a abaa abab from h’s Not a solution!

16. Another Sample PCP • g1 = aba h1 = abaa • g2 = bbab h2 = abab • g3 = baaa h3 = a • g4 = a h4 = bb 1,4,2,1,3 corresponds to • aba a bbab aba baaa from g • abaa bb abab abaa a from h Solution!

17. PCP is undecidable? • PCP shown to be undecidable by Post in 1946. • What about PCP with limited-size inputs • PCP with size 2 has been proved decidable. • PCP with size 7 has been proved undecidable • The decidablility of problems with size between 3 and 6 is still pending.

18. What Computers Can’t DoIn Your Lifetime • We’ve now seen examples of problems that computers can’t solve, even if computers have unlimited speed and unlimited time. • Are there more real world problems (eg that arise in business, science, … ) that can be solved but take far too much time to be solved in practice?

19. “Search” Problems • We’ll focus on what we’ll call “search problems”.(In Computer Science terms, we’ll be talking about what are termed NP problems) • Intuitively, a search problem is a problem where you are looking for something which you can recognize quickly if you find it. • Recognizing a good solution is easy • Problem is finding it

20. Example: Coloring

21. Example: Coloring with 3 colors • Suppose we are given a collection of circles (nodes). • Some circles connect to others by edges, forming a graph • Rule: No two connected circles can have the same color. • You only have three colors (Green, Red, Yellow) • Is there a valid coloring? • Note: easy to check validity. Hard to find?

22. Search Problems:More Precise Definition • Intuitively, a search problem is a problem where you are looking for something which you can recognize quickly if you find it. • A bit more precisely, we require that there is a small circuit that can quickly check the validity of a solution. • For coloring, it would just be a circuit that checks that for every pair of connected circles, the colors are different.

23. Another Example: Traveling Salesperson Problem • A saleswoman wants to visit n different cities. • She knows the costs associated with flights between each city. • Can she visit all the cities spending less than $B in total? • Note: Easy to check that a given flight plan visits all the cities and costs less than $B. • Seems hard to find the flight plan …

24. Search Problems… • Unfortunately, we don’t know how to solve these and many other search problems with a computer in our lifetimes for large inputs. • Large graphs, large number of cities • For many years, computer scientists wondered which search problems could be solved, and which couldn’t. • But just because computer scientists couldn’t solve the Coloring problem for 40 years doesn’t mean it is impossible, right? • Proof of Fermat’s Last Theorem took over 300 years

25. Search Problems… • Can we prove that it is impossible to solve them quickly for large inputs? • Unfortunately, not yet • But now we are much more confident that they really are impossible to solve quickly • How do we make progress toward determining an answer?

26. Breakthrough: Reduction • A surprise: • In the early 70’s, Cook, Levin, and Karp showed us that if we can solve the Coloring problem quickly, then we can solve ALL search problems quickly! • But general search problems are defined in terms of circuits (that can validate their solutions), not colors… • Need to map circuits to graph coloring problems • Then we can map the circuit for any search problem to the corr. graph coloring problem

27. Coloring and Circuits • Suppose we think of : • Green as meaning True. • Red as meaning False • Yellow as meaning nothing

28. Input X Output NOT X What is the circuit for this graph? Valid coloring for this graph leads to a NOT gate T F So this is a graph for a NOT circuit

29. F T Y X Output

30. F T Y X F T Output

31. F T Y X Output

32. F T Y X Output

33. F T Y X Output

34. F T Y X Output

35. F T Y X Output

36. F T Y X Output OR =

37. Coloring ! • And you thought coloring was for kids.. • In fact, we can encode any circuit into a collection of connected circles (graph) waiting to be colored. • Any valid coloring for the graph conforms to the circuit (which verifies a search problem) • Thus, we can efficiently reduce any search problem to Coloring. • If we can solve Coloring quickly, we can solve any search problem quickly!

38. Many more • Traveling Salesperson Problem is also as “hard” as any search problem. • Search Problems with this property are called complete problems • Although we don’t know how to prove that they are hard, we know that if find a way to solve one of them quickly, we can solve all search problems quickly! • Gives us more confidence that they really are hard.

39. Recap: Hard search problems • A hard search problem is a problem where • It is hard to find a solution • It is easy to check possible solutions for validity • A complete search problem is a problem that is as hard as any search problem • Search problem is believed to be hard because • No one found a way to solve any of the complete search problems quickly

40. What does “Quickly” mean? • In time polynomial in the size of the input • E.g. if one could solve an n-city TSP in time • n2+2n+5 • or even n2000+7n1999+…+8n2+5n+6 • Then these are a Polynomial-time algorithms (quick) • But if one can only solve it in time: • 2n • or 2n+6n-1+…+3 • Then these are Exponential-time algorithms (slow)

41. n2 versus 2n • n = 10 • n2 = 100 • 2n = 1024 • n = 100 • n2 = 10,000 • 2n = (1024)10 > 1 trillion * 1 trillion * 1 million Hence we call exponential-time algorithms slow

42. Classes of search problems • In computer-science terminology: • NP = All Search Problems • P = Problems we can solve quickly • We believe that P  NP, i.e. not every search problem can be solved quickly on a computer.

43. NP-Complete Problems • Coloring is complete • In particular, we can reduce solving any search problem to finding a valid coloring for some collection of circles! • So, if we could solve Coloring quickly, then P = NP • That’s why we believe Coloring can’t be solved quickly by any computer. • We call such problems NP-Complete.