Remaining Topics

1. Remaining Topics • Decidability Concept 4.1 • The Halting Problem 4.2 • P vs. NP 7.2 and 7.3 • NP-completeness & Cook-Levin Theorem 7.4

2. Review: Turing Machines in a nutshell Church-Turing Thesis • Turing Machine equal Notion of an Algorithm Turing Machine • Most simple machine possible • Computational power of • modern computer and • high-level language • Not particularly efficient in a practical sense

3. Review: Turing Machines in a nutshell Most simply model possible… • Adding another tape can improve • efficiency (time or computational “speed”) • but not computational “power”(ability to solve a problem). • Every multi-tape TM has an equivalent single-tape TM (Theorem 3.13, p.149) • Similar to an Automata Non-determinism does NOT add any “computational power” (Theorem 3.15, p.150)

4. Review: Turing Machines in a nutshell • Computational power of • modern computer and • high-level language • Every operation and statement in a high level language can be implemented with a Turing Machine (TM) • Just as statements can be combinedSo can TMs (HW5 illustrates this)

5. Decidability 4.1 • Is there an algorithm that can decide if • An item is in a set. • A string is in a language • A formula is a member of a theory • These are all variations of the same concept, i.e., the concept of decidability

6. Decidability in Languages • We will concentrate on this: • Algorithms for deciding if a string is in a language • But, the strings and languages are going to represent deeper problems ADFA = {<B,w> | B is a DFA that accepts input string w}

7. ADFA ADFA = {<B,w> | B is a DFA that accepts input string w} • B is the encoding of a DFA • Remember that you can encode a DFA as follows: • B = (Q,Σ, δ, qstart, F) • We are literally encoding the machine and the input (w) as a string “<({1,2,3},{a,b},{(1,2,a),(1,3,b)},1,{3}),abc>”

8. ADFA ADFA = {<B,w> | B is a DFA that accepts input string w} • Testing whether a DFA accepts an input w is the same as the problem of testing whether the string <B,w> is a member of the language ADFA • Just as A = {w | w = (11)*} would accept the set {ε, 11, 1111, 111111, …} • ADFA would enumerate all the <B,w>’s such that w is accepted by the encoded B.

9. ADFA is decidable ADFA = {<B,w> | B is a DFA that accepts input string w} • What does this mean in plain English? • How can we prove it?

10. ADFA is decidable ADFA = {<B,w> | B is a DFA that accepts input string w} • What does this mean in plain English? • “An algorithm exists that can accept strings that adhere to the definition of ADFA and reject string that don’t”

11. ADFA is decidable ADFA = {<B,w> | B is a DFA that accepts input string w} • How can we prove it? • Proof is on p.167

12. ANFAis decidable ANFA = {<B,w> | B is a NFA that accepts input string w} • How do we know this to be true? • Hint: How are NFAs and DFAs different?

13. AREX is decidable AREX = {<R,w> | R is a Regular Expression that generates the string w} • How do we know this to be true? • Hint: How are Regular Expressions and DFAs related?

14. EDFA is decidable EDFA = {<A> | A is a DFA and L(A) is empty} • Prove it • Hint: Just as Turing Machine can “simulate” a DFA it can also determine if a state is unreachable.

15. EQDFA is decidable EQDFA = {<A,B> | A and B are DFAs and L(A) = L(B)} • How do we know this to be true? • Hint: Symmetric Difference formula

16. Decidability of Regular Languages • Deciding if • a language is Regular or not • If given DFA, NFA or REX • a Regular language is empty • two Regular languages are equal

17. Decidability of Context Free • Deciding if • a language is Context Free or not (Theorem 4.7) • If given CFG • a Context Free language is empty (Theorem 4.8) • two Context Free languages are equal

18. Classes of languages Turing-recognized Decidable Context-Free (ACFG) Regular (ADFA)

19. The Halting Problem • Will an algorithm halt on a given input. • Intuition: • Can you ever be sure that a loop is infinite? • It might just terminate in a few minutes, hours, years, millenniums, etc. • Sometimes you can make such a determination: • while (x > 0) {x=1;} • But is it always possible to make such a determination?

20. Infinite Looping • DFA: • by definition, upon consuming the input, the machine rejects unless it is in an accept state. • Looping is simply not an option by definition. • PDA: • very, very hard to make deterministic PDA’, but it can be done. • Once the input is consumed, empty transitions can move to a reject/accept state. • Every CF language has a PDA that will halt (not loop). • TM: • Just like a high-level language TMs can loop forever. • Intuition: you don’t consume the input, you can move on the tape infinitely, and the states can have a loop with no accept or reject.

21. ATM • ATM= {<M,w> | M is a TM and M accepts w} • U = “on input <M,w> simulate M on w” • If M accepts, U accepts • If M rejects, U rejects • Simple intuition: • M could be a Turing Machine that loops forever on certain input. • If M loops forever, U cannot be a decider for ATM

22. Is ATM decidable? • ATM= {<M,w> | M is a TM and M accepts w} • U = “on input <M,w> simulate M on w” • If M accepts, U accepts • If M rejects, U rejects • BUT! Perhaps there is a way to implement M such that we can detect the infinite loop? • Upon infinite loop detection, U rejects. • U could still be a decider for ATM

23. The Halting Problem • H(<M,w>) = if M accepts w accept if M rejects w reject

24. The Halting Problem is Undecidable • Proof: First, consider the machine/algorithm D: • D = “on input <M>, where M is a TM: • Run H on input <M,<M>> • Output the opposite of what H outputs; that is; if H accepts, reject and if H rejects, accept.” Recall H: • H(<M,w>) = if M accepts w accept if M rejects w reject

25. D is a crazy Decider Algorithm • D is implemented with a Turing Machine • D(<M>) = if D does not accept <M>, accept if D accepts <M>, reject • What happens if we run D with its own Turing Machine description? • D(<D>) = if D does not accept <D>, accept if D accepts <D>, reject

26. A paradox emerges • D(<D>) = if D does not accept <D>, accept if D accepts <D>, reject • If D accepts, how can D(<D>) reject? • We assumed that H could decide ATM because it could ‘somehow detect an infinite loop” • Think of H as a deterministic decider if a Turing Machine loops • Then, we use H to build D (the crazy decider) • Here we assume H can stop D from looping infinitely • Then, we run D on its own encoding, which creates a paradox.

27. Paradox resolved • Either H or D cannot exist. • Which one? • D is a TM machine that can simulate another Turing Machine, which has been elegantly proven. • Intuition: Consider a program that can take another program and simulate its execution. • Program, Algorithm, and Turing Machine are all synonymous (Church-Turing Thesis) • Compilers • Virtual Machines

28. Significance of Turing Machines • Turing Machines are the “tool” we used to prove that the Halting Problem is un-decidable. • In other words, no algorithm exists to determine if a general algorithm will halt or not. • Note: There are some algorithms where its easy to show/prove that it will halt, but we are interested in the general case (any/all algorithms).

29. Un-decidable Languages… • …there are many, but this is the interesting one: • ATM= {<M,w> | M is a TM and M accepts w} • Obviously, this language can’t be generated by a REX or CFG. • So, a NFA, DFA, and PDA can’t be used as a decider to accept/reject strings • But, even a Turing machine cannot act as a decider. • It may be able to decide some input on some machines, but not all. • There are strings in ATM that will cause the decider to loop infinitely. Specifically <D,<D>> and likely other strings.

30. Significance of ATM • A formal language that cannot be decided by Turing Machine. • We can define this language’s concept • But we cannot create an algorithm (TM) to determine if a string is in this language or not. ATM Turing-Decidable Context-Free (ACFG) Regular (ADFA)

31. Decidable vs. Recognizable Turing Decidable Languages • Language such that some TM will accept all of its strings • And, reject strings in the language’s compliment • Halts on all input Turing Recognizable Languages • Language such that some TM will accept all of its strings • But, might not halt on strings in the language’s compliment • Its it looping infinitely or will it accept? • We don’t know.

32. ATM is Turing Recognizable • ATM= {<M,w> | M is a TM and M accepts w} • U = “on input <M,w> simulate M on w” • If M accepts, U accepts • If M rejects, U rejects • U will always halt if M halts. If M doesn’t halt on w than M doesn’t accept w, so <M,w> isn’t in the language. By its very definition U will always halt on strings in ATM. • The un-decidability is when U has been looping for 10 million years, • we really don’t know • Is it eventually going to be an accepted w or • an infinite loop caused by a rejected w. This is why infinity is trouble.

33. Time Complexity • TM’s are a formal way to describe algorithms • Some problems don’t have algorithms that will always halt, i.e., determining if a string is in ATM. • Algorithms that do halt can still take a long time. • How long is long?

34. General Time Unit • With Turing Machines we can define a unit of time to be the execution time of one TM transition. • With more practical machines, a time unit could be a CPU clock cycle, which might execute one machine-level instruction. • Some machines can execute 1 billion instructions per second, so the time unit would be 1/100000000 seconds.

35. Time as a function of input size • N is the size of the input • f(N) is the number of time unit to solve the problem. • The running time of algorithms can be expressed as functions: • f(N) = 2N + 5; • Two loops of size N and 5 setup instructions • Or, on loop of size N with two instruction inside and 5 instructions outside the loop.

36. Constants don’t matter • For really big problems, constants don’t matter • f(N) = 2N is the same as g(N) = 100N • While 100 days seems like forever compared to 2 days, parallel computation and faster computers can eventually make up the difference (we hope). • For big problems, • f(N) = N4is much different than g(N) = N2 • A faster computer may not help, why?

37. 1.15 Days vs. 11 billion years

38. Big-O • Review • Constants don’t matter • Only the leading exponent matters • Why?

39. 1.157407 days vs. 1.157419 days

40. Why do we only care about big N’s • Same reason I would worry about a $10,000 bill in my wallet but not a penny. • Same reason I would worry about a trip to Mars but not a trip to Menands.

41. Real Algorithm  TM Decider • Prepare for “hand-waving magic:” • Any algorithm that can be programmed can be reduced into a language problem. • A = {<p,i,o> | p is the encoding/description of a problem, i is the input, and o is the correct output.} • Deciding if a string is in L is the same thing as solving the problem. • The TM that decides A solves problem p.

42. The class P • The class of languages that can be decided in polynomial time. • Corresponds, the set of problems that can be solved in polynomial time. • Polynomial is O(nk) • What are some problem in P that you have studied?

43. Did you know? • Every context free language is in P

44. The class NP • Non-deterministically Polynomial. • One way to think of this is NOT Polynomial. • Or, exponential • Or N! • But that is not the whole story.