Programming Languages Wrap-up

1. Programming Languages Wrap-up

2. Your Toolkit • Object-oriented • Imperative • Functional • Logic

3. Your problem… • Make a GUI front end for a store inventory system • Go through a file and count occurrences of certain words • Given a graph, find all paths from node A to node B.

4. Making a new language • Why? • What do you need to decide?

5. Theory Component • Provides mathematical models of • Programming languages (grammars) • Computers (automata) • And how they work together • Gives foundation for studying what programming languages can and can’t do.

6. Turing Machines • Read input letter & tape letter, write tape letter, move left or right.

7. Example: anbncn • Read an a, Replace it with an x. • Move right over the rest of the a’s. • Read a b, replace it with a y. • Move right over the rest of the b’s • Read a c, replace it with a z. • Move all the way left until you see an x (then you’re at the beginning again). • Repeat until all a’s have been taken. • Make one last pass over string to make sure only x,y, and z are remaining.

8. Turing’s Thesis • Can Turing Machines do anything computers can do? • Hmm - depends on the definition of “computer”! • Turing’s thesis(mid-30’s): Any computation carried out by mechanical means can be carried out by a Turing Machine.

9. Turing’s thesis: true or false? • Can’t prove it, because we can’t foresee what “mechanical means” people might come up with in the future. • Could disprove it with one counterexample...

10. Evidence Supporting Turing’s thesis • Anything that can be done on an existing digital computer can be done on a Turing Machine • We can prove this - just write a TM for each machine instruction. • No one has been able to suggest a problem solvable by an algorithm but not a TM • Many alternative models of digital computers have been proposed - but none are more powerful than TMs

11. Algorithm definition • An algorithm for a function is a Turing Machine that halts with the correct answer on the tape for any input in the domain.

12. Other types of automata • Given another type of automata, how do you know which is more powerful? • Simulation - show how anything done with one type of automata can be done with the other • Example: I can simulate with a PDA any FSA. Therefore PDAs as powerful (maybe more) than FSAs.

13. Example: • Any FSA can be written as a PDA: • So PDAs are at least as powerful.

14. Variations on TMs • Add a stay-option • Tape is only unbounded in one direction • Separate read-only tape for input (off-line TM) • Multiple storage tapes • Multi-dimensional tape (extends infinitely in 2 dimensions) • Non-deterministic TMs (not presently possible with today’s computers) All these are equivalent automata to Turing Machines!

15. Non-deterministic TMs • An interesting class of TMs... • Equivalent in power to TMs (so everything computable by a ND-TM is computable by a TM) • BUT, it takes more steps on a TM than a ND-TM. We think it takes exponentially more steps... • This is where the classes P and NP come from... • P: set of all problems computable by a TM in polynomial time • NP: set of all problems computable by a ND-TM in polynomial time • P = NP? Unknown...

16. Detour into 331... • NP-Complete: set of all problems in NP that are polynomial-time reducible to ALL problems in NP. • Meaning, if you can solve 1 NPC problem in polynomial time, you can solve ALL NP problems in polynomial time, and P = NP • Cook’s Theorem: SAT is NP-Complete • How did this work? He showed that every ND-TM has an equivalent SAT expression!! • From that it is easy to reduce other NP-problems to SAT to prove that they too are NPC: • Hamiltonian paths (Traveling salesman) • Cliques

17. Recursively Enumerable Languages • A language is recursively enumerable if there is a Turing Machine that accepts it. • This allows the TM to go into an infinite loop if the string is not in the language.

18. Recursive Languages • A language is Recursive if there is a TM that accepts the language and it halts on every string in Σ+. • That is, a recursive language has to have a membership algorithm.

19. Recursively Enumerable, contd • Not all languages are recursively enumerable. • Some languages are recursively enumerable but not recursive. • Unrestricted grammars produce recursively enumerable languages. These statements are proven, yet examples are hard to come by...

20. Context-Sensitive Grammars • All productions are of the form x -> y, where x and y are elements of (VUT)+ and |x| <= |y| Example: S -> ABCAB -> aaabC -> DFDF -> cccab

21. Context-Sensitive Languages • CSL example: anbncn • Automata for CSLs are linear-bounded automata: TM’s where only the area of tape used by the input is allowed as storage. • All CSL’s are recursive, but not all recursive languages are context-sensitive.

22. Chomsky’s Hierarchy Recursively Enumerable Recursive Context-Sensitive Non-det Context-Free Det Context-Free Regular

23. So can anything be computed?No! • Computability – the study of which problems can and which cannot be solved by a Turing Machine.

24. The Halting Problem • Given an algorithm decide will it halt? • Assume it can be done: Halt(M,w) = yes if algorithm (TM) M halts on input w, and no if it doesn’t. That is: • Halt(M,w) = yes if M(w) halts, else no. • Create new algorithm:Strange(String s): if Halt(s,s)= “no” then return yes else while(true){ }

25. Halting Problem Strange(String s): if Halt(s,s)= “no” then return yes else while(true){ } So, for any program P, if P(P) halts then Strange(P) doesn’t halt. if P(P) doesn’t halt then Strange(P) does halt. But that means: if Strange(Strange) halts then Strange(Strange) doesn’t halt. if Strange(Strange) doesn’t halt then Strange(Strange) does halt. Contradiction!

26. Computability • So, not all problems can be solved with a computer. • The Halting Problem is said to be “undecidable”. • DO NOT confuse this with an NP-complete problem! Remember, problems in NP are solvable by non-deterministic TMs and therefore also solvable by TMs, just not necessarily in polynomial time.

27. Reducing one problem to another... • The state-entry problem: Given a TM and string w, tell whether or not a given state q is ever entered when processing w. • Is it decidable? Why or why not?

28. Other Undecidable Problems • Given a TM M, letter a, and string w, determine if the letter a ever gets written when processing M(w). • Determine if two TMs are equal • Determine if a CFG is ambiguous • Given 2 CFGs, G1 and G2, determine if L(G1)  L(G2) = {}

29. Rice’s Theorem • Any non-trivial property of a recursively enumerable language is undecidable. • Particularly, the following are undecidable. • For language G, is L(G) = {}? • For language G, is string w in L(G)? • Is TM M finite? • Does TM M accept 2 different strings of the same length?

30. The moral of the story... • Researchers first find the properties of the problem before trying to solve it. • Know beforehand the possibilities of the best answer, or even that an answer is possible. • Often leads to a definition of assumptions or concessions to make the problem solvable in a more practical way.