Theory of computing, part 1

1. Theory of computing, part 1

2. Von Neumann Turing machine Finite state machines NP complete problems -maximum clique -travelling salesman problem -colour graph

3. Course outline

4. Old computers

5. Abacus

6. Abacus

7. The difference engine (1832) Born December 26, 1791 in Teignmouth, Devonshire UK, Died 1871, London; Known to some as the Father of Computing for his contributions to the basic design of the computer through his Analytical machine..

8. ENIAC (1946) The ENIAC machine occupied a room 30x50 feet. The controls are at the left, and a small part of the output device is seen at the right.

9. ENIAC (1946) The IBM 360 was a revolutionary advance in computer system architecture, enabling a family of computers covering a wide range of price and performance.

10. Books

11. Books

12. Introduction

13. Models of computing • Finite state machines (automata) Pattern recognition Simple circuits (e.g. elevators, sliding doors) • Automata with stack memory (pushdown autom.) Parsing computer languages • Automata with limited tape memory • Automata with infinite tape memory Called `Turing machines’, Most powerful model possible Capable of solving anything that is solvable

14. Chomsky hierarchy of grammars • Regular grammars • Context free grammars • Context sensitive grammars • Unrestricted grammars

15. Computers can recognise languages Computers can be made to recognize, or accept, the strings of a language. There is a correspondence between the power of the computing model and the complexity of languages that it can recognize! • Finite automata only accept regular grammars. • Push down automata can also accept context free grammars. • Turing machines can accept all grammars.

16. Languages

17. Sets • A set is a collection of things called its elements. If x is an element of set S, we can write this: x  S • A set can be represented by naming all its elements, for example: S = {x, y, z} • There is no particular order or arrangement of the elements, and it doesn’t matter if some appear more than once. These are all the same set: {x, y, z}={y, x, z}={y, y, x, z, z}

18. Combining sets • A set with no elements is called the empty set, or a null set. It is denoted by  ={}. • If every element of set A is also an element of set B, then A is called a subsetof B : A  B • The union of two sets is the set with all elements which appear in either set: C = A  B • The intersection of two sets is the set with all the elements which appear in both sets: C = A  B

19. Alphabet and strings • A string a sequence of symbols that are placed next to each other in juxtaposition • The set of symbols which make up a string are taken from a finite set called an alphabet E.g.{a, b, c} is the alphabet for the string abbacb. • A string with no elements is called an empty string and is denoted . • If Σ is an alphabet, the infinite set of all strings made up from Σ is denoted Σ*. E.g., if Σ={a}, then Σ*={, a, aa, aaa, …}

20. Languages • A language is a set of strings. • If Σis an alphabet, then a language over Σ is a collection of strings whose components come from Σ. • So Σ* is the biggest possible language over Σ, and every other language over Σ is a subset of Σ*.

21. Examples of languages • Four simple examples of languages over an alphabet Σ are the sets , {}, Σ, and Σ*. • For example, if Σ={a} then these four simple languages over Σare , {}, {a}, and {, a, aa, aaa, …}. • Recall {} is the empty string while  is the empty set. Σ* is an infinite set.

22. Example: English • The alphabet isΣ = {a,b,c,d,e…x,y,z} • The English language is made of strings formed from Σ: e.g.fun, excitement. • We could define the English Language as the set of strings over Σ which appear in the Oxford English dictionary (but it is clearly not a unique definition).

23. Concatenation • The natural operation of concatenation of strings places two strings in juxtaposition. • For example, if then the concatenation of the two strings aab and ba is the string aabba. • Use the name cat to denote this operation. cat(aab, ba) = aabba.

24. Combining languages • Languages are sets of strings, so they can be combined by the usual set operations of union, intersection, difference, and complement. • Also we can combine two languages L and M by forming the set of all concatenations of strings in L with strings in M.

25. Products of languages • This new language is called the product of L and M and is denoted by L  M. • A formal definition can be given as follows: L  M = {cat(s, t) | s  L and t  M} • For example, if L = {ab, ac} and M = {a, bc, abc}, then the product L•M is the language L  M = {aba, abbc, ababc, aca, acbc, acabc}

26. Properties of products • The following simple properties hold for any language L: L  {} = {} L = L L= L =  • The product is not commutative. In other words, we can find two languages L and M such that L  M  M  L • The product is associative. In other words, if L, M, and N are languages, then L  (M  N) = (L  M)  N

27. Powers of languages • If L is a language, then the product L  L is denoted by L2. • The language product Ln for every n{0, 1, 2, …} is as follows: L0 = {} Ln = L  Ln-1 if n > 0

28. Example For example, if L = { a, bb} then the first few powers of L are L0 = {} L1 = L = {a, bb} L2 = L  L = {aa, abb, bba, bbbb} L3 = L  L2 = {aaa, aabb, abba, abbbb, bbaa, bbabb, bbbba, bbbbbb}

29. Closure of a language • If L is a language over Σ (i.e.L  A*) then the closure of L is the language denoted by L* and is defined as follows: L* = L0L1L2… • The positive closure of L is the language denoted by L+ and defined as follows:  L+ = L1L2L3 …

30. L* vs. L+ • It follows that L* = L+ {}. But it’s not necessarily true that L+= L* - {} • For example, if we let our alphabet be Σ = {a} and our language be L = {, a}, then L+= L* • Can you find a condition on a language L such that L+ = L* - {}?

31. Closure of an alphabet The closure of Σ coincides with our definition of Σ* as the set of all strings over Σ. In other words, we have a nice representation of Σ* as follows: A* = A0A1A2….

32. Properties of closure Let L and M be languages over the alphabet Σ. Then:  a) {}* = * = {} b)L* = L*  L* = (L*)* c)L if and only if L+ = L* d)(L*  M*)* = (L*  M*)* = (L  M)* e)L  (M  L)* = (L  M)*  L

33. Grammars

34. Grammars • A grammar is a set of rules used to define the structure of the strings in a language. • If L is a language over an alphabet Σ, then a grammar for L consists of a set of grammar rules of the following form:    • where  and  denote strings of symbols taken from Σ and from a set of grammar symbols (non-terminals) that is disjoint from Σ

35. Productions A grammar rule  is often called a production, and it can be read in any of several ways as follows: replace  by   produces   rewrites to   reduces to 

36. Where to begin …… • Every grammar has a special grammar symbol called a start symbol, and there must be at least one production with left side consisting of only the start symbol. • For example, if S is the start symbol for a grammar, then there must be at least one production of the form S

37. The 4 parts of a grammar • An alphabet N of grammar symbols called non-terminals. (Usually upper case letters.) 2. An alphabet T of symbols called terminals. (Identical to the alphabet of the resulting language.) 3. A specific non-terminal called the start symbol. (Usually S. ) 4. A finite set of productions of the form , where  and  are strings over the alphabet N  T

38. Example Let Σ = {a, b, c}. Then a grammar for the language Σ* can be described by the following four productions: S   S  aS S  bS S  cS Or in shorthand: S | aS | bS | cS S can be replaced by either , or aS, or bS, or cS.

39. String • A string made up of grammar symbols and language strings is called a sentential form. • If x and y are sentential forms and  is a production, then the replacement of  by  in xy is called a derivation, and we denote it by writing xy  xy

40. Sample derivation S | aS | bS | cS, S  aS S  aS  aaS. S  aS  aaS  aacS  aacbS.. S  aS  aaS  aacS  aacbS  aacb = aacb A short hand way of showing a derivation exists: S * aacb

41. Other shorthand The following three symbols with their associated meanings are used quite often in discussing derivations:  derives in one step, + derives in one or more steps, * derives in zero or more steps.

42. A more complex grammar S  AB A| aA B | bB We can deduce that the grammar non-terminal symbols are S, A, and B, the start symbol is S, and the language alphabet includes , a, and b.

43. Another derivation • Let's consider the string aab. • The statement S + aab means that there exists a derivation of aab that takes one or more steps. • For example, we have S  AB  aAB  aaAB  aaB  aabB  aab

44. Grammar specifies the language If G is a grammar, then the language of G is the set of language strings derived from the start symbol of G. The language of G is denoted by: L(G)

45. Grammar specifies the language If G is a grammar with start symbol S and set of language strings T, then the language of G is the following set: L(G) = {s | s  T* and S + s}

46. Finite languages If the language is finite, then a grammar can consist of all productions of the form S  w for each string w in the language. For example, the language {a, ba} can be described by the grammar S  a | ab.

47. Infinite languages • If the language is infinite, then some production or sequence of productions must be used repeatedly to construct the derivations. • Notice that there is no bound on the length of strings in an infinite language. • Therefore there is no bound on the number of derivation steps used to derive the strings. • If the grammar has n productions, then any derivation consisting of n + 1 steps must use some production twice

48. Finite languages • For example, the infinite language {anb | n 0 } can be described by the grammar, S  b | aS • To derive the string anb, use the production S  aS • repeatedly --n times to be exact-- and then stop the derivation by using the production S  b • The production S  aS allows us to say If S derives w, then it also derives aw

49. Recursion • A production is called recursive if its left side occurs on its right side. • For example, the production S  aS is recursive. • A production S  is indirectly recursive if S derives (in two or more steps) a sentential form that contains S.

50. Indirect recursion • For example, suppose we have the following grammar: S  b | aA A  c | bS • The productions S  aA and A  bS are both indirectly recursive   S  aA  abS  A  bS  baA