::ICS 804:: Theory of Computation - Ibrahim Otieno - iotieno@uonbi.ac.ke +254-0722-429297

1. ::ICS 804:: Theory of Computation - Ibrahim Otieno - iotieno@uonbi.ac.ke +254-0722-429297 SCI/ICT Building Rm. G15

2. ICS 804

3. ICS 804 - Delivery • ±60 contact hours • 4.5 hours a week • Tuesday 7:00 8:30 • Thursday 5:30  8:30 • Assessment • CAT • Assignment • Exams • Theoretical course, but • Interactivity • Exercises (reflect exam questions)‏ • Laboratory sessions

4. ICS 804 - Delivery • Online access to course materials: • Notes • Slides • Assignments • http://learning.uonbi.ac.ke • Login: • Username: registration number (P58/10202/2010)‏ • Password: msc

5. ICS 804 – Course Text

6. Introduction • Computation from the point of view of mathematics (discrete mathematics)‏ • Introduction to computability and complexity theory • What kind of problems are inherently computable and which options are there? • Regular languages and finite-state automata • Very recent field started in the 1930s

7. Course Outline • Mathematical Preliminaries • Turing Machines • Recursion Theory • Markov Algorithms • Register Machines • Regular Languages and finite-state automata • Aspects of Computability

8. Course Outline • Mathematical Preliminaries • Turing Machines • Recursion Theory • Markov Algorithms • Register Machines • Regular Languages and finite-state automata • Aspects of Computability

9. Discrete Mathematics • Theory of computability = new branch of mathematics • Presupposes background in discrete mathematics • Topics: • Set-theoretic concepts • Formal language theory • Functions • Big-O notation • Propositional logic • Proof Techniques • Number-theoretic predicates

10. Sets

11. Sets • Elements of a set: may be physical objects • Set: never physical – is an abstract concept • a S – a member of set S • Members of sets may be sets S = {0, 1, {0, 1}} • Cardinality: card(S) = 3 • Singleton: S = {a} – set containing single element • Sets  lists (unordered, elements are unique)‏ • Set = family = class

12. Specifying (Naming) Sets • Finite sets {a,e,i,o,u} – elements can be listed • Infinite sets: come from mathematics • enumerate elements in an extended sense • Natural numbers N = { 0,1,2,…} • Integers Z = {…,-2,-1,0,1,2,…} • Set abstraction • {n|n is prime} or {n:n is prime} • {n|…n…} or {n:…n…}: abstraction operator: the set of all n such that n… • {n|n N & 2|n}: all even natural numbers

13. Specifying (Naming) Sets … • By convention: • Use variables n,m,k,j,…: natural numbers / integers • Use variables x,y,z,…: rational/real numbers • Set variables: uppercase letters (A,B,C, …S)‏ • Universal set: U – all elements in current domain • U by set abstraction {x|x=x} • Empty Set:  - with no element •  by set abstraction {x|xx}

14. Subsets • A = {0,1,2,3,4} A is subset of N or AN • A is a subset of B • If for any x, x  A implies x  B • Consequences: • Every set is a subset of itself i.e. A A, for any set A •  A, for any set A • “proper subset”: non-identical subset {1,2,3}  {1,2,3,4} • Power set – all subsets of a set • For a set S: P(S) = { A|A  S } • What is the power set of S = {1,2,3} ? • Card(P(S) = 2card(s) for a finite set S

15. Tuples • A tuple is an ordered list of elements • Ordered pair: <a,b> = <c,d> iff a = c and b = d • defined Ordered triple: <a,b,c> • 0-tuple: A = {<>} • 1-tuple: A = <a> • For any two sets A and B: • are said to be disjoint if AB =  • Card(AB)  Card(A) + Card(B)‏ • Card(AB) = Card(A) + Card(B) iff AB =  • Compliment of B relative to A denoted A\B is defined as {x| x  A and x  B} • We write BC and speak of compliment of B for the set U\B • Symmetric difference denoted by AB is as {x| (x  A and x  B) or (x  B and x  A) }

16. Some set properties • A ∪ U = U • A ∩  =  • C = U • UC =  • Idempotent Law: For any set A, • A ∪ A = A • A ∩ A = A • Identity Law: For any set A, • A ∪  = A • A ∩ U = A

17. Some set properties • Commutative Law: For any two sets A and B • A ∪ B = B ∪ A • A ∩ B = B ∩ A • Associative Law: For any three sets A, B and C • (A ∪ B) ∪ C = A ∪ (B ∪ C)‏ • A ∩ (B ∩ C) = (A ∩ B) ∩ C • Distributive Law: For any three sets A, B and C • A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C)‏ • A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C)‏ • De Morgan’s Law: For any two sets A and B • (A ∪ B)' = A' ∩ B' • (A ∩ B)' = A' ∪ B'

18. Special Sets • N: set of natural numbers closed under addition and multiplication not closed under subtraction nor division • Z: set of integers closed under addition, subtraction and multiplication • Q: set of rational numbers Q = {x|x = p/q for some p,q  Z with q  0} closed under addition, subtraction and multiplication Q\{0} is closed under division • R: set of real numbers closed under addition, subtraction and multiplication R\{0} closed under division • We define a partition of a set A to be a family of mutually disjoint subsets such that A = A1 ∪ A2 ….An

19. Exercises Exercise 1 • If A and B are two sets, then A ∩ (A ∪ B) equals • (A)   A      (B)      (C)         (D)       none of these Exercise 2 • The set (A ∪ B ∪ C) ∩ (A ∩ B' ∩ C')' ∩ C' is equal to •  (A)  B ∩ C'   (B)   A ∩ C (C)    B' ∩C'    (D) None of these

20. Formal Language Theory

21. Formal Language • Formal language: specific kind of set • Formal = not natural language, programming language, … but: NLP try to formalize natural language • Ignoring semantics: • possible to regard natural languages like Spanish and programming languages such as PROLOG as formal

22. Formal Language • Alphabet : finite set of symbols, e.g.  = {a,b} • Word over alphabet : finite string of elements of  allowing repetition • Length of word w = |w| = number of symbol occurrences in w • Card() = m then there exist mn distinct words over  of length n • For alphabet = {a,b} we have 2n distinct words over  of length n

23. Formal Language • Special superscript notation: anbm word consisting of n times a and m times b Example: a3b4a2 aaabbbbaa • Distinct occurrences: na(aba)=2 nb(aba)=1

24. Formal Language • Null word: word of length 0 • Zero occurrences of a and b •  = a0 • na() = 0 and nb() = 0 • Universal Language: * set of all words over  • Language over : any subset of * • Empty language:  - containing no word • Universal language: * - containing all words • Unit language L = {w} – containing word, w, only • Alphabet  = {a,b} • {w *: |w| = 3} = aaa, aab, aba, abb, baa, bab, bba, bbb}

25. Operations on words • Concatenation: ww’ of words w and w’ w is aba and w’ is bba ww’ = ababba • Reverse of a word: wR abbR = bba • Palindromes over  is the language {w*|where w=wR}

26. Palindrome Examples • Examples: • Radar – level – mum – gig • “Madam, in Eden, I’m Adam” • “A man, a plan, a canal: Panama” • “Doc, note. I dissent. A fast never prevents a fatness. I diet on cod.” (Peter Hilton)‏ • Longest 1-word English palindrome: • detartrated - (to remove tartrates)‏ • Longest 1-word palindrome in the world (Finnish)‏ • saippuakivikauppias - (lye dealer)‏ • Swahili palindromes: amesema, ataufuata, akatutaka • Longest palindrome: 31358 words

27. Operations on words • A word w’ is a prefix of a word w if w can be written w’w’’ for some word w’’ • ,a,ab,abb,abba are prefixes of the word abba (first 4 are proper)‏ • A word w’’ is a suffix of a word w if w can be written w’w’’ for some word w’ • ,a,ba,bba,abba are suffixes of the word abba (first 4 are proper)‏

28. Operations on words • A word w’’ is a substring of word w if w can be written as w’w’’w’’’ for some words w’ and w’’ abba:  a ab bb ba abb bba abba

29. Exercises • 1 = {a,b,c} • 2 = {a,b,ca} • 3 = {a,b,Ab} • Determine to which * each word belongs aba bAb cba cab caab baAb

30. Solution • 1 = {a,b,c} • 2 = {a,b,ca} • 3 = {a,b,Ab} • Determine to which * each word belongs aba 1,2,3 bAb 3 cba 1 cab 1,2 caab 1,2 baAb 3

31. Exercises • Calculate how many words there are in the following languages. List 3 elements in each of them. Which language contains  • *, where  = {a,b,c} • {w*| |w|  3}, where  = {a,b} • {w*| |w| =4}, where  = {a,b} • {anb|n is prime}

32. Solution • *, where  = {a,b,c} • Infinite number of words • {, abba, ccccba, aaccccccccc, …} • {w*| |w|  3}, where  = {a,b} • 15 words (23+22+21+20)‏ • = {, a, b, ab, ba, bb, aa, aaa, baa, aba, aab, bba, bab, abb, bbb} • {w*| |w| =4}, where  = {a,b} • 16 words (24)‏ • = {aaaa, aaab, aaba, abaa, baaa, aabb, abba, bbaa, baba, abab, baab, abbb, babb, bbab, bbba, bbbb} • {anb|n is prime} • Infinite number of words • {aab, aaab, aaaaab, …}

33. Exercises • Let  = {a,b,c,d}. List as many words as you can that are words in the language {w *| |w| = 3 and w is a word in the English language}

34. Solution • Let  = {a,b,c,d}. List as many words as you can that are words in the language {w *| |w| = 3 and w is a word in the English language} dad cab bad dab add cad baa

35. Exercises • Suppose w is ab and w’ is bab. Identify each of the following words: ww’ wR (ww’)R ww’R

36. Solution • Suppose w is ab and w’ is bab. Identify each of the following words: ww’ abbab wR ba (ww’)R babba ww’R abbab

37. Exercise • Which of the following languages are palindrome languages • wwR for any w * • wR for any palindrome w * • ww for any palindrome w * • wawR for any w * • ww’wR for any w * and any palindrome w’ * • 

38. Solution • Which of the following languages are palindrome languages • wwR for any w * yes • wR for any palindrome w * yes • ww for any palindrome w * yes • wawR for any w * yes • ww’wR for any w * and any palindrome w’ * yes •  yes

39. Mapping and Functions

40. Mapping and Functions • Mapping: association of members of one set with members of another (not necessarily different) set • First: domain • Second: codomainorrange • Members of domain serve as: arguments • Members of codomain serve as: values • Mapping f between Dom(f)=A and Cod(f)=B f: A  B • A mapping f is a function if each member of Dom(f) is mapped to one and only one member of Cod(f)‏

41. Function • Function = single valued mapping • Complete description of a mapping f consists of: • specification of Dom(f) and Cod(f)‏ • description of values of f for any member of Dom(f) taken as arguments • Examples • f: N  N f(n) = n + 1 Unary constant-7 function – returns 7 for all n • C17: N N C17(n) = 7 • f: ** f(w) = wR

42. Functions • Examples …continued • Floor function f: RZ f(x) = [x] = greatest integer ≤ x • Ceiling function f: RZ f(x) = [x] = least integer ≥ x • Multi-valued mapping • f: Z Z f(n) = n ± 3 f(n) = n  3 Not a function

43. Functions • Image of a function f defined as Image(f) is subset of Cod(f) consisting precisely of the values of f for members of Dom(f)‏ • Precisely in set language: Image(f) = {yCod(f)|y=f(x) for some x  Dom(f)}

44. Number-theoretic Functions • Number-theoretic functions: map natural numbers to natural numbers: f: NN • Unary number-theoretic functions f(n) = n + 1 • Binary number-theoretic functions f: N2 N f(n,m) = n + m • K-ary number-theoretic functions f: N k N • 0-ary number theoretic functions C02: N 0 N C02() = 2

45. Injective (1-to-1) Functions • A function f is 1-to-1 or injective ifno two distinct elements of Dom(f) are mapped to one and the same member of the Cod(f)‏ • Examples: • Successor function f: N N f(n) = n + 1 • Word reversal function f: ** f(w) = wR • Ceiling, floor and constant-7 functions not injective

46. Surjective (onto) Functions • A function f is onto or surjective provided that every element of Cod(f) is a value of f for at least one element of the Dom(f)‏ • Cod(f) = Image (f)‏ • Examples: • Word reversal function f: ** f(w) = wR • Ceiling function f: RZ f(x) = [x] = least integer ≥ x • Successor function not surjective – 0 not successor • A function that is both injective and surjective is termed as 1-1 correspondence or a bijective function

47. Functions forming operations

48. Inverse • For two functions g and f: g: N N g(n) = n/2 f: NN f(n) = 2n • g is the inverse of f • g = f-1 • The inverse operation (though not always defined) -1is; Unary function forming operation

49. Composition • For two functions g and f: g: * N where ={a,b} g(w) = na(w)‏ f: NN f(n) = 3n + 2 h: * N h(w) = f(g(w))‏ h: composition of g and f (h = g  f)‏ • The composition operation is: Binary function forming operation

50. Partial Functions • For a function g: g: N 2N g(n,m) = n – m • The function g is undefined for pairs <n,m> where n < m • Number-theoretic functions that are undefined for some n N are Partial functions.