Chapter 1 Introduction to the Theory of Computation

1. Chapter 1Introduction to the Theory of Computation

2. Theory of Computation • Can we mathematically model the computation? • Automata Theory • Regular languages: DFA, NFA, Regular expressions • Context-free languages: Context-free grammar, pushdown automata • Turing machines: the most powerful model of computation • Can the computer solve all problems? • Computability Theory • Halting problem • How efficient the solution is? • Complexity Theory • Time and space complexity (Big O notation)

3. 1.1 Mathematical Preliminaries Sets (1) • A set is a collection of objects • Order and repetition is irrelevant • X = {1, 2, 3} • X = {n | 1  n  3, n  N } • Y = { } =   Y is an empty set • The power set of X is the set of all subsets of X • (X) = 2X = { Y | Y  X} • |(X)| = 2|X| • Example X = {1, 2, 3} • 2X = (X)={{},{1},{2},{3},{1,2},{1,3},{2,3}, {1,2,3}} • {1,2}  X is a proper subset of X • {1,2,3}  X is NOT a proper subset of X • {1,2,3} is a superset of {1, 2}

4. Sets (2) • The symbol signifies membership (belong to): • xX indicates that x is a member or element of the set X • xX indicates that x is not a member or element of the set X • Assume X = {1, 2, 3}, then • 1  X, but 4  X • Two sets are equal if they contain the same members (elements) • If X = {1, 2, 3}, Y = {1, 2, 3}, and Z = {2, 3, 4} • Then X = Y, but X  Z, and so Y  Z

5. Sets (3) • Assume X = {1,2,3} and Y = {2,4} • Union • X  Y = {z | z X or zY} = {1,2,3,4} • Intersection • X  Y = {z | z X and zY} = {2} • Difference • X - Y = {z | z X and z Y} = {1,3}

6. Sets (4) • Complement • Let X be a subset of Y • The complement of X ( X ) with respect to Y is the set of elements in Y but not in X • X = { z | z Y and z X } = Y - X • Assume X = {1,2,3} is a subset of Y = {1,2,3,4} • X = {4} • DeMorgan’s Laws • (X  Y) = X  Y • (X  Y) = X  Y

7. Inductive Proofs • Prove a statement S(X) about a family of objects X (e.g., integers, trees) in three parts: • Basis Step: • Prove for one or several small values of X directly • Assumption step : • Assume S(Y) for Y smaller than X • Prove S(X) using that assumption

8. Inductive Proofs (EX1) • Prove by induction that • S: 1 + 2 + 3 + ……+ n = n (n + 1) / 2 • Basis step: • n = 1 • 1 ( 1 + 1) / 2 = 1 • Assumption step: • Assume that S is true for all series of length < n • Prove that S is true for a series of length n? • The series of length n is • 1+2+3+…+n-1+n • By the assumption 1+2+…+n-1 = (n-1)((n-1)+1)/2 • Add n to the summation proves S.

9. Inductive Proofs (EX2) • Example • A complete binary tree with n leaves has 2n - 1 nodes

10. Inductive Proofs (EX2) • Basis: • If T has 1 leaf, it is a one-node tree. 1 = 2*1 - 1, so OK • Assumption: • Assume that the statement is true for all trees that have leaves less than n • Proof: Proof that the statement is true for trees that have n leaves • T has two subtrees, U and V • U has u leaves and V has v leaves • According to the assumption: U has (2u – 1) of nodes and V has (2v – 1) of nodes • T must be a root plus two subtreesU and V. The number of leaves in T equal number of leaves in U + number of leaves in V = u + v • The number of nodes in T = (2u – 1) + (2v – 1) + 1 = 2 ( u + v) - 1

11. If-And-Only-If Proofs • Often, a statement we need to prove is of the form • “X if and only if Y” • We are then required to do two things: • Prove the if-part: Assume Y and prove X • Prove the only-if-part: Assume X, prove Y • Remember: • The if and only-if parts are converses of each other • One part, say “if X then Y” says nothing about whether Y is true when X is false. • An equivalent form to “if X then Y” is “if not Y then not X”; the latter is the contra-positive of the former.

12. Equivalence of Sets • Many important facts in language theory are of the form that two sets of strings, described in two different ways, are really the same set • To prove sets S and T are the same, prove: • x is in S if and only if x is in T. That is: • Assume x is in S; prove x is in T . • Assume x is in T ; prove x is in S.

13. Functions • Let f be a function defined on a set A and taking values to a set B. Set A is called the domain of f; set B is called the range of f. • The arityof the functions • The number and type of input arguments the number and type of output • Examples: • The arity of the Multiplication: Num × Num Num • The arity of the SquareRoot : Num  Num

14. 1.2: Strings and Languages (1) • Alphabet: Finite, nonempty set of symbols • Examples: •  = {0, 1}: binary alphabet •  = {a, b, c, …, z}: the set of all lower case letters • The set of all ASCII characters • String: Finite sequence of symbols from an alphabet  • Examples: • 01101 where  = {0, 1} • abracadabra where  = {a, b, c, …, z}

15. 1.2: Strings and Languages (2) • Empty String: The string with zero occurrences of symbols from  and is denoted e or  • Length of String: Number of symbols in the string • The length of a string w is usually written |w| • |1010| = 4 • |e| = 0 • |uv| = |u| + |v| • Reverse : wR • If w = abc, wR = cba

16. 1.2: Strings and Languages (3) • Concatenation: if x and y are strings, then xy is the string obtained by placing a copy of y immediately after a copy of x • x = a1a2 …ai, y = b1b2 …bj • xy = a1a2 …aib1b2 …bj • Example: x = 01101, y = 110, xy = 01101110 • xe = ex = x

17. 1.2: Strings and Languages (4) • Power of an Alphabet: k = the set of strings of length k with symbols from S • Example:  = {0, 1} • S1 = S = {0, 1} • S2 = {00, 01, 10, 11} • S0 = {e} • Question: How many strings are there in S3? • The set of all strings over  is denoted * • S* = S0 S1 S2 S3 … • Also • S+ = S1 S2 S3 … • S* = S+ {e} • S+ = S* - {e}

18. 1.2: Strings and Languages (5) • Substring: any string of consecutive characters in some string w • If w = abc • e, a, ab, abc are substrings of w • Prefix and suffix: • if w = vu • v is a prefix of w • u is a suffix of w • Example • If w = abc • a, ab , abc are prefixes of w • c, bc, abc are suffixes of w

19. 1.2: Strings and Languages (6) • Suppose: S is the string banana • Prefix :ban, banana • Suffix :ana, banana • Substring :nan, ban, ana, banana

20. 1.2: Strings and Languages (8) • Language: set of strings chosen from some alphabet • A language is a subset of * • Example of languages: • The set of valid Arabic words • The set of strings consisting of n0’s followed by n1’s • {e, 01, 0011, 000111, …} • The set of strings with equal number of 0’s and 1’s • {e, 01, 10, 0011, 0101, 1010, 1001, 1100, …} • Empty language:  = { } • The language {e} consisting of the empty string • Note:   {e}

21. 1.2: Strings and Languages (9) • Can concatenate languages • L1L2 = {xy | x L1, y L2} • Ln= L concatenated with itself n times • L0 = {e}; L1 = L • Star-closure • L* = L0L1L2 ... • L+ = L* - L0 • Languages can be finite or infinite • L = {a, aba, bba} • L = {an | n > 0}

22. 1.2: Strings and Languages (10) OPERATION DEFINITION union of L and M written L  M L  M = {s | s is in L or s is in M} concatenation of L and M written LM LM = {st | s is in L and t is in M} Kleene closure of L written L* L*=  Li , i=0,..,  L* denotes “zero or more concatenations of “ L positive closure of L written L+ L+=  Li, i=1,..,  L+ denotes “one or more concatenations of “ L L+ = LL*

23. 1.2: Strings and Languages (11) L = {A, B, …, Z, a, b, …z} D = {1, 2, …, 9} L  D = the set of letters and digits LD = all strings consisting of a letter followed by a digit L2 = the set of all two-letter strings L4 = L2 L2 = the set of all four-letter strings L* = { All possible strings of L plus  }, L+ = L* -  D+ = set of strings of one or more digits L (L  D ) = set of all strings consisting of a letter followed by a a letter or a digit L (L  D )* = set of all strings consisting of letters and digits beginning with a letter

24. 1.2: Strings and Languages (12) • The language L consists of strings over {a,b} in which each string begins with an a should have an even length • aa, ab L • aaaa,aaab,aaba,aabb,abaa,abab,abba,abbb L • baa  L • a  L

25. 1.2: Strings and Languages (13) • The language L consists of strings over {a,b} in which each occurring of b is immediately preceded by an a • e L • a  L • abaab L • bb  L • bab L • abbL

26. 1.2: Strings and Languages (14) • Let X = {a,b,c} and Y = {abb, ba}. Then • XY = {aabb, babb, cabb, aba, bba, cba} • X0 = {e} • X1 = X = {a,b,c} • X2 = XX = {aa,ab,ac,ba,bb,bc,ca,cb,cc} • X3 = XXX = {aaa,aab,aac,aba,abb,abc,aca,acb,acc,baa,bab,bac,bba,bbb,bbc,bca,bcb,bcc,caa,cab,cac,cba,cbb,cbc,cca,ccb,ccc}

27. 1.2: Strings and Languages (15) • The language L = {a,b}*{bb}{a,b}*= *{bb}* • consists of the strings over {a,b} that contain the substring bb • bb  L • abb L • bbb L • aabb L • bbaaa L • bbabba L • abab L • bab L • b L

28. 1.2: Strings and Languages (16) • Let L be the language that consists of all strings that begin with aa or end with bb • L1 = {aa}{a,b}* • L2 = {a,b}*{bb} • L = L1 L2 ={aa}{a,b}*  {a,b}*{bb} • bb  L • abb L • bbb L • aabb L • bbaaa L • bbabba L • abab L • bab L • baL

29. 1.2: Strings and Languages (16) • Let L1 = {bb} and L2 = {e, bb, bbbb} over b • The languages L1* and L2* both contain precisely the strings consisting of an even number of b’s. • e , with length zero, is an element of L1* and L2*

30. 1.2: Strings and Languages (17) • What is the language of all even-length strings over {a,b} • L = {aa, bb, ab, ba}* = (aa|bb|ab|ba)* • What is the language of all odd-length strings over {a,b} • L = {a,b}* - {aa, bb, ab, ba}* or • L = {a,b}{aa, bb, ab, ba}* or • L = {aa, bb, ab, ba}* {a,b}