Introduction to the Theory of Computation

Introduction to the Theory of Computation Qiuyue Wang qiuyuew@yahoo.com qiuyuew@ruc.edu.cn

Goals • What are the fundamental capabilities and limitations of computers? • What is a computer? • What can and can not be computed? • How quickly can a problem be computed?

Computational Models • What is a computer? • Formal definition of computation • Computational models • Turing machines (= real computers) • Simpler computing devices: finite state automata, push-down automata

Computability • What can and can not be computed? • Some problems have no algorithms (we will prove this) • Halting problem

Complexity • How quickly can a problem be computed? • Many problems probably have no efficient algorithms (no one knows how to prove this yet) • NP-hardness (= what can not be computed efficiently) • factoring an integer into primes • determining the shortest tour of given n cities

What is Theory Good for? • Elegant way of thinking • Expanding your minds • Useful tools in practice • finite automata, regular expressions: text processing • grammars: programming language design and specification • NP-hardness: approximate solutions or randomized computation

Text Book and References • Michael Sipser, Introduction to the Theory of Computation, 计算理论导论，机械工业出版社，2002. • J. E. Hopcroft, R. Motivani, and J. D. Ullman, Introduction to Automata Theory, Languages, and Computation, Second Edition, Addison-Wesley, 2001. • http://info.ruc.edu.cn/wangqiuyue/comp_theory.htm

Contents (1) • 0. Introduction • Mathematical notations, proofs • 1. Regular Languages • 1.1. Finite Automata • 1.2. Regular Expressions • 1.3. Properties of Regular Languages • 2. Context-Free Languages • 2.1. Context-Free Grammar • 2.2. Pushdown Automata • 2.3. Properties of Context-Free Languages

Contents (2) • 3. The Church-Turing Thesis • 3.1. Turing Machines • 3.2. Variants of Turing Machines • 3.3. The Definition of Algorithm • 4. Decidability • 4.1. Decidable Languages • 4.2. The Halting Problem • 5. Reducibility • 5.1. Undecidable Problems • 5.2. Mapping Reducibility

Contents (3) • 7. Time Complexity • 7.1. P and NP • 7.2. NP-Completeness • 7.3. NP-Complete Problems • Final Exam

Grading • Assignments: 20% • Gradiance Online Accelerated Learning • Project: 20% • Final Exam: 60%

Mathematical Notations • Sets • Sequences and Tuples • Functions and Relations • Graphs • Strings and Languages

Set • No order, no repetition among members of a set. • Subset: A  B • Union: A  B • Intersection: A  B • Complement: A

Power Set • Power set: the set of all subsets • P (A) = {S| S  A} • E.g. A = {0, 1} P (A) = { , {0}, {1}, {0, 1} } • |P (A)| = 2|A|

Sequences and Tuples • A sequence of objects is a list of these objects in some order. • A tuple is a finite sequence of objects. • k-tuple, pair • Cartesian product of sets: AB = {(a, b)| aA, bB} • A = {0, 1}, B={3, 4}, AB = {(0, 3), (0, 4), (1, 3), (1, 4)}

Functions • f: DR • f is a function, D: domain, R: range • E.g. f: {0, 1}  {2, 3} f(x) = x+2 • A function takes an input and produces an output • The same input always produces the same output • k-ary function • D = A1… Ak, i.e. k arguments • k=1: unary function, k=2: binary function

Relations • A set of k-tuples S…S is called an k-ary relation on S • E.g. binary relation “<“ on N • R is an equivalence relation if and only if • Reflexive: xRx for all x in S; • Symmetric: xRy implies yRx; • Transitive: xRy and yRz imply xRz; • E.g. m is an equivalence relation

Closures of Relations • P-closure of a relation R is the smallest relation R’ that includes all the pairs of R and possesses the properties in P. • Transitive closure of R, denoted R+, is 1) If (a, b) is in R, then (a, b) is in R+. 2) If (a, b) is in R+ and (b, c) is in R, then (a, c) is in R+. 3) Nothing is in R+ unless it so follows from (1) and (2). • Reflexive and transitive closure of R, denoted R*, is easily seen to be R+  {(a,a)| aS}

Example • R = {(1,2), (2,2), (2,3)} be a relation on the set {1, 2, 3}. Then • R+ = {(1,2), (2,2), (2,3), (1,3)} • R* = {(1,1), (1,2), (1,3), (2,2), (2,3), (3,3)}

Graphs • G = (V, E), V is a set of vertices (or nodes), and E is a set of pairs of vertices (called edges). • E.g. G=(V, E), V={1,2,3,4}, E={(1,2), (2,3), (1,3), (2,4), (1,4)} • A path in a graph is a sequence of nodes connected by edges • Directed graph • Tree

Strings • Symbol: e.g. 0, a, #, !, … • String: a finite sequence of symbols • E.g. 011001, abbc • Length of string: |011001| = 6 • Empty string: , || = 0 • Substring: 100 is a substring of 011001 • Concatenation of strings: 011001abbc

Languages • Alphabet: a finite set of symbols • E.g.  = {0, 1} • Language: a set of strings from some one alphabet • Empty set:  • Set consisting of the empty string: {} • E.g. L = {, 01, 0011, 000111, …}, English

Proofs • Proof by construction • Proof by contradiction • Proof by induction

Inductive Proofs • Prove a statement S(X) about a family of objects X (e.g., integers, trees) in two parts: • Basis: Prove for one or several small values of X directly. • Inductive step: Assume S(Y ) for Y “smaller than” X; prove S(X) using that assumption.

Example • A binary tree with n leaves has 2n-1 nodes.

If-And-Only-If Proofs • Often, a statement we need to prove is of the form “X if and only if Y .” We are then required to do two things: • Prove the if-part: Assume Y and prove X. • Prove the only-if-part: Assume X, prove Y .

Example • Equivalence of two sets • To prove two sets S=T, prove that x is in S if and only if x is in T

Introduction to the Theory of Computation