Complexity and Computability Theory I

Complexity and Computability Theory I Rina Zviel-Girshin Leah Epstein Winter 2002-2003

Grading policy • Exam 70% • Assignments 30% • Assignments – • Every week • Printed work • N-1 best Rina Zviel-Girshin @ARC

Computation Not formally - Any sequence of operation that solves a problem Problem – • mathematical • logical • problems that can be solved with computer Rina Zviel-Girshin @ARC

Topics Three major areas of complexity theory are: • Computability – what can be computed • Complexity – how much the computation costs • Automata – how the computation looks (abstraction) General question is - What can or can not be solved by computer? Rina Zviel-Girshin @ARC

Complexity ? • Is it possible to write a program that solves a given problem • What are the general properties of the program that solves a given program • What are the properties that program has to hold to solve a given problem • Division of the problems into different groups holding the same computational properties Rina Zviel-Girshin @ARC

Formal computational models • The basic computational model – automaton • Additional models: • Pushdown automaton • Turing machine • The basic topic of computability theory – formal language Rina Zviel-Girshin @ARC

Formal languages • Problems: • Does a given word belongs to the language? • Is the language is infinite? • Does the sequence of operations (steps) creates (derives) a given word? • Given a word and a sequence of basic steps - does the word belongs to the language? • Is the word grammatically correct? • Given a grammar (a set of rules) what language it creates? Rina Zviel-Girshin @ARC

Mathematic foundations • Set theory • Functions and relations • Graphs • Formal languages • Boolean logic • Formal systems: • Axioms (definitions) • Theorems • Proves Rina Zviel-Girshin @ARC

Set theory Set theory is important for: • Mathematics • Logic • All formal systems Basic terms that are not defined: • Set • Element of the set or set member Rina Zviel-Girshin @ARC

Signs • Set – { } • Set name – big letters of English alphabet – A, B, C or a big letter with index X1, X2, X3 Rina Zviel-Girshin @ARC

Set definition • There are several ways to define set • One way is to define each set element Example: S={1,2,3} or S={ green, red, black} • Any problem? • This way does not allow to define an infinite set Rina Zviel-Girshin @ARC

Set definition 2 • Another way to define set is to give some property to its elements. • Set will include only the elements that hold the property Example: S= { x | x is a natural number} Pronunciation: S is a set of all natural numbers Rina Zviel-Girshin @ARC

Empty set • A set with no elements called an empty set. • The sign –  Example: S - a set of all odd numbers that can be divided by two without any reminder. S is an empty set S = {xN | xODD and x mod 2=0}=  Rina Zviel-Girshin @ARC

Belonging to the set • The sign -  s  S , means that an element s belongs to the set S • Another sign -  b  S, means that an element b does not belong to the set S Example: Let N be a set of natural numbers. 1 N 1.2 N Rina Zviel-Girshin @ARC

Subset • Given two sets A and B we say that A is a subset of B, if each element of A is also an element of B. • The sign -  AB Example: A set of natural numbers N is a subset of a set of all real numbers R. N  R Rina Zviel-Girshin @ARC

Formal definition • In sign language the formal definition looks: A  B  x  A  x  B • A set A is subset of a set B if”f belonging of the element to A implies belonging of the element to B. Rina Zviel-Girshin @ARC

Proper subset • Given two sets A and B we say that A is a proper subset of B if each element of A is also an element of B and exist at least one element in B that does not belong to A. • The sign -  A  B Example: A set of natural numbers N is a proper subset of a set of all real numbers R. N  R Rina Zviel-Girshin @ARC

Some facts • Each set is a subset of itself: AA • Each set has an empty set  as it’s subset:  A Rina Zviel-Girshin @ARC

Order • Order of the elements in set does not matter. • Set is unordered group of elements. • Two groups of elements, having the same elements but in different order are the same set. Example: A = { 1,2,3,4} B = { 2,1,4,3} A = B Rina Zviel-Girshin @ARC

Repetitions • Element can not occur twice in the same set. • Element occur only once. • A structure in which occurrence of the elements in the set matters is called: • multiset or • bag Rina Zviel-Girshin @ARC

Set size • Set size is a number of elements in the set. • The sign: |A| • A set can be: • finite • infinite Rina Zviel-Girshin @ARC

Operations on sets • union • intersection • complement • cartesian product • Visual model, called Venn diagram can be used. Rina Zviel-Girshin @ARC

A A A B B B Union • Binary operation • The sign: AB • Formally: AB = { x | xA or xB } Rina Zviel-Girshin @ARC

Intersection • Binary operation • The sign: AB • Formally: AB = { x | xA and xB } A B Rina Zviel-Girshin @ARC

Complement • Unary operation • The sign:~ ~A • Formally: ~A = { x | xA} Rina Zviel-Girshin @ARC

Power set • Power set is a set of all subsets of the current set. Example: S = { 1,2 } P(S) = 2S = { ,{1},{2},{1,2}} Rina Zviel-Girshin @ARC

Cartesian product • Binary operation • The sign: AB • Cartesian product (cross product) is a set of all pairs where the first element of the pair is from A and the second element of the pair is from B. Example: A = {2,3} B = {b,c} AB ={ (2,b), (2,c), (3,b), (3,c) } Rina Zviel-Girshin @ARC

Formal languages • Central term in computer science. • Variation on set definition • First we need to define the following terms: • symbol • alphabet • string, word Rina Zviel-Girshin @ARC

Letters and Alphabet • Letter can be written in any way. • Any finite set of letters (symbols) is called an alphabet. • The sign: alphabet-  letter(symbol)-  Rina Zviel-Girshin @ARC

Example Letter – binary symbol – 0 or 1 Alphabet – of binary symbols – 0 and 1 1=0 2=1  = { 0,1 } Rina Zviel-Girshin @ARC

Words or strings • A word over some alphabet is a finite sequence of letters from the alphabet. Example:  = { 0, 1} w = 101 Rina Zviel-Girshin @ARC

Length of the word • Length of the word – the number of letters in it. • The length of the word w is signed: |w| Example: w = abca |w| = 4 w = 001011 |w| = 6 Rina Zviel-Girshin @ARC

An empty word • An empty word is a word with zero length, a word without letters. • Sign of an empty word is:  • || = 0 • Any number of empty words is still an empty word.     =  Rina Zviel-Girshin @ARC

Number of occurrences • A number of occurrences of some letter  in word w is signed by: #  (W) w=aaba #a(w)=3 #b(w)=1 #c(w)=0 Rina Zviel-Girshin @ARC

Operations over words • reverse • substring • preffix • suffix • concatenation Rina Zviel-Girshin @ARC

Reverse string • Reverse string is a string in which all letters are written in opposite order – the first letter is the last one, the second is one before last…, the last letter is the first. Example: w = 10 wR = 01 s = abcb sR = bcba Rina Zviel-Girshin @ARC

Substring • Sub-string is a sequence of letters from a string Example: w = 101 All sub-strings of w are: P(w) = { , 1, 0, 10, 01, 101 { Rina Zviel-Girshin @ARC

Prefix • Prefix is a substring of the string starting from the first letter of the string or an empty word. • Proper prefix of the word is a prefix that is not a whole word. Example: w = acdb Prefix of w:  a ac acd acbd Proper prefix of w:  a ac acd Rina Zviel-Girshin @ARC

Suffix • Suffix is a substring of the string ending with the last letter of the string or an empty word. • Proper suffix of the word is a suffix that is not a whole word. Example: w = acdb Suffix of w:  b db cdb acbd Proper suffix of w:  b db cdb Rina Zviel-Girshin @ARC

Concatenation • Binary operation • Not formally,concatenation of two words x,y together means to write them one after another in such a way that first word is a prefix and a second one is a suffix. • The sign: concatenation of two strings x and y : xy Rina Zviel-Girshin @ARC

Example x= 01 y=abc xy=01abc yx=abc01 z=23 xyz=01abc23 Rina Zviel-Girshin @ARC

Concatenation with an empty word • The result of concatenation with an empty word – is a word itself. Example: w=ab w=w= w=ab Rina Zviel-Girshin @ARC

Language • Language is a set of words. • Language can be finite and infinite. • The sign: L or L with index Li Rina Zviel-Girshin @ARC

Examples • Infinite languages: Language of all natural numbers over decimal alphabet. Language of all even length words over binary alphabet. Language of all strings over binary alphabet that ends with 0. • Finite language: L = {abc, bc} Language of all natural numbers less than 5. Rina Zviel-Girshin @ARC

Special signs • Language of all words over some alphabet  is signed by *(sigma star). • An empty language – language with zero words -. Rina Zviel-Girshin @ARC

Operations over languages • reverse • concatenation • iteration • positive closure • Kleene closure Rina Zviel-Girshin @ARC

Reverse Language • Reverse language – LR – language with all reversed words in it. • Formally: LR= { w | wR in L} Example: L = {abc, bc} LR = {cba, cb} Rina Zviel-Girshin @ARC

Concatenation • Concatenation of two languages is a language in which each word is a concatenation of two words one from the first language and the second one from the second language. Example: L1 = {ab, cd } L2 = {00, 1} L1L2 = { ab00, ab1, cd00, cd1 } Rina Zviel-Girshin @ARC

Concatenation with special languages • Concatenation with empty language – the result is an empty language. L= L =  • Concatenation with a language that includes only an empty word – the result is a language itself. L {}= {}L = L Rina Zviel-Girshin @ARC

Iteration • The sign: LK • K’s iteration – concatenation of the language with itself k times. LK = LL….LL k times L0 = {} Rina Zviel-Girshin @ARC

Complexity and Computability Theory I