LING/C SC/PSYC 438/538

1. LING/C SC/PSYC 438/538 Lecture 11 Sandiway Fong

2. Administrivia • Homework 3 graded

3. Last Time • Introduced Regular Languages • can be generated by regular expressions • or Finite State Automata (FSA) • or regular grammars --- not yet introduced • Deterministic and non-deterministic FSA • DFSA can be easily encoded in Perl: • hash table for the transition function • foreach loop over a string (character by character) • conditional to check for end state • NDFSA can be converted into DFSA • example of the set of states construction • Practice: ungraded homework exercise

4. a b > ε Ungraded Homework Exercise • do not submit, do the following exercise to check your understanding • apply the set-of-states construction technique to the two machines on the ε-transition slide (repeated below) • self-check your answer: • verify in each case that the machine produced is deterministic and accurately simulates its ε-transition counterpart a b > ε

5. Ungraded Homework Exercise Review • Converting a NDFSA into a DFSA a b b a Note: this machine with an ε-transition is non-deterministic > 1 2 3 ε {1,3} {2} {3} Note: this machine is deterministic >

6. Ungraded Homework Exercise Review • Converting a NDFSA into a DFSA a b Note: this machine with an ε-transition is non-deterministic > 1 2 3 a b ε {1,2} {2} {3} Note: this machine is deterministic > b

7. FSA Regular Expressions Regular Grammars Last Time Regular Languages • Three formalisms • All formally equivalent (no difference in expressive power) • i.e. if you can encode it using a RE, you can do it using a FSA or regular grammar, and so on … Perl regular expressions Regular Languages stuff out here talk more about formal equivalence later today…

8. Perl Regular Expressions • Perl regex can include backreferences to groupings (i.e. \1, etc.) • backreferences give Perl regexs expressive power beyond regular languages: can be proved using the Pumping Lemma for regular languages (later) • the set of prime numbers is nota regular language • Lprime= {2, 3, 5, 7, 11, 13, 17, 19, 23,.. } can have regular Perl code inside a regex

9. Backreferences and FSA s x a > a • Deep question: • why are backreferences impossible in FSA? Example: Suppose you wanted a machine that accepted /(a+b+)\1/ One idea: link two copies of the machine together b Doesn’t work! Why? y y b a • Perl implementation: • how to modify it get the backreference effect? x2 a b y2 b

10. Regular Languages and FSA • Formal (constructive) set-theoretic definition of a regular language • Correspondence between REs and Regular Languages • concatenation (juxtaposition) • union (| also [ ]) • Kleene closure (*) = (x+ = xx*) • Note: • backreferences are memory devices and thus are too powerful • e.g. L = {ww} and prime number testing (earlier slides)

11. Regular Languages and FSA • Other closure properties: • Not true higherup: e.g. context-free grammars as we’ll see later

12. Equivalence: FSA and Regexs Textbook gives one direction only • Case by case: • Empty string • Empty set • Any character from the alphabet

13. Equivalence: FSA and Regexs • Concatenation: • Link final state of FSA1 to initial state of FSA2 using an empty transition Note: empty transition can be eliminated using the set of states construction (see earlier slides in this lecture)

14. Equivalence: FSA and Regexs • Kleene closure: • repetition operator: zero or more times • use empty transitions for loopback and bypass

15. Equivalence: FSA and Regexs • Union: aka disjunction • Non-deterministically run both FSAs at the same time, accept if either one accepts

16. Regular Languages and FSA • Other closure properties: Let’s consider building the FSA machinery for each of these guys in turn…

17. Regular Languages and FSA • Other closure properties:

18. Regular Languages and FSA • Other closure properties:

19. Regular Languages and FSA • Other closure properties:

20. Regular Languages and FSA • Other closure properties:

21. Regular Expressions from FSA Textbook Exercise: find a RE for • Examples (* denotes string not in the language): • *ab*ba • bab • λ (empty string) • bb • *baba • babab

22. Regular Expressions from FSA • Draw a FSA and convert it to a RE: b b [Powerpoint Animation] > 1 2 3 4 b a b ε b* b ( )+ ab+ = b+(ab+)* | ε

23. Regular Expressions from FSA • Perl implementation: $s = "abbabab bb babababab"; while ($s =~ /\b(b+(ab+)*)\b/g) { print "<$1> match!\n"; } • Output: perltest.perl <bab> match! <bb> match! <babab> match! Note: doesn’t include the empty string case Note: /../g global flag for multiple matches