Understanding Regular Languages in Programming

1. Why do we care about regular languages? • Programs are composed of tokens: • Identifier • Number • Keyword • Special symbols • Each of these can be defined by regular grammars. • Problem: How do we handle multiple symbol operators (e.g., ++ in C, =+ in C, := in Pascal)? • ?? -multiple final states?

2. Sample token classes

3. Scanner for a language turns out to be a giant NFA for grammar.(i.e., have -rules going from start state to the start state of each token-type on previous slide). integer   identifier  keyword  symbol

4. Grammars • Grammar: generative description of a language • Automaton: analytical description • Example: programming languages are defined by a grammar (BNF),but recognized with an analytical description (the parser of a compiler), • Language theory establishes links between analytical and generative language descriptions.

5. Grammars • Grammars express languages • Example: the English language

7. A derivation of “the boy walks”:

8. A derivation of “a dog runs”:

9. Language of the grammar: L = { “a boy runs”, “a boy walks”, “the boy runs”, “the boy walks”, “a dog runs”, “a dog walks”, “the dog runs”, “the dog walks” }

10. Notation Variable or Non-terminal Terminal Production rule

11. Another Example • Grammar: • Derivation of sentence :

12. Grammar: • Derivation of sentence :

13. Other derivations:

14. Language of the grammar

15. More Notation • Grammar Set of variables Set of terminal symbols Start variable Set of Production rules

16. Example • Grammar :

17. More Notation • Sentential Form: • A sentence that contains • variables and terminals • Example: Sentential Forms sentence

18. We write: • Instead of:

19. In general we write: • If:

20. By default:

21. Example Grammar Derivations

22. Example Grammar Derivations

23. Another Grammar Example • Grammar : Derivations:

24. More Derivations

25. Language of a Grammar • For a grammar • with start variable : String of terminals

26. Example • For grammar : Since:

27. A Convenient Notation

28. Linear Grammars

29. Linear Grammars • Grammars with • at most one variable at the right side • of a production • Examples:

30. A Non-Linear Grammar Grammar :

31. Another Linear Grammar • Grammar :

32. Right-Linear Grammars • All productions have form: • Example: or

33. Left-Linear Grammars • All productions have form: • Example: or

34. Regular Grammars

35. Regular Grammars • A regular grammaris any • right-linear or left-linear grammar • Examples:

36. Observation • Regular grammars generate regular languages • Examples:

37. Regular Grammars GenerateRegular Languages

38. Theorem Languages Generated by Regular Grammars Regular Languages

39. Theorem - Part 1 Languages Generated by Regular Grammars Regular Languages Any regular grammar generates a regular language

40. Theorem - Part 2 Languages Generated by Regular Grammars Regular Languages Any regular language is generated by a regular grammar

41. Proof – Part 1 Languages Generated by Regular Grammars Regular Languages The language generated by any regular grammar is regular

42. The case of Right-Linear Grammars • Let be a right-linear grammar • We will prove: is regular • Proof idea: We will construct NFA • with

43. Grammar is right-linear Example:

44. Construct NFA such that • every state is a grammar variable: special final state

45. Add edges for each production: