Context Free Grammars

1. Context Free Grammars Xiaoyin Wang CS 5363 Spring 2019

2. Context Free Grammars Xiaoyin Wang CS 5363 Spring 2016

3. Today’s Class Derivation Trees Ambiguity Normal Forms CYK Algorithm

4. Derivation Trees Illustrate the derivation of a certain sentence from a grammar Different derivation orders may generate same derivation tree

5. Derivation Order Consider the following example grammar with 5 productions:

6. Leftmost derivation order of string : At each step, we substitute the leftmost variable

7. Rightmost derivation order of string : At each step, we substitute the rightmost variable

8. Leftmost derivation of : Rightmost derivation of :

9. Derivation Trees Consider the same example grammar: And a derivation of :

10. yield

11. yield

12. yield

13. yield

14. Derivation Tree (parse tree) yield

15. Sometimes, derivation order doesn’t matter Leftmost derivation: Rightmost derivation: Give same derivation tree

16. Ambiguity A grammar can have multiple derivation tree to derive a certain sentence Classification: inherent ambiguity and non-inherent ambiguity

17. Grammar for mathematical expressions Example strings: Denotes any number

18. A leftmost derivation for

19. Another leftmost derivation for

20. Two derivation trees for

21. take

22. Good Tree Bad Tree Compute expression result using the tree

23. Two different derivation trees may cause problems in applications which use the derivation trees: • Evaluating expressions • In general, in compilers for programming languages

24. Ambiguous Grammar: A context-free grammar is ambiguous if there is a string which has: two different derivation trees or two leftmost derivations (Two different derivation trees give two different leftmost derivations and vice-versa)

25. Example: this grammar is ambiguous since string has two derivation trees

26. this grammar is ambiguous also because string has two leftmost derivations

27. Another ambiguous grammar: IF_STMT if EXPR then STMT if EXPR then STMT else STMT Variables Terminals Very common piece of grammar in programming languages

28. If expr1 then if expr2 then stmt1 else stmt2 IF_STMT if expr1 then STMT if expr2 then stmt1 else stmt2 Two derivation trees IF_STMT if expr1 then STMT else stmt2 if expr2 then stmt1

29. In general, ambiguity is bad and we want to remove it Sometimes it is possible to find a non-ambiguous grammar for a language But, in general we cannot do so

30. A successful example: Equivalent Ambiguous Grammar Non-Ambiguous Grammar generates the same language

31. Unique derivation tree for

32. An un-successful example: is inherently ambiguous: every grammar that generates this language is ambiguous

33. Ambiguity: Summary A grammar can have multiple parser tree to derive a certain sentence Inherent ambiguous language All grammars are ambiguous Non-inherent ambiguous language There exist at least one grammar that is not ambiguous Checking ambiguity of a grammar or a language: un-decidable problem

34. Today’s Class Derivation Trees Ambiguity Normal Forms CYK Algorithm

35. Normal Forms Chomsky Normal Forms BNF Normal Forms

36. A context free grammar is said to be in Chomsky Normal Form if all productions are in the following form: A → BC A → α • A, B and C are non terminal symbols • α is a terminal symbol

37. There are three preliminary simplifications Eliminate Useless Symbols Eliminate ε productions Eliminate unit productions 1 2 3

38. Eliminate Useless Symbols We need to determine if the symbol is useful by identifying if a symbol is generating and is reachable * • X is generating if X ω for some terminal string ω. • X is reachable if there is a derivation X αXβ for some α and β *

39. Example: Removing non-generating symbols S → AB | a A → b Initial CFL grammar S→AB | a A→b Identify generating symbols S→a A→b Remove non-generating

40. Example: Removing non-reachable symbols S→a Eliminate non-reachable S→a A → b Identify reachable symbols

41. The order is important? Looking first for non-reachable symbols and then for non-generating symbols can still leave some useless symbols. S → AB | a A → b S → a A → b

42. Finding generating symbols If there is a production A →α, and every symbol of α is already known to be generating. Then A is generating We cannot use S → AB because B has not been established to be generating S → AB | a A → b

43. Finding reachable symbols S is surely reachable. All symbols in the body of a production with S in the head are reachable. S → AB | a A → b In this example the symbols {S, A, B, a, b} are reachable.

44. There are three preliminary simplifications Eliminate Useless Symbols Eliminate ε productions Eliminate unit productions 1 2 3

45. Eliminate ε Productions • In a grammar ε productions are convenient but not essential • If L has a CFG, then L – {ε} has a CFG A ε * Nullable variable

46. If A is a nullable variable • Whenever A appears on the body of a production A might or might not derive ε S → ASA | aB A → B | S B → b | ε Nullable: {A, B}

47. Eliminate ε Productions • Create two version of the production, one with the nullable variable and one without it • Eliminate productions with ε bodies S → ASA| aB A → B | S B → b | ε S →ASA| aB | AS | SA | S | a A → B | S | ε B → b | ε

48. Eliminate ε Productions • Create two version of the production, one with the nullable variable and one without it • Eliminate productions with ε bodies S → ASA |aB A → B | S B → b | ε S → ASA | aB | AS | SA | S | a A → B | S | ε B → b | ε

49. Eliminate ε Productions • Create two version of the production, one with the nullable variable and one without it • Eliminate productions with ε bodies S → ASA | aB A → B | S B → b | ε S → ASA | aB | AS | SA | S | a A → B | S | ε B → b | ε

50. There are three preliminary simplifications Eliminate Useless Symbols Eliminate ε productions Eliminate unit productions 1 2 3