1 / 23

Course Overview

Course Overview. PART I: overview material 1 Introduction 2 Language processors (tombstone diagrams, bootstrapping) 3 Architecture of a compiler PART II: inside a compiler 4 Syntax analysis 5 Contextual analysis 6 Runtime organization 7 Code generation PART III: conclusion

southern
Download Presentation

Course Overview

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Course Overview PART I: overview material 1 Introduction 2 Language processors (tombstone diagrams, bootstrapping) 3 Architecture of a compiler PART II: inside a compiler 4 Syntax analysis 5 Contextual analysis 6 Runtime organization 7 Code generation PART III: conclusion • Interpretation • Review

  2. Systematic Development of Rec. Descent Parser (1) Express grammar in EBNF (2) Grammar Transformations: Left factorization and Left recursion elimination (3) Create a parser class with • private variable currentToken • methods to call the scanner: accept and acceptIt (4) Implement a public method for main function to call: • public parsemethod that • fetches the first token from the scanner • calls parseS (where S is start symbol of the grammar) • verifies that scanner next produces the end–of–file token (5) Implement private parsing methods: • add private parseN method for each non terminal N

  3. Developing RD Parser for Mini Triangle • Before we begin: • The following non-terminals are recognized by the scanner • They will be returned as tokens by the scanner Identifier := Letter (Letter|Digit)* Integer-Literal ::= Digit Digit* Operator ::= + | - | * | / | < | > | = Comment ::= ! Graphic* eol Assume scanner returns instances of this class: public class Token { byte kind; String spelling; final static byte IDENTIFIER = 0, INTLITERAL = 1; ...

  4. Left recursion elimination needed Left factorization needed (1)&(2) Developing RD Parser for Mini Triangle Program ::= single-Command Command ::= single-Command | Command ; single-Command single-Command ::= V-name :=Expression | Identifier ( Expression ) | ifExpression thensingle-Command elsesingle-Command | while Expression dosingle-Command | letDeclaration insingle-Command | beginCommandend V-name ::= Identifier ...

  5. (1)&(2) Express grammar in EBNF and transform After factorization etc. we get: Program ::= single-Command Command ::= single-Command (; single-Command)* single-Command ::= Identifier ( :=Expression | ( Expression ) ) | ifExpression thensingle-Command elsesingle-Command | while Expression dosingle-Command | letDeclaration insingle-Command | beginCommandend V-name ::= Identifier ...

  6. (1)&(2) Developing RD Parser for Mini Triangle Expression ::= primary-Expression | Expression Operator primary-Expression primary-Expression ::= Integer-Literal | V-name | Operator primary-Expression | ( Expression ) Declaration ::= single-Declaration | Declaration ;single-Declaration single-Declaration ::= constIdentifier ~ Expression | var Identifier : Type-denoter Type-denoter ::= Identifier Left recursion elimination needed Left recursion elimination needed

  7. (1)&(2) Express grammar in EBNF and transform After factorization and recursion elimination : Expression ::= primary-Expression ( Operator primary-Expression )* primary-Expression ::= Integer-Literal | Identifier | Operator primary-Expression | ( Expression ) Declaration ::= single-Declaration (; single-Declaration)* single-Declaration ::= constIdentifier ~ Expression | var Identifier :Type-denoter Type-denoter ::= Identifier

  8. (3)&(4) Create a parser class and public parse method public class Parser { private Token currentToken; private void accept (byte expectedKind) { if (currentToken.kind == expectedKind) currentToken = scanner.scan( ); else report syntax error } private void acceptIt( ) { currentToken = scanner.scan( ); } public void parse( ) { acceptIt( ); // get the first token parseProgram( ); // Program is the start symbol if (currentToken.kind != Token.EOT) report syntax error } ...

  9. (5) Implement private parsing methods Program ::= single-Command private void parseProgram( ) { parseSingleCommand( ); }

  10. (5) Implement private parsing methods single-Command ::= Identifier ( :=Expression | ( Expression ) ) | ifExpression thensingle-Command elsesingle-Command | ... other alternatives ... private void parseSingleCommand( ) { switch (currentToken.kind) { case Token.IDENTIFIER : ... case Token.IF : ... ... other cases ... default: report a syntax error } }

  11. N ::=  private void parseN( ) { parse  // as explained on next two slides } Algorithm to convert EBNF into a RD parser • The conversion of an EBNF specification into a Java or C++ implementation for a recursive descent parser is so “mechanical” that it could easily be automated (such tools exist, but we won’t use them in this course) • We can describe the algorithm by a set of mechanical rewrite rules

  12. parsetwhere t is a terminal accept(t); parseNwhere N is a non-terminal parseN( ); parsee // a dummy statement parseX Y parseX parseY Algorithm to convert EBNF into a RD parser

  13. parseX* while (currentToken.kind is in starters[X]) { parseX } parseX | Y switch (currentToken.kind) { cases instarters[X]: parseX break; cases instarters[Y]: parseY break; default: if neither X nor Y generates  thenreport syntax error } Algorithm to convert EBNF into a RD parser

  14. Example: “Generation” of parseCommand Command ::= single-Command ( ;single-Command )* private void parseCommand( ) { parse single-Command (;single-Command )* } private void parseCommand( ) { parse single-Command parse (;single-Command )* } private void parseCommand( ) { parseSingleCommand( ); parse (;single-Command )* } private void parseCommand( ) { parseSingleCommand( ); while (currentToken.kind==Token.SEMICOLON) { parse;single-Command } } private void parseCommand( ) { parseSingleCommand( ); while (currentToken.kind==Token.SEMICOLON) { parse; parse single-Command } } private void parseCommand( ) { parseSingleCommand( ); while (currentToken.kind==Token.SEMICOLON) { acceptIt( );// because SEMICOLON has just been checked parseSingleCommand( ); } }

  15. Example: Generation of parseSingleDeclaration single-Declaration ::= const Identifier ~Expression | varIdentifier : Type-denoter private void parseSingleDeclaration( ) { switch (currentToken.kind) { case Token.CONST: acceptIt( ); parseIdentifier( ); accept(Token.IS); parseExpression( ); case Token.VAR: acceptIt( ); parseIdentifier( ); accept(Token.COLON); parseTypeDenoter( ); default: report syntax error } } private void parseSingleDeclaration( ) { switch (currentToken.kind) { case Token.CONST: parse const parse Identifier parse ~ parse Expression case Token.VAR: parse var Identifier : Type-denoter default: report syntax error } } private void parseSingleDeclaration( ) { switch (currentToken.kind) { case Token.CONST: acceptIt( ); parseIdentifier( ); accept(Token.IS); parseExpression( ); case Token.VAR: parse var Identifier : Type-denoter default: report syntax error } } private void parseSingleDeclaration( ) { parse const Identifier ~ Expression | var Identifier : Type-denoter } private void parseSingleDeclaration( ) { switch (currentToken.kind) { case Token.CONST: parse const Identifier ~ Expression case Token.VAR: parse var Identifier : Type-denoter default: report syntax error } }

  16. LL 1 Grammars • The presented algorithm to convert EBNF into a parser does not work for all possible grammars. • It only works for so called “LL 1” grammars. • Basically, an LL 1 grammar is a grammar which can be parsed with a top-down parser with a lookahead (in the input stream of tokens) of one token. • What grammars are LL 1? How can we recognize that a grammar is (or is not) LL 1? => We can deduce the necessary conditions from the parser generation algorithm.

  17. LL 1 Grammars parseX* while (currentToken.kind is in starters[X]) { parseX } Condition: starters[X] must be disjoint from the set of tokens that can immediately follow X * parseX |Y switch (currentToken.kind) { cases instarters[X]: parseX break; cases instarters[Y]: parseY break; default: if neither X nor Y generates  thenreport syntax error } Conditions: starters[X] and starters[Y] must be disjoint sets, and if either X or Y generates  then must also be disjoint from the set of tokens that can immediately follow X | Y

  18. LL 1 grammars and left factorization The original Mini-Triangle grammar is not LL 1: For example: single-Command ::= V-name :=Expression | Identifier ( Expression ) | ... V-name ::= Identifier Starters[V-name :=Expression] = Starters[V-name] = Starters[Identifier] Starters[Identifier ( Expression )] = Starters[Identifier] NOT DISJOINT!

  19. wrong: overlapping cases LL 1 grammars: left factorization What happens when we generate a RD parser from a non LL 1 grammar? single-Command ::= V-name :=Expression | Identifier ( Expression ) | ... private void parseSingleCommand( ) { switch (currentToken.kind) { case Token.IDENTIFIER: parse V-name := Expression case Token.IDENTIFIER: parse Identifier ( Expression ) ...other cases... default: report syntax error } }

  20. Left factorization (and substitution of V-name) LL 1 grammars: left factorization single-Command ::= V-name :=Expression | Identifier ( Expression ) | ... single-Command ::= Identifier ( :=Expression | ( Expression ) ) | ...

  21. LL 1 Grammars: left recursion elimination Command ::= single-Command | Command ;single-Command What happens if we don’t perform left-recursion elimination? public void parseCommand( ) { switch (currentToken.kind) { case in starters[single-Command] parseSingleCommand( ); case in starters[Command] parseCommand( ); accept(Token.SEMICOLON); parseSingleCommand( ); default: report syntax error } } wrong: overlapping cases

  22. LL 1 Grammars: left recursion elimination Command ::= single-Command | Command ;single-Command Left recursion elimination Command ::= single-Command (;single-Command)*

  23. Abstract Syntax Trees • So far we have talked about how to build a recursive descent parser which recognizes a given language described by an (LL 1) EBNF grammar. • Next we will look at • how to represent AST as data structures. • how to modify the parser to construct an AST data structure. • We make heavy use of Object–Oriented Programming! (classes, inheritance, dynamic method binding)

More Related