Introduction to ANTLR

1. Introduction to ANTLR

2. Calculator Example Operators: plus, minus and multiply 4+5*2-1 or (4+5)*2-1 to enforce evaluation order All ANTLR grammars are subclasses of Lexer, Parser, or TreeParser

3. Parser Subclass class ExprParser extends Parser; Specify rules in EBNF notation expr: mexpr ((PLUS|MINUS) mexpr)* ; mexpr : atom (STAR atom)* ; atom: INT | LPAREN expr RPAREN ;

4. Lexer Subclass Similar pattern, only needs to define some operators and whitespaces class ExprLexer extends Lexer; options { k=2; // needed for newline junk charVocabulary='\u0000'..'\u007F'; // allow ascii } LPAREN: '(' ; RPAREN: ')' ; PLUS : '+' ; MINUS : '-' ; STAR : '*' ; INT : ('0'..'9')+ ; WS : ( ' ' | '\r' '\n' | '\n' | '\t' ) {$setType(Token.SKIP);} ;

5. What does ANTLR generate? Java antlr.Tool ExprParser.g? name of grammar file ExprLexer.java ExprParser.java ExprParserTokenTypes.java ExprParserTokenTypes.txt

6. Testing the lexer/parser import antlr.*; public class Main { public static void main(String[] args) throws Exception { ExprLexer lexer = new ExprLexer(System.in); ExprParser parser = new ExprParser(lexer); parser.expr(); }}

7. Expression Evaluation Add actions to parser that will evaluate tokens on the fly class ExprParser extends Parser; expr returns [int value=0] {int x;} : value=mexpr ( PLUS x=mexpr {value += x;} | MINUS x=mexpr {value -= x;} )* ; mexpr returns [int value=0] {int x;} : value=atom ( STAR x=atom {value *= x;} )* ; atom returns [int value=0] : i:INT {value=Integer.parseInt(i.getText());} | LPAREN value=expr RPAREN ;

8. Evaluation via AST Powerful strategy to build an IR For example �3+4� is represented as an Ast via + / \ 3 4

9. Some Notations #(A B C) is a tree with A at the root, and children B and C. This notation can be nested to describe trees of arbitrary structure, for example: #(A B #(C D E)) is a tree with A at the root, B as a first child, and an entire subtree as the second child. The subtree, in turn, has C at the root and D,E as children.

10. Grammar annotations for building ASTs LEAF NODES ANTLR assumes that any nonsuffixed token reference or token-range is a leaf node If no suffixes at all are specified in a grammar, then a Parser will construct a linked-list of the tokens

11. Root Nodes Any token suffixed with the "^" operator is considered a root token a : A B^ C^ ; results in tree #(C #(B A)). First A is matched and made a lonely child, followed by B which is made the parent of the current tree, A. Finally, C is matched and made the parent of the current tree, making it the parent of the B node. Note that the same rule without any operators results in the flat tree A B C.

12. AST Construction class ExprParser extends Parser; options { buildAST=true; } expr: mexpr ((PLUS^|MINUS^) mexpr)* ; mexpr : atom (STAR^ atom)* ; atom: INT | LPAREN! expr RPAREN! ;

13. AST parsing and evaluation class ExprTreeParser extends TreeParser; options { importVocab=ExprParser; } expr returns [int r=0] { int a,b; } : #(PLUS a=expr b=expr) {r = a+b;} | #(MINUS a=expr b=expr) {r = a-b;} | #(STAR a=expr b=expr) {r = a*b;} | i:INT {r = (int)Integer.parseInt(i.getText());} ;

14. Testing the AST import antlr.*; import antlr.collections.*; public class Main { public static void main(String[] args) throws Exception { ExprLexer lexer = new ExprLexer(System.in); ExprParser parser = new ExprParser(lexer); parser.expr(); AST t = parser.getAST(); System.out.println(t.toStringTree()); } }