CS 153: Concepts of Compiler Design October 29 Class Meeting

1. CS 153: Concepts of Compiler DesignOctober 29 Class Meeting Department of Computer ScienceSan Jose State UniversityFall 2014Instructor: Ron Mak www.cs.sjsu.edu/~mak

2. JJDoc • JJDoc produces documentation for your grammar. • Right-click in the .jj edit window. • It generates an HTML file from a .jj grammar file. • Read Chapter 5 of the JavaCC book. • Ideal for your project documentation!

3. JJDoc on the Command Line • Example bash script for a Mac: java –classpathEclipse_path/plugins/sf.eclipse.javacc_1.5.24/javacc.jarjjdoc.jj file #!/bin/bash java -classpath /Applications/Eclipse/plugins/sf.eclipse.javacc_1.5.24/javacc.jarjjdoc $1

4. Assignment #6 • Generate a working parser with JavaCC • This assignment is the start of your compiler project. • This assignment due Monday, November 10. • By the end of the semester, you must implement enough of your compiler to be able to write nontrivial programs in your chosen source language, compile them, and run them. • Your final compiler project will be dueFriday, December 12.

5. Assignment #6, cont’d • Keep the grammar of your source language simple! • Add features incrementally. • This assignment represents a snapshot of your early thinking about language design. • You can change or add new features later._

6. Assignment #6: Suggested Implementation Order • Expressions with numeric constants and scalar variables • no type checking yet • no arrays and records yet • Assignment statements • Control statements • Variable declarations • no type definitions yet • Procedure and function declarations • Procedure and function calls • Type definitions • type checking • arrays and records For this assignment, do at least the first 3. (Some control statements; you can add more later.)

7. Assignment #6: Turn in a Zip File • The EBNF production rules for your language. • At least 20 rules for this assignment (you can add more later). • Generate with JJDoc. • A .jj file based on your EBNF rules. • A sample program (or a set of statements) written in your source language. • Show off features of your parser as far as you implemented it for this assignment. • Can be error-free (i.e., no syntax error handling yet). • Output from compiling your sample program.

8. Assignment #6 • Add syntactic actions to print messages when the parser recognizes major source language constructs (such as statements and expressions) in the source program. • Doesn’t have to be fancy output._

9. JJTree • JJTreebuilds an Abstract Syntax Tree (AST); i.e., a parse tree. • It works as a preprocessor for JavaCC grammars._

10. .java JavaC JJTree JavaCC Generated Parser .jj source program AST JJTree • JJTreeaugments a JavaCC grammar by inserting tree-building code. • When a parser is generated from the augmented grammar, and • The generated parser is executed against a source program … • An AST is created. .jjt

11. Example: Simple Calculator void Expression() : {} { {System.out.println("EXPRESSION STARTS");} Operator() {System.out.println("EXPRESSION ENDS");} } void Operator() : {} { Operand() "+" {System.out.println("Operator: " + tokenImage[PLUS]);} Operand() } void Operand() : {Token t;} { t=<DIGITS> {System.out.println("Operand: " + t.image);} } calculator_simple.jj

12. Example: A Simple AST PARSER_BEGIN(Calculator) import java.io.*; public class Calculator { public static void main(String[] args) { Reader sr = new StringReader(args[0]); Calculator calc = new Calculator(sr); try { SimpleNode node = calc.Expression(); node.dump(">"); } catch (ParseException ex) { ex.printStackTrace(); } } } PARSER_END(Calculator) SimpleNode Expression() : {} { Operator() {return jjtThis;} } void Operator() : {} { Operand() "+" Operand() } void Operand() : {} { <DIGITS> } jjtThis represents the current AST node. SimpleNode implements the JJTree Node class. dump() is the AST print method. calculator_tree.jjt

13. Example: Print AST Node Images • You can modify the generated SimpleNode class. SimpleNodeExpression() : {} { Operator() {return jjtThis;} } void Operator() : {Token t;} { Operand() t="+" {jjtThis.setImage(t.image);} Operand() } void Operand() : {Token t;} { t=<DIGITS> {jjtThis.setImage(t.image);} } Modify the generated SimpleNode class to add the image field and the setImage() method. calculator_tree_image.jjt

14. Example: Operating Calculator SimpleNodeExpression() : {} { Operator() { SimpleNodeoperator = (SimpleNode) jjtThis.jjtGetChild(0); SimpleNodefirst = (SimpleNode) operator.jjtGetChild(0); intfirstValue = (Integer) first.jjtGetValue(); SimpleNodesecond = (SimpleNode) operator.jjtGetChild(1); intsecondValue = (Integer) second.jjtGetValue(); if (((String) operator.jjtGetValue()).equals("+")) { jjtThis.jjtSetValue(firstValue + secondValue); } else { System.out.println("Unknown operator"); } return jjtThis; } } First child Second child calculator_calculating.jjt

15. The Visitor Design Pattern • Purpose: • Visit the different node types of a tree structure. • Perform operations on the nodes without needing to modify the nodes. • The visitor interface declares a visit() method for each node type that wants to be visited. • visit(NodeTypeA node)visit(NodeTypeB node) • Each tree node has an accept() method to accept a visitor. • accept(Visitor v)

16. The Visitor Design Pattern • A tree node’s accept() method calls the visitor’s visit() method that’s appropriate for the node type, passing itself as an argument. • Accept a visitor  call the visitor’s visit() method • “Here’s the house key. Go visit my house now.” • In order to perform its operation, the visitor’s visit() method uses the node argument (the house key) to access the node’s public data and to call the node’s public methods.

17. Calculator and the Visitor Design Pattern • The visitor interface declares an overloaded visit() method for each node type. • Each tree node has an accept() method to accept a visitor. • A tree node’s accept() method calls the visitor’s visit() method that’s appropriate for the node type, passing itself as an argument. • The visitor’s visit() method uses the node argument to access the node’s public data and to call the node’s public methods in order to perform its operation. JJTree generates the black classes. You write the blue classes.

18. Calculator and the Visitor Design Pattern • First create a visitor object from class SumVisitor. • There is only one visitor object! • Tell the root node of the expression parse tree to accept the visitor. • Call the root node’s accept() method and pass the visitor object that will visit the root node. • The root node’s accept() method makes the visit happen by calling the visitor’s overloaded visit() method and passing itself (the node = house key) as a parameter. • The parameter’s node type determines which visit() method of the visitor is called. Reader sr = new StringReader(args[0]); Calculator calc = new Calculator(sr); SimpleNode root = calc.Expression(); SumVisitor visitor = new SumVisitor(); root.jjtAccept(visitor, null); System.out.println("Sum is " + visitor.sum);

19. Calculator and the Visitor Design Pattern public class CalculatorVisitorAdapter implements CalculatorVisitor { public Object visit(SimpleNode node, Object data) { return node.childrenAccept(this, data); } public Object visit(ASTExpression node, Object data) { return node.childrenAccept(this, data); } public Object visit(ASTOperator node, Object data) { return node.childrenAccept(this, data); } public Object visit(ASTOperand node, Object data) { return node.childrenAccept(this, data); } } JJTree generates the CalculatorVisitor interface. One visit() method for each tree node type that can accept visitors. The default action when visiting each type of tree node is to tell each of the node’s children to accept the visitor. (In other words, recursively walk down the tree.) This action can be overridden by subclasses.

20. Calculator and the Visitor Design Pattern, cont’d public class SumVisitor extends CalculatorVisitorAdapter { public int sum = 0; public Object visit(ASTOperand operand, Object data) { sum += (Integer) operand.jjtGetValue(); return super.visit(operand, data); } } • SumVisitor’svisit() method is called whenever the tree node is an ASTOperand (i.e., an ASTOperand node is being visited). • The SumVisitor object simply adds the value stored in the operand node to its running sum . • Then it performs the default action for any children. • SumVisitor is a subclass of CalculatorVisitorAdaptor. • The adaptor implements visit() methods for the other node types. Only override the visit method of the ASTOperand node. After summing, do the default action (super.visit) for any children. (This is a preorder traversal.) calculator_visitor.jjt

21. JJTree: New Node Names • By default, JJTree creates an AST where the nodes are named after the nonterminalsin the grammar. • This makes the tree design tightly-coupled to the source language and its grammar. • Use node descriptors to change the name of a node. SimpleNodeExpression() #Expr : {} { Operator() {return jjtThis;} } void Operator() #AddOp : {} { Operand() "+" Operand() } void Operand() #Opnd : {} { <DIGITS> } node_name_change.jjt

22. JJTree: Node Descriptors • You can use node descriptors to combine nodes by reusing node names. • Now both production rules Expression and SimpleExpression will generate Expr nodes. SimpleNodeExpression() #Expr : {} { Operator() {return jjtThis;} } SimpleNodeSimpleExpression() #Expr : {} { Operator() {return jjtThis;} }

23. JJTree: Node Descriptors, cont’d SimpleNodeExpression() #Expr : {} { Operator() {return jjtThis;} } void Operator() #void : {} { Operand() "+" Operand() } void Operand() #Opnd : {} { <DIGITS> } • Use the node descriptor #void to cause a production rule not to generate an AST node. • There are other things you can do with node descriptors. • Read Chapter 4 of the JavaCC book. node_void.jjt

24. JJTree: Tree Shaping • Will our simple calculator grammar accept 1+2+3+4? • Change to • Tree: void Operator() : {Token t;}{ Operand() "+" Operand()} void Operator() : {Token t;} { Operand() ( "+" Operand() )* } Expression Operator Operand Operand Operand Operand calculator_tree_shape_1.jjt

25. JJTree: Tree Shaping, cont’d • Solution: Use embedded definite node descriptors: • Tree: void Operator() #void : {Token t;} { Operand() ( "+" Operand() #add(2) )* } Expression add add add Operand Operand Operand Operand calculator_tree_shape_2.jjt

26. What JJTree, JJDoc, and JavaCC Do • You feed JJTreea .jjtgrammar file • Token specifications using regular expressions • Production rules using EBNF • JJTreeproduces a .jj grammar file • JavaCCgenerates a scanner, parser, and tree-building routines • Code for the visitor design pattern to walk the parse tree. • JJDoc produces a .html containing the ENBF

27. What JJTree, JJDoc, and JavaCC Do • However, JJTree and JavaCC will not: • Generate code for a symbol table • Generate any backend code You have to provide this code!

28. Pcl • Pclis a teeny, tiny subset of Pascal. • Use JavaCC to generate a Pcl parser and integrate with our Pascal interpreter’s • symbol table components • parse tree components • We’ll be able to parse and print the symbol table and the parse tree • in our favorite XML format • Sample program test.pcl: PROGRAM test; VAR i, j, k : integer; x, y, z : real; BEGIN i := 1; j := i + 3; x := i + j; y := 314.15926e-02 + i - j + k; z := x + i*j/k - x/y/z END.

29. Pcl Challenges • Get the JJTree parse trees to build properly with respect to operator precedence. • Use embedded definite node descriptors! • Decorate the parse tree with data type information. • Can be done as the tree is built, or as a separate pass. • You can use the visitor pattern to implement the pass. • Hook up to the symbol table and parse tree printing classes from the Pascal interpreter.

30. Pcl, cont’d options{ JJTREE_OUTPUT_DIRECTORY="src/wci/frontend"; NODE_EXTENDS="wci.intermediate.icodeimpl.ICodeNodeImpl"; ... } PARSER_BEGIN(PclParser) ... public class PclParser { // Create and initialize the symbol table stack. symTabStack = SymTabFactory.createSymTabStack(); Predefined.initialize(symTabStack); ... // Parse a Pcl program. Reader reader = new FileReader(sourceFilePath); PclParser parser = new PclParser(reader); SimpleNoderootNode = parser.program(); ...

31. Pcl, cont’d ... // Print the cross-reference table. CrossReferencercrossReferencer = new CrossReferencer(); crossReferencer.print(symTabStack); // Visit the parse tree nodes to decorate them with type information. TypeSetterVisitortypeVisitor = new TypeSetterVisitor(); rootNode.jjtAccept(typeVisitor, null); // Create and initialize the ICode wrapper for the parse tree. ICodeiCode = ICodeFactory.createICode(); iCode.setRoot(rootNode); programId.setAttribute(ROUTINE_ICODE, iCode); // Print the parse tree. ParseTreePrintertreePrinter = new ParseTreePrinter(System.out); treePrinter.print(symTabStack); } PARSER_END(PclParser) Demo