Language Processors

1. Language Processors

2. Language processor doesn’t mean “Compiler” • There are many other ways to use them

3. Different kinds of Language Processors • Lexical • Syntactical • Contextual

4. Lexical Processors • They are interested only in word • There’s no interest in phrase structure

5. What is good for a Lexer void class ; “hello” pippo pluto static int double try while ‘c’ abstract (()( [] - = long goto for do

6. With a lexer you can build.... A Text Component with Syntax Highlight

7. Syntax Processors • They are interested in phrase structure • They doesn’t care about consistency

8. What is good for a Syntactical Processor public class a { private int a; public void a() { String a; a.method(); } }

9. Syntax processor in Ginipad pretty printer public class a { private int i; public void a() { String s; s = “Hello World”; String s1 = s.substring(7); } } public class a { private int i; public void a() {String s; s = “Hello World”; String s1 = s. substring(7);}}

10. Contextual Analizer • It validate type and scope rules

11. Contextual processors issues • Is it correct to use the identifier “a” both for class and method? • what is the type of identifier “s” inside the method? • Does the related type define a mehod call like “substring(int)”? public class a { private int s; public void a() { String s; s = “Hello World”; String s1 = s.substring(7); } }

12. Where Ginipad does contextual analysis

13. A Case Study on Syntax Language Processors Building a Visual Tree Representation From the Source Code

14. The problem

15. Tree Component update Source Code Parser - AST Generator DST Traversal Abstract Syntax Tree Domain Specific Tree Representation AST Traversal Strategy

16. Grammar Design • We can start with a generic java grammar • We modify it to skip method bodies • Finally we use a parser generator tool to build the actual parser - AST generator

17. A simple Java Grammarfor Javacc: Class Node void Class() : {} { ( modifier() )* "class" <IDENTIFIER> ClassBody() } void modifier() : {} { "abstract" | "final” | "public” | "protected” | "private” | "static" }

18. Class Body void ClassBody() : {} { "{" ( Method() | Field() )* "}" }

19. Field void Field() : {} { ( modifier() )* Type() <IDENTIFIER> ";" } void PrimitiveType() : {} { "boolean” | "char” | "byte” | "short” | "int” | "long” | "float” | "double" } void Name() : {} { <IDENTIFIER> ( "." <IDENTIFIER>)* } void Type() : {} { ( PrimitiveType() | Name() ) ( "[" "]" )* }

20. Method void Method() : {} { ( modifier() )* ResultType() <IDENTIFIER> Parameters() ( "[" "]" )* ( Block() | ";" ) } void ResultType() : {} { "void” | Type() } void Parameters() : {} { "(" [ Parameter() ( "," Parameter() )* ] ")" } void Parameter() : {} { Type() <IDENTIFIER> }

21. Block JAVACODE void skipMethodBody() { Token tok; int nesting = 1; while (true) { tok = getToken(1); if (tok.kind == LBRACE) nesting++; if (tok.kind == RBRACE) { nesting--; if (nesting == 0) break; } tok = getNextToken(); } } void Block() : {} { "{" skipMethodBody() "}" }

22. A Sample Source Code public class MyClass { private int i; public int m1() { } }

23. Token

24. Building AST fromthe Token sequence

25. Domain Specific Data Structure