1 / 12

Getting Started with ANTLR

Getting Started with ANTLR. Chapter 1. Domain Specific Languages. DSLs are high-level languages designed for specific tasks DSLs include data formats, configuration file formats, text-processing languages, … DSLs make their users effective in a specific domain. The Big Picture.

takoda
Download Presentation

Getting Started with ANTLR

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Getting Started with ANTLR Chapter 1

  2. Domain Specific Languages • DSLs are high-level languages designed for specific tasks • DSLs include data formats, configuration file formats, text-processing languages, … • DSLs make their users effective in a specific domain

  3. The Big Picture • Translators map input sentences to output sentences • Translators have to recognize many different sentences • We break recognition into two similar but distinct tasks: lexical analysis and parsing

  4. Lexical Analysis • Lexical analysis consists of reading the input stream, character by character. • Characters are combined and output as “tokens” • if (x > 312){ system.out.println(“Hi”);} • Tokens: if, (, x,WS, >,WS, 312, ),{, system.out.println, (,”Hi”, ), ;, }

  5. Lexical Analysis • Tokens carry additional information in addtion to the characters they represent • ANTLR generates a lexical analyser, a Lexer, based on an input grammar it is provided • We will be building grammars and having ANTLR generate the lexer code for us

  6. Parsing • Parsing consists of reading tokens and trying to organize them into a valid sentence in the language • The parser can generate output immediately based on the sentences it recognizes or preserve the structure in the form of an abstract syntax tree (AST) which can be further processed

  7. Translation Data Flow Tokens Output Characters Lexer Parser AST Tree Walker

  8. Finally • An emitter can take the output of the parser and emit output based on all computations of the previous phases • Emitter can use templates (documents with holes) that can be filled in • ANTLR uses the StringTemplate engine to make it easier to build emitters

  9. Characters, Tokens, ASTs • Lexers consume characters from a CharStream such as ANTLRStream or ANTLRFileStream • These streams assume that the entire input fits into memory and, as a result, can buffer all characters in memory • Tokens point directly to character strings in the buffer rather than creating String objects for each token

  10. Characters, Tokens, ASTs Characters (CharStream) ... W I D T H = 2 0 0 ; \ n … tokens (Token) … ID WS = WS INT ; WS … x x x = AST (CommonTree) INT ID

  11. Characters, Tokens, ASTs • AST nodes point at token objects rather than copying token data into a tree node • CommonTree is a predefined node containing a Token payload. • The type of an ANTLR AST node is treated as Object so there are no restrictions on tree data types

  12. A-mazing Analogy • Book focuses on the discovery of the implicit tree structure in input sentences and the generation of structured text • A maze can be thought of as a language recognizer. Imagine a maze with words written on the floor • Any sentence that guides you from the entrance to the exit is “valid”

More Related