1 / 8

Lexical analysis {week 04}

The College of Saint Rose CIS 433 – Programming Languages David Goldschmidt, Ph.D. Lexical analysis {week 04}. from Concepts of Programming Languages , 9th edition by Robert W. Sebesta, Addison-Wesley, 2010, ISBN 0-13-607347-6. Syntax analysis (i).

adanna
Download Presentation

Lexical analysis {week 04}

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The College of Saint Rose CIS 433 – Programming Languages David Goldschmidt, Ph.D. Lexical analysis{week 04} from Concepts of Programming Languages, 9th edition by Robert W. Sebesta, Addison-Wesley, 2010, ISBN 0-13-607347-6

  2. Syntax analysis (i) • Syntax analysisconsists oftwo key components: • A low-level componentcalled a lexical analyzer • Based on a regular grammar • A high-level component calleda syntax analyzer or parser • Based on a context-free grammar or BNF

  3. Syntax analysis (ii) • The syntax analyzer(parser)uses a context-free grammaror BNF • BNF is clear and concise • BNF is exact • BNF is readable by humansand machines alike • BNF is relatively easy to maintain

  4. Lexical analysis (i) • Lexical analysisidentifiesthe small-scale languageconstructs • Including names, operators,numeric literals, punctuation,line numbers (BASIC), etc. • Such small-scale language constructsare called lexemes

  5. Lexical analysis (ii) • Why separate lexical analysisfrom syntax analysis? • Simplicity: simpler approachesare suitable for lexical analysis • Efficiency: focuses optimizationefforts on lexical analysis andsyntax analysis separately • Portability: lexical analyzer notalways portable (due to file I/O), whereassyntax analyzer may remain portable

  6. Lexical analysis (iii) • The lexical analyzer is simplya pattern matcher • Identifies and isolates lexemes • Lexemes are logical substringsof the source program thatbelong together • Lexical analyzer assigns codescalled tokens to the lexemes • e.g. sumis a lexeme; and IDENTis the token

  7. Building a lexical analyzer • To build a lexical analyzer, we first must define the state transition diagram • Describes how the token is identified • Consists of a start state andat least one final state • Transitions occur based oncharacters encountered onthe input stream • What’s wrong with this diagram? minus(-) start neg digit (0-9) digit (0-9) digit (0-9) int

  8. What next? • Read and studyCh.4 (§4.1-§4.2) • Do Exercises at the end of Chapter 4 • Draw a complete state transition diagram for a valid floating-point numeric literal

More Related