Compiler Structures

Compiler Structures 241-437, Semester 1, 2011-2012 • Objective • describe general syntax analysis, grammars, parse trees, FIRST and FOLLOW sets 4. Syntax Analysis

Overview 1. What is a Syntax Analyzer? 2. What is a Grammar? 3. Parse Trees 4. Types of CFG Parsing 5. Syntax Analysis Sets

Source Program In this lecture Lexical Analyzer Front End Syntax Analyzer Semantic Analyzer Int. Code Generator Intermediate Code Code Optimizer Back End Target Code Generator Target Lang. Prog.

Lexical Analyzer if ( a == 0 ) a = b ; Syntax Analyzer 1. What is a Syntax Analyzer? if (a == 0) a = b; IF builds a parse tree EQ ASSIGN a 0 a b

sentence (action) (object) (subject) verb phrase (indirect object) noun phrase pronoun verb proper noun article noun Syntax Analyses that we do • - Identify the function of each word • - Recognize if a sentence is grammatically correct grammar types / categories the card I gave Jim

Languages • We use a natural language to communicate • its grammar rules are very complex • the rules don’t cover important things • We use a formal language to define a programming language • its grammar rules are fairly simple • the rules cover almost everything

2. What is a Grammar? • A grammar is a notation for defining a language, and is made from 4 parts: • the terminal symbols • the syntactic categories (nonterminal symbols) • e.g. statement, expression, noun, verb • the grammar rules (productions) • e,g, A => B1 B2 ... Bn • the starting nonterminal • the top-most syntactic category for this grammar continued

We define a grammar G as a 4-tuple: G = (T, N, P, S) • T = terminal symbols • N = nonterminal symbols • P = productions/rules • S = starting nonterminal

2.1. Example 1 • Consider the grammar: T = {0, 1} N = {S, R} P = { S => 0 S => 0 R R => 1 S } S is the starting nonterminal the right hand sides of productions usually use a mix of terminals and nonterminals

Is “01010” in the language? • Start with a S rule: • Rule String Generated-- SS => 0 R 0 RR => 1 S 0 1 SS => 0 R 0 1 0 RR => 1 S 0 1 0 1 SS => 0 0 1 0 1 0 • No more rules can be applied since there are no more nonterminals left in the string. Yes, it is in the language.

Example 2 • Consider the grammar: T = {a, b, c, d, z} N = {S, R, U, V} P = { S => R U z | z R => a | b R U => d V U | c V => b | c } S is the starting nonterminal

The notation: X => Y | Z is shorthand for the two rules: X => YX => Z • Read ‘|’ as ‘or’.

Is “adbdbcz” in the language? • Rule String Generated-- SS => R U z R U zR => a a U zU => d V U a d V U zV => b a d b U zU => d V U a d b d V U zV => b a d b d b U zU => c a d b d b c z Yes! This grammar has choices about how to rewrite the string.

Example 3: Sums e.g. 5 + 6 - 2 • The grammar: T = {+, -, 0, 1, 2, 3, ..., 9} N = {L, D} P = { L => L + D | L – D | DD => 0 | 1 | 2 | ... | 9 } L is the starting nonterminal

Example 4: Brackets • The grammar: T = { '(', ')' } N = {L} P = { L => '(' L ')' LL => ε} L is the starting nonterminal ε means 'nothing'

2.2. Derivations A sequence of the form: w0 w1 …  wn is a derivationof wn from w0(or w0* wn) Example: L rule L => ( L ) L ( L ) L rule L => e ( ) L rule L => e ( ) L * ( ) This means that the sentence ( ) is a derivation of L

so L * (( )) ( )

2.3. Kinds of Grammars • There are 4 main kinds of grammar, of increasing expressive power: • regular (type 3) grammars • context-free (type 2) grammars • context-sensitive (type 1) grammars • unrestricted (type 0) grammars • They vary in the kinds of productions they allow.

Regular Grammars S => wTT => xTT => a • Every production is of the form: A => a | a B | e • A, B are nonterminals, a is a terminal • These are sometimes called right linear rules because if a nonterminal appears in the rule body, then it must appear last. • Regular grammars are equivalent to REs.

Context-Free Grammars (CFGs) A => aA => aBcdB => ae • Every production is of the form: A => d • A is a nonterminal, d can be any number of nonterminals or terminals • The Syntax Analyzer uses CFGs.

2.4. REs for Syntax Analysis? • Why not use REs to describe the syntax of a programming language? • they don’t have enough power • Examples: • nested blocks, if statements, balanced braces • We need the ability to 'count', which can be implemented with CFGs but not REs.

3. Parse Trees • A parse tree is a graphical way of showing how productions are used to generate a string. • The syntax analyzer creates a parse tree to store information about the program being compiled.

Example • The grammar: T = { a, b } N = { S } P = { S => S S | a S b | a b | b a } S is the starting nonterminal

Parse Tree for “aabbba” expand the symbol in the circle S The root of the tree is the start symbol S: Expand using S => S S S S S Expand using S => a S b continued

S S S S a b Expand using S => a b S S S a S b a b Expand using S => b a continued

Stop when there are no more nonterminals in leaf positions. Read off the string by reading the leaves left to right. S S S a b a S b a b

3.1. Ambiguity Two (or more) parse trees for the same string E => E + EE => E – EE => 0 | … | 9 E E or E + E E - E 4 2 E + E E - E 2 – 3 + 4 3 4 2 3

The two derivations: EE + E E E – E E – E + E  2 – E  2 – E + E  2 – E + E  2 – 3 + E 2 – 3 + E  2 – 3 + 4  2 – 3 + 4

Fixing Ambiguity • An ambiguous grammar can sometimes be made unambiguous: E =>E + T | E – T | T T =>0 | … | 9 • We'll look at some techniques in chapter 5.

4. Types of CFG Parsing • Top-down (chapter 5) • recursive descent (predictive) parsing • LL methods • Bottom-up (chapter 6) • operator precedence parsing • LR methods • SLR, canonical LR, LALR

4.1. A Statement Block Grammar • The grammar: T = {begin, end, simplestmt, ;} N = {B, SS, S} P = { B => begin SS endSS => S ; SS | εS => simplestmt | begin SS end } B is the starting nonterminal

Parse Tree begin simplestmt ; simplestmt ; end B B => begin SS end SS => S ; SS SS => e S => simplestmt S => begin SS end SS SS SS S S e begin simplestmt ; simplestmt ; end

4.2. Top Down (LL) Parsing B B => begin SS end SS => S ; SS SS => e S => simplestmt S => begin SS end SS begin simplestmt ; simplestmt ; end continued

B B => begin SS end SS => S ; SS SS => e S => simplestmt S => begin SS end SS SS S begin simplestmt ; simplestmt ; end continued

B B => begin SS end SS => S ; SS SS => e S => simplestmt S => begin SS end SS SS SS S S begin simplestmt ; simplestmt ; end continued

1 B B => begin SS end SS => S ; SS SS => e S => simplestmt S => begin SS end 2 SS 4 SS 3 SS 6 S 5 S e begin simplestmt ; simplestmt ; end

4.3. Bottomup (LR) Parsing B => begin SS end SS => S ; SS SS => e S => simplestmt S => begin SS end S begin simplestmt ; simplestmt ; end continued

B => begin SS end SS => S ; SS SS => e S => simplestmt S => begin SS end S S begin simplestmt ; simplestmt ; end continued

B => begin SS end SS => S ; SS SS => e S => simplestmt S => begin SS end SS S S e begin simplestmt ; simplestmt ; end continued

B => begin SS end SS => S ; SS SS => e S => simplestmt S => begin SS end SS SS S S e begin simplestmt ; simplestmt ; end continued

B => begin SS end SS => S ; SS SS => e S => simplestmt S => begin SS end SS SS SS S S e begin simplestmt ; simplestmt ; end continued

6 B B => begin SS end SS => S ; SS SS => e S => simplestmt S => begin SS end 5 SS 4 SS 1 SS 3 S 2 S e begin simplestmt ; simplestmt ; end

5. Syntax Analysis Sets • Syntax analyzers for top-down (LL) and bottom-up (LR) parsing utilize two types of sets: • FIRST sets • FOLLOW sets • These sets are generated from the programming language CFG.

5.1. The FIRST Sets • FIRST( <non-terminal> ) = set of all terminals that start productions for that non-terminal • Example: S => pingS => begin S end FIRST(S) = { ping, begin }

More Mathematically • A is a non-terminal. • FIRST(A) = • { c | A =>* c w , c is a terminal } { e } if A =>* e • w is the rest of the terminals and nonterminals after 'c'

Building FIRST Sets • For each non-terminal A,FIRST(A) = FIRST_SEQ(a)  FIRST_SEQ(b)  ... for all productions A => a, A => b, ... • a, b are the bodies of the productions

FIRST_SEQ() • FIRST_SEQ(e) = { e } • FIRST_SEQ(c w) = { c }, if c is a terminal • FIRST_SEQ(A w) = FIRST(A), if eFIRST(A) = (FIRST(A) – {e})  FIRST_SEQ(w), if eFIRST(A) • w is a sequence of terminals and non-terminals, and possibly empty

Compiler Structures