1 / 19

Lexical Analysis

Lexical Analysis. Uses formalism of Regular Languages Regular Expressions Deterministic Finite Automata (DFA) Non-deterministic Finite Automata (NDFA) RE  NDFA  DFA  minimal DFA (F)Lex uses RE as input, builds lexor. DFAs: Formal Definition. DFA M = (Q, S , d , q 0 , F)

decker
Download Presentation

Lexical Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lexical Analysis • Uses formalism of Regular Languages • Regular Expressions • Deterministic Finite Automata (DFA) • Non-deterministic Finite Automata (NDFA) • RE  NDFA  DFA  minimal DFA • (F)Lex uses RE as input, builds lexor

  2. DFAs: Formal Definition DFA M = (Q, S, d, q0, F) Q = states finite set S = alphabet finite set d = transition function function in Q S Q q0 = initial/starting state q0 Q F = final states F  Q

  3. a b …aa …ab b a a a e b a b a b b …ba …bb a a b b DFAs: Example strings over {a,b} with next-to-last symbol = a

  4. Nondeterministic Finite Automata “Nondeterminism” implies having a choice. Multiple possible transitions from a state on a symbol. d(q,a) is a set of states d : Q S Pow(Q) Can be empty, so no need for error/nonsense state. Acceptance: exist path to a final state? I.e., try all choices. Also allow transitions on no input: d : Q  (S {e})  Pow(Q)

  5. NFAs: Formal Definition NFA M = (Q, S, d, q0, F) Q = states a finite set S = alphabet a finite set d = transition function a total function in Q  (S {e})  Pow(Q) q0 = initial state q0 Q F = final states F  Q

  6. S a …a …aS … S Loop until we “guess” which is the next-to-last a. NFAs: Example strings over {a,b} with next-to-last symbol = a

  7. 0 0s 2 e e 2s e 1 e e 1s NFAs: Example strings over {0,1,2} having (either 0-or-more 0’s or 0-or-more 1’s) followed by 0-or-more 2’s

  8. Regular Expressions • Regular expression (over S) •  • e • a where aS • r+r’ • r r’ • r* • where r,r’ regular (over S) • Notational shorthand: • r0 = e, ri = rri-1 • r+ = rr*

  9. RE  NFA Defined inductively on structure of RE. • This construction produces NFA with single final state. • 6 cases: , e, a, r’+r’’, r’r’’, r’*

  10. RE  NFA:  q0 qf Accepts nothing since no edge to final state.

  11. RE  NFA: e q0

  12. RE  NFA: a q0 a qf

  13. q’0 q’f e e q0 qf e e q’’0 q’’f e edges guess whether to use r’ or r’’. RE  NFA: r’+r’’

  14. q0 e e e qf q’0 q’f q’’0 q’’f Could conflate q0 with q’0, q’’f with qf. RE  NFA: r’r’’

  15. e q0 qf e e q’0 q’f e Can loop r’ as many times as desired or skip it. RE  NFA: r’*

  16. e 0 e e e e e e e 0 1 e RE  NFA: Example (0+01)*

  17. RE  NFA: Notes Most constructions produce very large NFAs. • Not optimal for size. • But easy to construct.

  18. NFA -> DFA Subset Construction • Complicated but well described in the text • Section 3.7.1 (pp 152-155), Algorithm 3.20 (2nd edition) • In section 3.6 (pp 116-121) in 1st edition

  19. Minimizing DFA • Partition states of DFA, D, into two sets, final states, and non-final states. • Continue until no more partitions are needed • For each partition, P, split the DFA states of P so that, for each subpartition, all DFA states in that partition have the same transition for each input symbol, x.

More Related