90 likes | 111 Views
Compilers. Lexers break the character stream into tokens (Sometimes known as tokenizers)4 5 has three tokens:4 = NUM = ADD5 = NUMParsers give semantic meaning to tokens based on rulesadd rule: NUM ADD NUMParsers can then execute on the rules. Tokens. Tokens are regular expression used to m
E N D