Contents
Page-10
Prev
Next
Page+10
Index
Lexical Analyzer
The Lexical Analyzer (Lexer) will convert characters into
``words'' or tokens, such as:
- Identifiers, e.g. position
- Reserved words or keywords, e.g. begin
- Numbers, e.g. 3.1415926e2
- Operators, e.g. >=
The Lexer may be called as a subroutine such as gettoken()
to get the next token from the input string. It, in turn, calls the
Line Handler routines.
The Lexer returns a token data structure,
consisting of:
- Token Type: identifier, reserved word, number, operator.
- Token Value:
- Identifiers: string and symbol table pointer
- Reserved words: integer code.
- Numbers: internal binary form.
- Operators: integer code.