Hand-written Lexer

A lexical analyzer can easily be written by hand. Typically, such a program will call functions getchar() and peekchar() to get characters from the input.

The lexical analyzer is likewise called as a function, with an entry such as gettoken(). The program is structured as:

1. A `` big switch'' that skips white space, peeks at the next character to guess what kind of token will be next, and calls the appropriate routine.

2. A set of routines to get particular kinds of tokens, such as identifiers, numbers, strings, or operators.

Typically, a routine will process all tokens that look alike, e.g., all kinds of numbers, or both identifiers and reserved words.