The type of token produced by the lexer.

The exception raised by the lexer when it cannot produce a token.

Asspcoate a description to each token, to help with error messages.

The function producing token from a lexbuf.