Module LocalGenlexSource

A generic lexical analyzer.

This module implements a simple 'standard' lexical analyzer, presented as a function from character streams to token streams. It implements roughly the lexical conventions of OCaml, but is parameterized by the set of keywords of your language.

Example: a lexer suitable for a desk calculator is obtained by

     let lexer = make_lexer ["+";"-";"*";"/";"let";"="; "("; ")"]  

The associated parser would be a function from token stream to, for instance, int, and would have rules such as:

        let rec parse_expr = parser
          | [< n1 = parse_atom; n2 = parse_remainder n1 >] -> n2
        and parse_atom = parser
          | [< 'Int n >] -> n
          | [< 'Kwd "("; n = parse_expr; 'Kwd ")" >] -> n
        and parse_remainder n1 = parser
          | [< 'Kwd "+"; n2 = parse_expr >] -> n1+n2
          | [< >] -> n1

One should notice that the use of the parser keyword and associated notation for streams are only available through camlp4 extensions. This means that one has to preprocess its sources e. g. by using the "-pp" command-line switch of the compilers.