module LexerUtil:sig
..end
val init : string -> Lexing.lexbuf -> Lexing.lexbuf
init filename lexbuf
initializes the lexing buffer lexbuf
so
that the positions that are subsequently read from it refer to the
file filename
. It returns lexbuf
.
val read : string -> string * Lexing.lexbuf
read filename
reads the entire contents of the file filename
and
returns a pair of this content (a string) and a lexing buffer that
has been initialized, based on this string.
val newline : Lexing.lexbuf -> unit
newline lexbuf
increments the line counter stored within lexbuf
. It
should be invoked by the lexer itself every time a newline character is
consumed. This allows maintaining a current the line number in lexbuf
.
val range : Lexing.position * Lexing.position -> string
range (startpos, endpos)
prints a textual description of the range
delimited by the start and end positions startpos
and endpos
.
This description is one line long and ends in a newline character.
This description mentions the file name, the line number, and a range
of characters on this line. The line number is correct only if newline
has been correctly used, as described dabove.
val tabulate : ('a -> bool) -> (unit -> 'a) -> unit -> 'a
tabulate is_eof lexer
tabulates the lexer lexer
: that is, it
immediately runs this lexer all the way until an EOF token is found, stores
the tokens in an array in memory, and returns a new lexer which (when
invoked) reads tokens from this array. The function lexer
is not allowed
to raise an exception, and must produce a finite stream of tokens: that is,
after a finite number of invocations, it must return a token that is
identified by the function is_eof
as an EOF token.
Both the existing lexer lexer
and the new lexer returned by tabulate
are functions of type
is_eof lexerunit -> 'a
, where the type 'a
is
likely to be instantiated with a triple of a token and two positions, as
per the revised lexer API described in the module CamlinternalMenhirLib.Convert
.