package utop

  1. Overview
  2. Docs

Module UTop_tokenSource

Tokens.

The type of tokens is semi-structured: parentheses construct and quotations are nested and others tokens are flat list.

Sourcetype location = {
  1. idx1 : int;
    (*

    Start position in unicode characters.

    *)
  2. idx2 : int;
    (*

    Stop position in unicode characters.

    *)
  3. ofs1 : int;
    (*

    Start position in bytes.

    *)
  4. ofs2 : int;
    (*

    Stop position in bytes.

    *)
}

Locations in the source string, which is encoded in UTF-8.

Sourcetype t =
  1. | Symbol of string
  2. | Lident of string
  3. | Uident of string
  4. | Constant of string
  5. | Char
  6. | String of int * bool
    (*

    String (quote_size, terminated).

    *)
  7. | Comment of comment_kind * bool
    (*

    Comment (kind, terminated).

    *)
  8. | Blanks
  9. | Error
  10. | Quotation of (quotation_item * location) list * bool
    (*

    Quotation (items, terminated).

    *)
Sourceand comment_kind =
  1. | Comment_reg
    (*

    Regular comment.

    *)
  2. | Comment_doc
    (*

    Documentation comment.

    *)
Sourceand quotation_item =
  1. | Quot_data
  2. | Quot_anti of antiquotation
Sourceand antiquotation = {
  1. a_opening : location;
    (*

    Location of the opening $.

    *)
  2. a_closing : location option;
    (*

    Location of the closing $.

    *)
  3. a_name : (location * location) option;
    (*

    Location of the name and colon if any.

    *)
  4. a_contents : (t * location) list;
    (*

    Contents of the location.

    *)
}