Module `Cf_data_ingest`

Ingesting data according to an abstract model.

Overview

This module facilitates the ingestion of data conforming to an abstract model using any interchange language provided with a basis module defining the scanner type and some basic scanners and scanner composers.

Data models are functionally composable. Optional affix convenience operators are provide an indiomatic syntax.

With a fully composed data models and the ingest specialization of an interchange language, you produce a scanner that recognizes well-typed values according to the corresponding model.

A data model represents either the "primitive" type of a scalar value or a "composed" type of a group of values.

A primitive model represents an OCaml value identified by a Cf_type.nym. Some nyms are conventionally accepted by every interchange language in the Orsetto distribution:

Cf_type.Unit A type with exactly one value, e.g. the JSON "null" value.

Cf_type.Bool A boolean value type.

Cf_type.Int An integer value in the range that an OCaml nt can represent.

Cf_type.Int32 An integer value in the range that an OCaml nt32 can represent.

Cf_type.Int64 An integer value in the range that an OCaml nt64 can represent.

Cf_type.Float An IEEE 754 floating point value in the range that an OCaml float can represent.

Cf_type.String An octet string, not necessarily associated with a particular encoding.

Cf_type.Opaque An opaque value conventionally representing any scalar or group value in the underlying interchange language. (Note well: opaque values may contain interchange language specific elements that cannot expressed in other interchange languages.)

A group model takes one of the following forms:

Vector A sequence of zero, one or more elements all of with the same model.

Table A sequence of zero, one or more pairs, comprising a domain model and a codomain model, where the domain model corresponds to a totally ordered type and no two pairs in the sequence share the same domain value.

Record A sequence of zero, one or more elements, each of which may have a different model indicated by its position in the sequence.

Structure A sequence of zero, one or more pairs, comprising an index model and a field model, where the index model corresponds to a totally ordered type, the field model is indicated by the value of the index, and no two pairs in the sequence share the same index value.

Variant Exactly one pair, comprising an index model and a field mode, where the field model is indicated by the value of the index.

Generalized Data Models

The types and values in this section facilitate the composition of abstract data models for use in constructing scanners with interchange languages for which an ingest specialization is defined.

type 'v model

The type constructor 'v model represents the ingestion of 'v values decoded from some interchange language.

val primitive : 'v Cf_type.nym -> 'v model

Use primitive nym to compose a model that corresponds to all the values of the type associated with nym.

val cast : ('a -> 'b) -> 'a model -> 'b model

Use cast f m to compose a model that corresponds to the values produced by applying f to the values recognized by scanners for the model m. If applying f raises Not_found, then the scanner recognizes no value. If f raises Failure, then the exception is converted into a Bad_syntax exception from the scanner.

val defer : 'v model Lazy.t -> 'v model

Use defer m to compose a model by forcing m as needed. This is useful for composing recursive data models to be used in the array arguments of the choice and variant model constructors (see below).

val choice : 'v model array -> 'v model

Use choice a to compose a model that corresponds to the values produced by the first scanner in the array of scanners corresponding to the models in a.

val vector : ?a:int -> ?b:int -> 'v model -> 'v array model

Use vector m to compose a model that corresponds to a variable length array of values recognized by the scanner for the model m. Use the optional ~a and ~b arguments to constrain the length of the vector to a minimum of a and/or a maximum of b.

class 'key index : ?cmp:('key ->
  'key ->
  int) -> ?model:'key model -> 'key Cf_type.nym -> object ... end

Use new index nym to create an index object for the type indicated by nym required for composing table, structure and variant group models. Use the optional ~cmp argument to specify a concrete comparator function to use instead of Stdlib.( = ). Use the optional ~model argument to specify the model explicitly, otherwise the primitive model for nym is used for the domain or index in the compoed group model.

val table : ?a:int -> ?b:int -> 'k index -> 'v model -> ('k * 'v) array model

Use table k v to compose a model that corresponds to a variable length array of pairs comprising a domain value of the type described by the index k and a codomain value of the type described by the model v. Use the optional ~a and ~b arguments to constrain the length of the table to a minimum of a and/or a maximum of b.

type element

The abstract type parameter for group elements where the position in the sequence indicates the model for the element.

type ('g, 'f, 'v) group

The type constructor ['g, 'f, 'v] group represents the composition of a group with fields of disparate model in each element on the sequence. The 'g parameter is either element for record groups or 'k index for the other cases. The 'f parameter indicates the type of the composer function that produces the final value from its component parts. The 'v parameter indicates the type of the value produced.

type ('g, 'p) bind

The type constructor ['g, 'p] bind represents the model of a specific element in a group with fields or elements of disparate type. In the case where type 'g is element, the type 'p corresponds to the type indicated by the position in the sequence. In the case where type 'g is 'k index, the type 'p corresponds to the type indicated by its index value.

val nil : ('g, 'v, 'v) group

The empty group that comprises the composition of a value.

val bind : ('g, 'p) bind -> ('g, 'f, 'r) group -> ('g, 'p -> 'f, 'r) group

Use bind b g to prepend the group element b to the group g.

module Element : sig ... end

A submodule containing composers for elements indexed by position.

val record : (element, 'f, 'v) group -> 'f -> 'v model

Use record g f to compose a data model for a record consisting of the elements described by g and the composer f, which is applied to the values scanned in the sequence to produce the result.

module Field : sig ... end

A submodule containing composers for elements indexed by some totally ordered domain type.

val structure : 'k index -> ('k index, 'f, 'v) group -> 'f -> 'v model

Use structure i g f to compose a data model for a structure consisting of the fields indexed by i as described by g, with the composer f, which is applied to the values scanned in the sequence to produce the result.

val variant : 'k index -> ('k * 'v model) array -> 'v model

Use variant i s to compose a data model for a variant indexed by i with cases described by s. Raises Invalid_argument if more than one pair k, m in s has the same value of k. The model m describes how the field value is scanned in the case where the index value is k.

module Affix : sig ... end

A submodule containing affix operators the provide a convenient and idiosyncratic syntax for composing record and structure models.

Specialization

Usage of generalized data models with a specific interchange language entails composing an ingest specialization for it. The types and values in this section are provided for that purpose.

type (_, _) control = ..

The extensible control type. Facilitates composing data models with features extended by the ingest specialization for unique features of the interchange language. A value of type ('a, 'b) control represents a conversion of a model of type 'a model to a model of type 'b model as a side effect of any additional constraints applied.

type control += private

| Cast : ('a -> 'b) -> ('a, 'b) control

A control value Cast f represents the operation of applying f to a scanned value to produce that value admitted by the model.

val control : 'a model -> ('a, 'b) control -> 'b model

Use control m c to compose a new model comprising the application of c to the model m.

type 'k pair = [

| `Table of 'k Cf_type.nym
| `Structure of 'k Cf_type.nym
| `Variant of 'k Cf_type.nym

]

Values of type 'k pair are used to describe the structure of pair elements in sequences that comprise group data models indexed by values of type 'k.

type 'k container = [

| `Vector
| `Record
| 'k pair

]

Values of type 'k container are used to describe the structure of sequences that comprise group data models.

type occurs = private

| Occurs of {
1. a : int option;
2. b : int option;
}

Values of type occurs describe the optional minimum and maximum number of elements contained by a group data model.

val occurs : ?a:int -> ?b:int -> unit -> occurs

Use occurs () to construct a value of type occurs with validated content. Use ~a to specify a non-negative minimum number of elements. Use ~b to specify a non-negative maximum number of elements. Raises Invalid_argument if either a or b is negative, or if a > b.

module type Basis = sig ... end

Define a module of type Basis to create an ingest specialization.

module type Profile = sig ... end

The module type of ingest specializations.

module Create
  (B : Basis) : 
  Profile
    with module Basis := B
     and module Form := B.Form
     and module Scan := B.Scan

Use Create(B) to create an ingest specialization from the basis B.

Install

dune-project
Dependency

Authors

Maintainers

Sources

doc/orsetto.cf/Cf_data_ingest/index.html

Module `Cf_data_ingest`

Overview

Generalized Data Models

Specialization

package orsetto

Install

dune-project Dependency

Authors

Maintainers

Sources

doc/orsetto.cf/Cf_data_ingest/index.html

Module Cf_data_ingest

Overview

Generalized Data Models

Specialization

dune-project
Dependency

Module `Cf_data_ingest`