Jsont cookbook

A few conventions and recipes to describe JSON data with Jsont.

Conventions

Naming Jsont.t values

Given an OCaml type t its JSON type value should be called t_jsont. If your type follows the M.t module convention use M.jsont.

General tips

Note that constructing Jsont.t values has a cost. In particular when object descriptions are Jsont.Object.finished a few checks are performed on the definition. Hence it's better to construct them as toplevel values or at least make sure you are not repeatedly constructing them dynamically in a tight loop.

Dealing with null values

Nullable JSON values are naturally mapped to ocaml option types. The Jsont.option combinator does exactly that.

It is also possible to map JSON nulls to a default value with Jsont.null. This can then be combined with Jsont.any to compose with other JSON types.

For example the following maps JSON nulls to "" and JSON strings to string on decoding. On encoding we unconditionally map back "" to null:

let string_null_is_empty =
  let null = Jsont.null "" in
  let enc = function "" -> null | _ -> Jsont.string in
  Jsont.any ~dec_null:null ~dec_string:Jsont.string ~enc ()

See also Non-finite numbers and the tangentially related topic of Optional members.

Dealing with numbers

JSON is utterly broken to interchange numbers reliably as the standard provides no constraints on their representation. Generally interopable implementations, in particular the most widely deployed and formally specified ECMAScript implementation, use IEEE 754 binary64 values to represent finite JSON numbers and null values to represent non-finite one. This has the following consequences.

Integer numbers

For representing integers by JSON numbers one is limited to the range [-2⁵³;2⁵³] which are the only integers represented precisely in IEEE 754 binary64. If you want to serialize numbers beyond this range you need to represent them by a JSON string. These scheme can be seen in the wild:

• Integers are unconditionally represented by strings. In this case Jsont.int_as_string or Jsont.int64_as_string can be used.
• Integers are represented by numbers or strings depending on their magnitude. In this case Jsont.int or Jsont.int64 can be used.
• The integer range of interest can be fully represented in a JSON number. In this case Jsont.int8, Jsont.uint8, Jsont.int16, etc. can be used.

Non-finite numbers

JSON numbers cannot represent IEEE 754 binary64 numbers: infinities and NaNs cannot be represented. The formally defined ECMAScript's JSON.stringify function replaces these values by null.

For this reason in Jsont the domain of Jsont.Base.number maps is JSON numbers or JSON null. In the decoding direction a null is mapped to Float.nan and in the encoding direction any float not satisfying Float.is_finite is mapped to a JSON null.

If you can agree with a third party on a better encoding, the Jsont.any_float or Jsont.float_as_hex_string provide lossless representations of IEEE 754 binary64 values in JSON.

Transforming base types

The Jsont.map combinator is a general map over Jsont.t types. It should rather be used to alter the representation of existing Jsont.t values. For transforming base types it is better to use the base maps of Jsont.Base as more context is made available to the functions, notably when erroring.

Transforming strings

A few simple JSON string transformers like Jsont.enum or Jsont.binary_string are provided.

If you need to devise your own maps from your own M.{of,to}_string functions that return result or raise Faiulre _ you can adapt them with these functions. For example:

let m_jsont =
  let dec = Jsont.Base.dec_result M.result_of_string in
  let enc = Jsont.Base.enc M.to_string in
  Jsont.Base.string (Jsont.Base.map ~kind:"M.t" ~dec ~enc ())

let m_jsont' =
  let dec = Jsont.Base.dec_failure M.of_string_or_failure in
  let enc = Jsont.Base.enc M.to_string in
  Jsont.Base.string (Jsont.Base.map ~kind:"M.t" ~dec ~enc ())

If you are dealing with result decoders you can also simply use Jsont.of_of_string:

let m_jsont'' =
  Jsont.of_of_string ~kind:"M.t" M.result_of_string ~enc:M.to_string

which is a shortcut for the m_jsont written above.

Dealing with arrays

JSON arrays can be directly mapped to OCaml lists, arrays, bigarray or bespoke low-dimensional tuples. If your JSON is an array of objects keyed by some identifier you may find Jsont.array_as_string_map handy.

If none of that fits you can always devise your own Jsont.Array.map.

Dealing with objects

Objects as records

Suppose our JSON object is:

{ "name": "Jane Doe"
   "age": 56 }

We represent it with an OCaml record as follows:

module Person = struct
  type t = { name : string; age : int }
  let make name age = { name; age }
  let name p = p.name
  let age p = p.age
  let jsont =
    Jsont.Object.map ~kind:"Person" make
    |> Jsont.Object.mem "name" Jsont.string ~enc:name
    |> Jsont.Object.mem "age" Jsont.int ~enc:age
    |> Jsont.Object.finish
end

Objects as key-value maps

JSON objects can be used as maps from strings to a single type of value (example). Such maps can be easily converted to OCaml as follows:

module String_map = Map.Make (String)

let map : ?kind:string -> 'a Jsont.t -> 'a String_map.t Jsont.t =
fun ?kind t ->
  Jsont.Object.map ?kind Fun.id
  |> Jsont.Object.keep_unknown (Jsont.Object.Mems.string_map t) ~enc:Fun.id
  |> Jsont.Object.finish

Since the pattern is common this is directly exposed as Jsont.Object.as_string_map.

Optional members

By default members specified via Jsont.Object.mem are mandatory and decoding errors if the member is absent.

For those cases where the member is optional a default dec_absent value must be specified to use on decoding when absent. For encoding an enc_omit function can be specified to determine whether the member should be omitted on encoding.

In the following example we use an option type to denote the potential absence of the age member:

module Person_opt_age = struct
  type t = { name : string; age : int option }
  let make name age = { name; age }
  let name p = p.name
  let age p = p.age
  let jsont =
    Jsont.Object.map ~kind:"Person" make
    |> Jsont.Object.mem "name" Jsont.string ~enc:name
    |> Jsont.Object.mem "age" Jsont.(some int)
      ~dec_absent:None ~enc_omit:Option.is_none ~enc:age
    |> Jsont.Object.finish
end

When absence is represented by None like here the Jsont.Object.opt_mem function can be used. It's stricly equivalent to the above but more concise.

Unknown object members

In JSON objects maps, there are three different ways to handle object members that have not been declared by a Jsont.Object.mem or Jsont.Object.opt_mem.

Skipping

By default Jsont.Object.map skips unknown object members.

Erroring

To error on unknown members use Jsont.Object.error_unknown:

module Person_strict = struct
  type t = { name : string; age : int; }
  let make name age = { name; age }
  let name p = p.name
  let age p = p.age
  let jsont =
    Jsont.Object.map ~kind:"Person" make
    |> Jsont.Object.mem "name" Jsont.string ~enc:name
    |> Jsont.Object.mem "age" Jsont.int ~enc:age
    |> Jsont.Object.error_unknown
    |> Jsont.Object.finish
end

Keeping

If a JSON data schema allows foreign members or to partially model an object, unknown members can be collected into a generic Jsont.Json.t object and stored in an OCaml field by using Jsont.Object.keep_unknown and Jsont.json_mems:

module Person_keep = struct
  type t = { name : string; age : int; unknown : Jsont.json ; }
  let make name age unknown = { name; age; unknown }
  let name p = p.name
  let age p = p.age
  let unknown v = v.unknown
  let jsont =
    Jsont.Object.map ~kind:"Person" make
    |> Jsont.Object.mem "name" Jsont.string ~enc:name
    |> Jsont.Object.mem "age" Jsont.int ~enc:age
    |> Jsont.Object.keep_unknown Jsont.json_mems ~enc:unknown
    |> Jsont.Object.finish
end

The value of the unknown field can be further queried with other JSON types and Jsont.Json.decode. It is also possible to define your own data structure to keep unknown members, see Jsont.Object.Mems. See also Objects as key-value maps.

Object types or classes

Sometimes JSON objects have a distinguished case member, called "type", "class" or "version" whose value define the rest of the object.

The Jsont.Object.Case module handles this pattern. Each case is described by a Jsont.Object.map object description and the Jsont.Object.case_mem allows to chose between them according to the value of the case member.

In OCaml there are two main ways to represent these case objects. Either by an enclosing variant type with one case for each object kind:

type t = C1 of C1.t | C2 of C2.t | …

or with a record which holds common fields an a field that holds the cases:

type type' = C1 of C1.t | C2 of C2.t | …
type t = { type' : type'; … (* other common fields *) }

From Jsont's perspective there is not much difference.

We show both modellings on a hypothetic Geometry object which has a "name" member and a "type" string case member indicating whether the object is a "Circle" or a "Rect". Except for the position of the name field, not much changes in each modelling.

Using an enclosing variant type:

module Geometry_variant = struct
  module Circle = struct
    type t = { name : string; radius : float; }
    let make name radius = { name; radius }
    let name c = c.name
    let radius c = c.radius
    let jsont =
      Jsont.Object.map ~kind:"Circle" make
      |> Jsont.Object.mem "name" Jsont.string ~enc:name
      |> Jsont.Object.mem "radius" Jsont.number ~enc:radius
      |> Jsont.Object.finish
  end

  module Rect = struct
    type t = { name : string; width : float; height : float }
    let make name width height = { name; width; height }
    let name r = r.name
    let width r = r.width
    let height r = r.height
    let jsont =
      Jsont.Object.map ~kind:"Rect" make
      |> Jsont.Object.mem "name" Jsont.string ~enc:name
      |> Jsont.Object.mem "width" Jsont.number ~enc:width
      |> Jsont.Object.mem "height" Jsont.number ~enc:height
      |> Jsont.Object.finish
  end

  type t = Circle of Circle.t | Rect of Rect.t
  let circle c = Circle c
  let rect r = Rect r
  let jsont =
    let circle = Jsont.Object.Case.map "Circle" Circle.jsont ~dec:circle in
    let rect = Jsont.Object.Case.map "Rect" Rect.jsont ~dec:rect in
    let enc_case = function
    | Circle c -> Jsont.Object.Case.value circle c
    | Rect r -> Jsont.Object.Case.value rect r
    in
    let cases = Jsont.Object.Case.[make circle; make rect] in
    Jsont.Object.map ~kind:"Geometry" Fun.id
    |> Jsont.Object.case_mem "type" Jsont.string ~enc:Fun.id ~enc_case cases
    |> Jsont.Object.finish
end

Using a record with a type' field:

module Geometry_record = struct
  module Circle = struct
    type t = { radius : float; }
    let make radius = { radius }
    let radius c = c.radius
    let jsont =
      Jsont.Object.map ~kind:"Circle" make
      |> Jsont.Object.mem "radius" Jsont.number ~enc:radius
      |> Jsont.Object.finish
  end

  module Rect = struct
    type t = { width : float; height : float }
    let make width height = { width; height }
    let width r = r.width
    let height r = r.height
    let jsont =
      Jsont.Object.map ~kind:"Rect" make
      |> Jsont.Object.mem "width" Jsont.number ~enc:width
      |> Jsont.Object.mem "height" Jsont.number ~enc:height
      |> Jsont.Object.finish
  end

  type type' = Circle of Circle.t | Rect of Rect.t
  let circle c = Circle c
  let rect r = Rect r

  type t = { name : string; type' : type' }
  let make name type' = { name; type' }
  let name g = g.name
  let type' g = g.type'

  let jsont =
    let circle = Jsont.Object.Case.map "Circle" Circle.jsont ~dec:circle in
    let rect = Jsont.Object.Case.map "Rect" Rect.jsont ~dec:rect in
    let enc_case = function
    | Circle c -> Jsont.Object.Case.value circle c
    | Rect r -> Jsont.Object.Case.value rect r
    in
    let cases = Jsont.Object.Case.[make circle; make rect] in
    Jsont.Object.map ~kind:"Geometry" make
    |> Jsont.Object.mem "name" Jsont.string ~enc:name
    |> Jsont.Object.case_mem "type" Jsont.string ~enc:type' ~enc_case cases
    |> Jsont.Object.finish
end

Flattening nested objects

If you are only interested in extracting data it may be useful to flatten some objects whose members are too nested for your needs.

For that just remember that nothing says that JSON objects cannot be mapped to OCaml functions. For examples to gather this kind of data for a group of person into a single record:

{
  "info" : { "id" : 1, "name": "untitled" }
  "persons" : [ … ]
}

You can use the following structure:

module Group = struct
  type t = { id : int; name : string; persons : Person.t list }
  let make id name persons = { id; name; persons }

  let info_jsont =
    Jsont.Object.map make
    |> Jsont.Object.mem "id" Jsont.int
    |> Jsont.Object.mem "name" Jsont.string
    |> Jsont.Object.finish

  let jsont =
    Jsont.Object.map (fun k persons -> k persons)
    |> Jsont.Object.mem "info" info_jsont
    |> Jsont.Object.mem "persons" (Jsont.list Person.jsont)
    |> Jsont.Object.finish
end

This however will not allow you to use jsont to encode. If you wish to do so it's likely better to follow the JSON structure and hide the annoying access structure under an abstract type behind a nice API.

Dealing with recursive JSON

To describe recursive JSON values you need to define your description in a lazy expression and use Jsont.rec' to refer to the value you are defining. This results in the following structure:

let jsont : t Jsont.t =
  let rec t = lazy ( … Jsont.rec' t … ) in
  Lazy.force t

For example a tree encoded as a JSON object with:

{ "value": …,
  "children": […] }

Is modelled by:

module Tree = struct
  type 'a t = Node of 'a * 'a t list
  let make v children = Node (v, children)
  let value (Node (v, _)) = v
  let children (Node (_, children)) = children
  let jsont value_type =
    let rec t = lazy
      (Jsont.Object.map ~kind:"Tree" make
       |> Jsont.Object.mem "value" value_type ~enc:value
       |> Jsont.Object.mem "children" (Jsont.list (Jsont.rec' t)) ~enc:children
       |> Jsont.Object.finish)
    in
    Lazy.force t
end

The topojson.ml and geojson.ml examples in the source repository provide more extensive examples of recursive definition.