For performance reasons, it is sometimes preferable to handle functions specialising in one or two arguments rather than applying all the function’s arguments (and repeating internal calculations whenever that first or second argument appears). The Staged module allows you to define a boundary between what should be ‘pre-applied’ and what should be fully applied afterwards.

The cursor of a decoder or an encoder: the index of the next byte to read or to write. Start it with ref Off.zero and read it back with (!pos :> int).

A type to describe a binary format.

char is a representation of the character type.

uint8 is a representation of unsigned 8-bit integers.

int8 is a representation of 8-bit integers.

beint16 is a representation of big-endian unsigned 16-bit integers.

leint16 is a representation of little-endian unsigned 16-bit integers.

neint16 is a representation of native-endian unsigned 16-bit integers.

beint16 is a representation of big-endian 16-bit integers.

leint16 is a representation of little-endian 16-bit integers.

neint16 is a representation of native-endian 16-bit integers.

beint32 is a representation of big-endian 32-bit integers.

leint32 is a representation of little-endian 32-bit integers.

neint32 is a representation of native-endian 32-bit integers.

beint64 is a representation of big-endian 64-bit integers.

leint64 is a representation of little-endian 64-bit integers.

neint64 is a representation of native-endian 64-bit integers.

varint is a representation of an integer value using a sequence of bytes. The number of bytes that is needed depends on the value being represented, with larger magnitudes using more bytes than smaller ones.

len is a type that represents a size (fixed or dynamic) for certain primitives such as bytes or bstr.

fixed len is a fixed length of len bytes.

delim chr is a length until we reach the chr byte.

prefix t is a representation of a value which size is specified by a header/prefix t.

rest is the rest of the payload.

bytes n is a representation of a bytes sequence of n byte(s).

bstr n is a representation of a bigstring of n byte(s).

cstring is bytes (delim '\000').

until chr is bytes (delim chr).

const v is v without a serialization mechanism.

seq len t is a consecutive sequence of len elements represented by t. It returns an array.

list len t is a consecutive sequence of len elements represented by t. It returns an list.

map t f g describes a value of type 'a via the representation t of another type 'b, by supplying the coercions between them: decoding reads a 'b with t and turns it into an 'a with f, encoding turns the 'a back into a 'b with g and writes it with t.

f and g must be inverse of each other, otherwise encoding a value then decoding it does not give that value back.

type kind = Query | Response

let kind =
  map uint8
    (function 0 -> Query | _ -> Response)
    (function Query -> 0 | Response -> 1)

The shape of the encoding does not depend on the value: map is therefore transparent for the decoder, which can still fuse it into its static path. Use it whenever it is enough — see bind otherwise.

bind t prj inj is like map, except that the representation of what remains depends on the value which was just read. It is the combinator for the formats where a header decides how the rest must be interpreted: a length, a tag, a version number.

Decoding reads an 'a with t, then decodes the rest with prj a. Encoding is the mirror image and this is where inj comes in: from the value to write, inj recovers the 'a which describes it, bind writes that 'a with t, and then writes the value itself with prj (inj v). Without inj, the encoder would have no way to know which header to emit.

inj must therefore agree with what prj accepts: for any value v that prj a can decode, inj v must be a. Otherwise the header and the payload disagree and the result cannot be decoded back.

Before reaching for bind, check that a simpler combinator does not already cover the case. A size in bytes is prefix on bytes, a number of elements is prefix on list or seq, a tag which selects a constructor is a variant sealed with sealv, and a format which refers to itself is fix. bind is for what remains: when the value which was read has to be computed with to know what follows.

Here is such a case, a payload whose length is given in 32-bit words (as the IHL of an IPv4 header or the data offset of a TCP one). prefix cannot express it, since it would take the number for a count of bytes:

let payload =
  bind uint8
    (fun words -> bytes (fixed (words * 4)))
    (fun payload -> String.length payload / 4)

Note prj is called for every value encoded or decoded, and the representation it returns is interpreted on the fly. Unlike map, bind therefore never takes part in the static (fused) path of the decoder, and size_of_value can only be dynamic on it. Prefer map when the shape of the encoding does not depend on the value.

See fix for a recursive format, where bind is what lets the recursion stop on a value read from the input.

let+ is the binding operator of map: let+ t = (x, f, g) in expr is let t = map x f g in expr.

Note the identifier it binds is the representation returned by map — not a decoded value. Nothing is read nor written when this runs; it only names an intermediate representation.

let* is the binding operator of bind: let* t = (x, f, g) in expr is let t = bind x f g in expr. As with (let+), t is the representation, not a value.

It exists to keep a large bind readable: name the two functions, then assemble them.

let payload =
  let prj words = bytes (fixed (words * 4)) in
  let inj payload = String.length payload / 4 in
  let* t = (uint8, prj, inj) in
  t

On a bind as small as this one the plain function reads just as well; see fix for the shape which motivates the operator.

The type for representing open records of type 'a with a constructor of 'b. 'c represents the remaining fields to be described using the (|+) operator. An open record initially stisfies 'c = 'b and can be sealed once 'c = 'a.

record f is an incomplete representation of the record of type 'a with constructor f. To complete the representation, add fields with (|+) and then seal the record with sealr.

The type for fields holding values of type 'b and belonging to a record of type 'a.

field n t g is the representation of the field called n of type t with getter g. For instance:

type t = { foo: string }

let foo = field cstring (fun t -> t.foo)

r |+ f is the open record r augmented with the field f.

sealr r seals the open record r.

The type for representing open variants of type 'a with pattern-matching of type 'b. 'c represents the remaining constructors to be described using the (|~) operator. An open variant initially satisfies 'c = 'b and can be sealed once 'c = 'a.

variant n p is an incomplete representation of the variant type called n of type 'a using p to deconstruct values. To complete the representation, add cases with (|~) and then seal the variant with sealv.

The type for representing variant cases of type 'a with patterns of type 'b.

The type for representing patterns for a variant of type 'a.

case0 v is a representation of a variant constructor v with no arguments. For instance:

type t = Foo

let foo = case0 Foo

case1 n t c is a representation of a variant constructor c with an argument of type t. For instances:

type t = Foo of string

let foo = case1 cstring (fun s -> Foo s)

v |~ c is the open variant v augmented with the case c.

sealv v seals the open variant v.

fix @@ fun t -> ... computes the fixpoint of the given function and runs the resultant codec. The argument that fn receives is the result of fix fn, which fn must use, paradoxically, to define fix fn.

fix is useful when constructing codecs for inductively-defined types such as sequences, trees, etc. Consider for example the codec of a QNAME (see RFC1035 § 4.1.2). They describe it as:

> a domain name represented as a sequence of labels, where each label consists of a length octet followed by that number of octets. The domain name terminates with the zero length octet for the null label of the root.

Here is the equivalent using Bin:

type name = string list

let qname =
  let open Bin in
  fix @@ fun qname ->
  let prj = function
    | 0 -> const []
    | len ->
        record (fun label rest -> label :: rest)
        |+ field (bytes (fixed len)) List.hd
        |+ field qname List.tl
        |> sealr
  in
  let inj = function [] -> 0 | label :: _ -> String.length label in
  bind uint8 prj inj

decode_bstr repr is the binary decoder for values of type repr on Bstr.t.

len is the window the decoder is allowed to read within: len byte(s) counted from the cursor. It defaults to the rest of the buffer, and it is what rest means and what bounds a delim search. Such a window is what a slice describes, so decoding from one is:

  let decode = Bin.Staged.unstage (Bin.decode_bstr t) in
  let { Slice.buf; off; len } = slice in
  let pos = ref (Bin.Off.v off) in
  let v = decode buf ~len pos in
  ...

• raises Invalid_argument

  if we try to manipulate something outside the given window.

NOTE: that len is counted from the cursor, not from the beginning of the buffer.

decode repr is the binary decoder for values of type repr on string. len bounds the decoder as in decode_bstr.

encode_bstr repr is the binary encoder for value of type repr on Bstr.t. As for decode_bstr, len is the window the encoder is allowed to write within: len byte(s) counted from the cursor, the rest of the buffer by default. Writing into a slice is therefore:

let encode = Bin.Staged.unstage (Bin.encode_bstr t) in
let { Slice.buf; off; len } = slice in
encode v buf ~len (ref (Bin.Off.v off))

• raises Invalid_argument

  if we try to write outside the given window.

to_string repr is a function which returns the string representation of values of type repr.

size_of_value encoding value attempts to calculate the number of bytes needed to encode the given value according to the given encoding.