rpclib-html

A library to deal with RPCs in OCaml - html documentation generator
README

ocaml-rpc is a library that provides remote procedure calls (RPC)
using XML or JSON as transport encodings. The transport mechanism itself
is outside the scope of this library as all conversions are from and to
strings. The odoc generated documentation is available at
mirage.github.io/ocaml-rpc/rpclib.

RPC types

An RPC value is defined as follow:

type t =
    Int of int64
  | Int32 of int32
  | Bool of bool
  | Float of float
  | String of string
  | DateTime of string
  | Enum of t list
  | Dict of (string * t) list
  | Base64 of string
  | Null

Generating code

The idea behind ocaml-rpc is to generate type definitions that can be used to
convert values of a given type to and from their RPC representations.

In order to do so, it is sufficient to add [@@deriving rpcty] to the
corresponding type definition. Hence :

type t = ... [@@deriving rpcty]

This will give a value typ_of_t of type Rpc.Types.typ, which can be used in
conjunction with the Rpcmarshal module to:

  • Convert values of type t to values of type Rpc.t:

    let rpc_of_t t = Rpcmarshal.marshal typ_of_t t
    
  • Convert values of type Rpc.t to values of type t :

    let t_of_rpc rpc = Rpcmarshal.unmarshal typ_of_t rpc
    

Optionally, it is possible to have different field name in the OCaml
type (if it is a record) and in the dictionary argument (the first
elements of Dict):

type t = { foo: int [@key "type"]; bar: int [@key "let"]; } [@@deriving rpcty]

This will replace "foo" by "type" and "bar" by "let" in the RPC
representation. This is particularly useful when you want to integrate
with an existing API and the field names are not valid OCaml identifiers.

The library also provides the [@@deriving rpc] ppx, which is similar to
rpcty, but directly generates the conversion functions.

type t = ... [@@deriving rpc]

will give two functions:

  • A function to convert values of type t to values of type Rpc.t :
    val rpc_of_t : t -> Rpc.t

  • A function to convert values of type Rpc.t to values of type t :
    val t_of_rpc : Rpc.t -> (t,string) Result.t

It also supports the @key annotations for having different field names:

type t = { foo: int [@key "type"]; bar: int [@key "let"]; } [@@deriving rpc]

Conversion functions

ocaml-rpc currently support two protocols: XMLRPC and JSON(RPC). Function
signatures are:

val Xmlrpc.to_string : Rpc.t -> string
val Xmlrpc.of_string : string -> Rpc.t
val Jsonrpc.to_string : Rpc.t -> string
val Jsonrpc.of_string : string -> Rpc.t

So if you want to marshal a value x of type t to JSON, you can use the
following function:

Jsonrpc.to_string (rpc_of_t x)

IDL generator

The Idl module makes it possible to define an abstract interface in OCaml
using the following pattern:

module CalcInterface(R : Idl.RPC) = struct
  open R

  let int_p = Idl.Param.mk Rpc.Types.int

  let add = R.declare "add"
      ["Add two numbers"]
      (int_p @-> int_p @-> returning int_p Idl.DefaultError.err)

  let mul = R.declare "mul"
      ["Multiply two numbers"]
      (int_p @-> int_p @-> returning int_p Idl.DefaultError.err)

  let implementation = implement
      { Idl.Interface.name = "Calc"; namespace = Some "Calc"; description = ["Calculator supporting addition and multiplication"]; version = (1,0,0) }
end

Then we can generate various "bindings" from it by passing a module
implementing the RPC signature to this functor:

  • OCaml bindings for clients or servers can be generated using one of the
    GenClient* or GenServer* functors, respectively.

    For example one can generate an RPC client this way:

    module CalcClient :
      sig
        val add :
          (Rpc.call -> Rpc.response) ->
          int -> int -> (int, Idl.DefaultError.t) result
        val mul :
          (Rpc.call -> Rpc.response) ->
          int -> int -> (int, Idl.DefaultError.t) result
      end = CalcInterface(Idl.GenClient ())
    

    The functions in the resulting CalcClient module can be used to call their
    corresponding RPC methods.
    CalcClient does not implement the transport mechanism itself, that should
    be provided by passing an a rpc function of type Rpc.call -> Rpc.response.

    CalcClient.add rpc 4 5 will marshal the parameters 4 and 5 into their
    RPC representations, construct an Rpc.call, pass that call to the given rpc
    function, and return either an Ok containing the unmarshalled result or an
    Error with the error description depending on the response returned by
    rpc.

    There are variations of the GenClient module:

    GenClientExn raises an exception in case the response indicates a failure,
    instead of returning a result:

    module CalcClient :
      sig
        val add : (Rpc.call -> Rpc.response) -> int -> int -> int
        val mul : (Rpc.call -> Rpc.response) -> int -> int -> int
      end = CalcInterface(Idl.GenClientExn ())
    

    and GenClientExnRpc allows one to specify the rpc function once when
    constructing the client module:

    module CalcClient :
      sig
        val add : int -> int -> int
        val mul : int -> int -> int
      end = CalcInterface(Idl.GenClientExnRpc (struct let rpc = rpc end))
    

    Bindings for a server can be generated in a similar way:

    module CalcServer :
      sig
        val add : (int -> int -> (int, Idl.DefaultError.t) result) -> unit
        val mul : (int -> int -> (int, Idl.DefaultError.t) result) -> unit
        val implementation : Idl.server_implementation
      end = CalcInterface(Idl.GenServer ())
    

    The implementations of each RPC method should be specified by passing it to
    the corresponding function in CalcServer:

    CalcServer.add (fun a b -> Ok (a + b));
    CalcServer.mul (fun a b -> Ok (a * b));
    

    Then we can generate our server from the implementation (in case of
    GenClient, implementation is unused):

    let rpc : (Rpc.call -> Rpc.response) = Idl.server CalcServer.implementation
    

    Again, the transport mechanism is not implemented by CalcServer. We just
    get an rpc function that, given an Rpc.call, calls the implementation of
    that RPC method and performs the marshalling and unmarshalling.
    It is up to the user of this library to connect this rpc function to a real
    server that responds to client requests.

    Here we also have a GenServerExn functor, for server implementations that
    raise exceptions instead of returning a result.

    The rpclib-lwt and rpclib-async packages provide similar client and
    server generators that use Lwt and Async, respectively.

    The Xmlrpc and Jsonrpc modules can be helpful when implementing the rpc
    function for an XML-RPC or JSON-RPC client/server: they provide functions for
    converting rpc requests and responses to/from their respective wire formats.

  • Commandline interfaces can be generated using Cmdlinergen:

    module CalcCli :
      sig
        val implementation :
          unit ->
          ((Rpc.call -> Rpc.response) ->
           (unit -> unit) Cmdliner.Term.t * Cmdliner.Term.info)
          list
      end = CalcInterface(Cmdlinergen.Gen ())
    

    We can use the implementation to construct the CLI.
    Again, we need to pass an rpc function that knows how to make RPC calls.

    let () =
      let cmds = (List.map (fun t -> t rpc) (CalcCli.implementation ())) in
      let open Cmdliner in
      Term.(exit @@ eval_choice default_cmd cmds)
    
  • Some generators use the output of Codegen. This functor generates a structure
    that contains information about the methods, their parameters, return types,
    etc. Currently these generators that use the output of Codegen require the
    method parameters to be named.

    module CalcInterface(R : Idl.RPC) = struct
      open R
    
      let int_p_1 = Idl.Param.mk ~name:"int1" Rpc.Types.int
      let int_p_2 = Idl.Param.mk ~name:"int2" Rpc.Types.int
      let int_p = Idl.Param.mk Rpc.Types.int
    
      let add = R.declare "add"
          ["Add two numbers"]
          (int_p_1 @-> int_p_2 @-> returning int_p Idl.DefaultError.err)
    
      let implementation = implement
          { Idl.Interface.name = "Calc"; namespace = Some "Calc"; description = ["Calculator supporting addition and multiplication"]; version = (1,0,0) }
    end
    
    module CalcCode :
      sig
        val implementation : unit -> Codegen.Interface.t
      end = CalcInterface(Codegen.Gen ())
    
    let interfaces = Codegen.Interfaces.create
      ~name:"calc"
      ~title:"Calculator"
      ~description:["Interface for a Calculator"]
      ~interfaces:[CalcCode.implementation ()]
    
    • Markdowngen can generate a markdown file documenting these interfaces:

      let md = Markdowngen.to_string interfaces
      
    • Pythongen can generate Python code that contains various classes
      wrapping a Python implementation, providing typechecking & method
      dispatch, and a CLI.

      let code = Pythongen.of_interfaces interfaces |> Pythongen.string_of_ts
      

The possibilities are not limited to the above generators provided by
ocaml-rpc. Any third-party module implementing the RPC signature can be
used to generate something from an interface defined following the above
pattern. For example, it is possible to write an RPC implementation that
generates a GUI for a given interface.

Base64 Decoding

The treatment of line feeds (and other characters) in
XML-RPC base64-encoded data is underspecified.

By default, this library decodes values using the Base64.decode_exn
function of ocaml-base64. This
function implements RFC4648
which requires the rejection of non-alphabet characters for security reasons
(see section 3.3
and also section 3.1).

This is problematic when communicating with servers that are less strict.
For instance, the
encode
function of the Python xmlrpc.client refers to section 6.8 of
RFC2045
to
justify inserting a newline character every 76 characters.
For this reasons, the functions in Xmlrpc allow the caller to override the
base64_decoder. The following declaration gives a rough-and-ready
“dangerous” implementation based on the Base64.rfc2045 package.
A better implementation would only accept a \n every 76 characters.

let base64_2045_decoder s =
  let open Base64_rfc2045 in
  let buf = Buffer.create 1024 in
  let d = decoder (`String s) in
  let rec go () =
    match decode d with
    | `Flush s -> (Buffer.add_string buf s; go ())
    | `End -> Buffer.contents buf
    (* best-effort *)
    | `Malformed _   (* ignore missing '\r' before '\n', etc. *)
    | `Wrong_padding (* ignore *)
    | `Await -> go ()
  in
  go ()

Building

To build, first install the dependencies:

opam install dune base64 ppxlib async js_of_ocaml-ppx lwt cow cmdliner rresult yojson xmlm

For tests:

opam install alcotest alcotest-lwt
Install
Sources
rpclib-8.1.2.tbz
sha256=b1f0e975f8522a91cccef03569b8f372eca4e666e9f3c20d2a9d82d0e096b235
sha512=ef75ec6032805b08bd49716d5e9a4ca888c1a581b50e4ad5cc1d003cd182a47cfc87c945e7564251c8e041fc3ffa1b11e0579597fc9806db88955b3e029ba4a3
Dependencies
cow
>= "2.0.0"
rpclib
= version
dune
>= "2.0.0"
Reverse Dependencies