package ppx_sexp_conv
[@@deriving] plugin to generate S-expression conversion functions
Install
Dune Dependency
Authors
Maintainers
Sources
ppx_sexp_conv-v0.16.0.tar.gz
sha256=41bcb7a3b33bdf50428408bfaf1dbcede528a488ac8c436ce710681bcd91200d
Description
Part of the Jane Street's PPX rewriters collection.
Published: 02 Jun 2023
README
README.org
#+TITLE: ppx_sexp_conv * [@@deriving sexp] =ppx_sexp_conv= is a PPX syntax extension that generates code for converting OCaml types to and from s-expressions, as defined in the [[https://github.com/janestreet/sexplib][=sexplib=]] library. S-expressions are defined by the following type: #+begin_src ocaml type sexp = Atom of string | List of sexp list #+end_src and are rendered as parenthesized lists of strings, /e.g./ =(This (is an) (s expression))=. =ppx_sexp_conv= fits into the [[https://github.com/whitequark/ppx_deriving][=ppx_deriving=]] framework, so you can invoke it the same way you invoke any other deriving plug-in. Thus, we can write #+begin_src ocaml type int_pair = (int * int) [@@deriving sexp] #+end_src to get two values defined automatically, =sexp_of_int_pair= and =int_pair_of_sexp=. If we only want one direction, we can write one of the following. #+begin_src ocaml type int_pair = (int * int) [@@deriving sexp_of] type int_pair = (int * int) [@@deriving of_sexp] #+end_src These sexp-converters depend on having a set of converters for basic values (/e.g./, =int_of_sexp=) already in scope. This can be done by writing: #+begin_src ocaml open Sexplib.Std #+end_src If you're using [[https://github.com/janestreet/core][=Core=]], you can get the same effect with =open Core=. It's also possible to construct converters based on type expressions, /i.e./: #+begin_src ocaml [%sexp_of: (int * string) list] [1,"one"; 2,"two"] |> Sexp.to_string;; => "((1 one) (2 two))" [%sexp_of: (int * string) list] [1,"one"; 2,"two"] |> [%of_sexp: (int * string) list];; => [1,"one"; 2,"two"] #+end_src For =%sexp_of=, we can also omit the conversion of some types by putting underscores for that type name. #+begin_src ocaml [%sexp_of: (int * _) list] [1,"one"; 2,"two"] |> Sexp.to_string;; => "((1 _)(2 _))" #+end_src * [@@deriving sexp_grammar] If =ppx_sexp_conv= can derive =of_sexp=, it can also generate a description of the sexps that the resulting =t_of_sexp= would accept. This is the sexp grammar. See =Sexplib0.Sexp_grammar= for details. Use =[@@deriving sexp_grammar]= to derive the grammar for a type. It is possible to construct sexp grammars directly from type expressions, e.g., #+BEGIN_SRC ocaml [%sexp_grammar: (int, bool array) Either.t Base.Map.M(String).t] #+END_SRC ** Tagging grammars Use =[@sexp_grammar.tag key = value]=, where =(key : string)= and =(value : Sexp.t)=, to annotate a grammar with a tag that can be inspected at runtime. ** Custom grammars Use =[@sexp_grammar.custom grammar]= to override a type's sexp grammar with =grammar=. ** Stub grammars Annotate a type with =[@sexp_grammar.any]= to use a stub grammar that accepts any sexp. Alternately, write =[@sexp_grammar.any desc]= where =(desc : string)= to use =desc= as a human-readable description for the stub grammar. * Conversion rules In the following, we'll review the serialization rules for different OCaml types. ** Basic types Basic types are represented as atoms. For numbers like =int=, =int32=, =int64=, =float=, the string in the atom is what is accepted the standard ocaml functions =int_of_string=, =Int32.of_string=, etc. For the types =char= or =string=, the string in the atom is respectively a one character string or the string itself. ** Lists and arrays OCaml-lists and arrays are represented as s-expression lists. ** Tuples and unit OCaml tuples are treated as lists of values in the same order as in the tuple. The type =unit= is treated like a 0-tuple. /e.g./: #+begin_src ocaml (3.14, "foo", "bar bla", 27) => (3.14 foo "bar bla" 27) #+end_src ** Options With options, =None= is treated as a zero-element list, and =Some= is treated as a singleton list, as shown below. #+begin_src ocaml None => () Some value => (value) #+end_src We also support reading options following the ordinary rules for variants /i.e./: #+begin_src ocaml None => None Some value => (Some value) #+end_src The rules for variants are described below. ** Records Records are represented as lists of lists, where each inner list is a key-value pair. Each pair consists of the name of the record field (first element), and its value (second element). /e.g./: #+begin_src ocaml { foo = (3,4); bar = "some string"; } => ((foo (3 4)) (bar "some string")) #+end_src Type specifications of records allow the use of several attributes. The attribute =sexp.option= indicates that a record field should be optional. /e.g./: #+begin_src ocaml type t = { x : int option; y : int option [@sexp.option]; } [@@deriving sexp] #+end_src The following examples show how this works. #+begin_src ocaml { x = Some 1; y = Some 2; } => ((x (1)) (y 2)) { x = None ; y = None; } => ((x ())) #+end_src Note that, when present, an optional value is represented as the bare value, rather than explicitly as an option. The attribute =sexp.bool= indicates that a boolean record field is shown as either present or absent, but not as containing a value. #+begin_src ocaml type t = { enabled : bool [@sexp.bool] } [@@deriving sexp] { enabled = true } => ((enabled)) { enabled = false } => () #+end_src The attributes =sexp.list= and =sexp.array= indicate that a list or array record field, respectively, can be omitted when it is empty. #+begin_src ocaml type t = { arr : int array [@sexp.array] ; lst : int list [@sexp.list] } [@@deriving sexp] { arr = [||]; lst = [] } => () { arr = [|1;2|]; lst = [3;4] } => ((arr (1 2)) (lst (3 4))) #+end_src *** Defaults More complex default values can be specified explicitly using several constructs, /e.g./: #+begin_src ocaml type t = { a : int [@default 42]; b : int [@default 3] [@sexp_drop_default (=)]; c : int [@default 3] [@sexp_drop_if fun x -> x = 3]; d : int Queue.t [@sexp.omit_nil] } [@@deriving sexp] #+end_src The =@default= annotation lets one specify a default value to be selected if the field is not specified, when converting from an s-expression. The =@sexp_drop_default= annotation implies that the field will be dropped when generating the s-expression if the value being serialized is equal to the default according to the specified equality function. =@sexp_drop_if= is like =@sexp_drop_default=, except that it lets you specify the condition under which the field is dropped. Finally, =@sexp.omit_nil= means to treat a missing field as if it has value =List []= when reading, and drop the field if it has value =List []= when writing. **** Specifying equality for [@sexp_drop_default] The equality used by [@sexp_drop_default] is customizable. There are several ways to specify the equality function: #+begin_src ocaml type t = { a : u [@default u0] [@sexp_drop_default (=)]; (* explicit user-provided function *) b : u [@default u0] [@sexp_drop_default.compare]; (* uses [%compare.equal: u] *) c : u [@default u0] [@sexp_drop_default.equal]; (* uses [%equal: u] *) d : u [@default u0] [@sexp_drop_default.sexp]; (* compares sexp representations *) e : u [@default u0] [@sexp_drop_default]; (* deprecated. uses polymorphic equality. *) } [@@deriving sexp] #+end_src *** Allowing extra fields The =@sexp.allow_extra_fields= annotation lets one specify that the sexp-converters should silently ignore extra fields, instead of raising. This applies only to the record to which the annotation is attached, and not to deeper sexp converters that may be called during conversion of a sexp to the record. #+begin_src ocaml type t = { a: int } [@@deriving sexp] ((a 0)(b b)) => exception type t = { a: int } [@@deriving sexp] [@@sexp.allow_extra_fields] ((a 0)(b b)) => {a = 0} type t = A of { a : int } [@sexp.allow_extra_fields] [@@deriving sexp] (A (a 0)(b b)) => A {a = 0} #+end_src ** Variants Constant constructors in variants are represented as strings. Constructors with arguments are represented as lists, the first element being the constructor name, the rest being its arguments. Constructors may also be started in lowercase in S-expressions, but will always be converted to uppercase when converting from OCaml values. For example: #+begin_src ocaml type t = A | B of int * float * t [@@deriving sexp] B (42, 3.14, B (-1, 2.72, A)) => (B 42 3.14 (B -1 2.72 A)) #+end_src The above example also demonstrates recursion in data structures. Variants support the attribute =sexp.list= when a clause has a single list as its argument. #+begin_src ocaml type t = | A of int list | B of int list [@sexp.list] A [1; 2; 3] => (A (1 2 3)) B [1; 2; 3] => (B 1 2 3) #+end_src *** Inline records Constructors with inline records are represented as lists, the first element being the constructor name, the rest being the record fields, represented the same way as in record types, but without being wrapped in an extra layer of parentheses. #+begin_src ocaml type t = A of { x : int } A { x = 8 } => (A (x 8)) #+end_src ** Polymorphic variants Polymorphic variants behave almost the same as ordinary variants. The notable difference is that polymorphic variant constructors must always start with an either lower- or uppercase character, matching the way it was specified in the type definition. This is because OCaml distinguishes between upper and lowercase variant constructors. Note that type specifications containing unions of variant types are also supported by the S-expression converter, for example as in: #+begin_src ocaml type ab = [ `A | `B ] [@@deriving sexp] type cd = [ `C | `D ] [@@deriving sexp] type abcd = [ ab | cd ] [@@deriving sexp] #+end_src However, because `ppx_sexp_conv` needs to generate additional code to support inclusions of polymorphic variants, `ppx_sexp_conv` needs to know when processing a type definition whether it might be included in a polymorphic variant. `ppx_sexp_conv` will only generate the extra code automatically in the common case where the type definition is syntactically a polymorphic variant like in the example above. Otherwise, you will need to indicate it by using `[@@deriving sexp_poly]` (resp `of_sexp_poly`) instead of `[@@deriving sexp]` (resp `of_sexp`): #+begin_src ocaml type ab = [ `A | `B ] [@@deriving sexp] type alias_of_ab = ab [@@deriving sexp_poly] type abcd = [ ab | `C | `D ] [@@deriving sexp] #+end_src ** Polymorphic values There is nothing special about polymorphic values as long as there are conversion functions for the type parameters. /e.g./: #+begin_src ocaml type 'a t = A | B of 'a [@@deriving sexp] type foo = int t [@@deriving sexp] #+end_src In the above case the conversion functions will behave as if =foo= had been defined as a monomorphic version of =t= with ='a= replaced by =int= on the right hand side. If a data structure is indeed polymorphic and you want to convert it, you will have to supply the conversion functions for the type parameters at runtime. If you wanted to convert a value of type ='a t= as in the above example, you would have to write something like this: #+begin_src ocaml sexp_of_t sexp_of_a v #+end_src where =sexp_of_a=, which may also be named differently in this particular case, is a function that converts values of type ='a= to an S-expression. Types with more than one parameter require passing conversion functions for those parameters in the order of their appearance on the left hand side of the type definition. ** Opaque values Opaque values are ones for which we do not want to perform conversions. This may be, because we do not have S-expression converters for them, or because we do not want to apply them in a particular type context. /e.g./ to hide large, unimportant parts of configurations. To prevent the preprocessor from generating calls to converters, simply apply the attribute =sexp.opaque= to the type. If the type is for a record field, it will likely need parentheses to avoid applying the attribute to the record field itself, /e.g./: #+begin_src ocaml type foo = int * (stuff [@sexp.opaque]) [@@deriving sexp] type bar = { a : int ; b : (stuff [@sexp.opaque]) } [@@deriving sexp] #+end_src Thus, there is no need to specify converters for type =stuff=, and if there are any, they will not be used in this particular context. Needless to say, it is not possible to convert such an S-expression back to the original value. Here is an example conversion: #+begin_src ocaml (42, some_stuff) => (42 <opaque>) #+end_src ** Exceptions S-expression converters for exceptions can be automatically registered. #+begin_src ocaml module M = struct exception Foo of int [@@deriving sexp] end #+end_src Such exceptions will be translated in a similar way as sum types, but their constructor will be prefixed with the fully qualified module path (here: =M.Foo=) so as to be able to discriminate between them without problems. The user can then easily convert an exception matching the above one to an S-expression using =sexp_of_exn=. User-defined conversion functions can be registered, too, by calling =add_exn_converter=. This should make it very convenient for users to catch arbitrary exceptions escaping their program and pretty-printing them, including all arguments, as S-expressions. The library already contains mappings for all known exceptions that can escape functions in the OCaml standard library. ** Hash tables The Stdlib's Hash tables, which are abstract values in OCaml, are represented as association lists, /i.e./ lists of key-value pairs, /e.g./: #+begin_src scheme ((foo 42) (bar 3)) #+end_src Reading in the above S-expression as hash table mapping strings to integers (=(string, int) Hashtbl.t=) will map =foo= to =42= and =bar= to =3=. Note that the order of elements in the list may matter, because the OCaml-implementation of hash tables keeps duplicates. Bindings will be inserted into the hash table in the order of appearance. Therefore, the last binding of a key will be the "visible" one, the others are "hidden". See the OCaml documentation on hash tables for details. * A note about signatures In signatures, =ppx_sexp_conv= tries to generate an include of a named interface, instead of a list of value bindings. That is: #+begin_src ocaml type 'a t [@@deriving sexp] #+end_src will generate: #+begin_src ocaml include Sexpable.S1 with type 'a t := 'a t #+end_src instead of: #+begin_src ocaml val t_of_sexp : (Sexp.t -> 'a) -> Sexp.t -> 'a t val sexp_of_t : ('a -> Sexp.t) -> 'a t -> Sexp.t #+end_src There are however a number of limitations: - the type has to be named t - the type can only have up to 3 parameters - there shouldn't be any constraint on the type parameters If these aren't met, then =ppx_sexp_conv= will simply generate a list of value bindings. ** Weird looking type errors In some cases, a type can meet all the conditions listed above, in which case the rewriting will apply, but lead to a type error. This happens when the type [t] is an alias to a type which does have constraints on the parameters, for instance: #+begin_src ocaml type 'a s constraint 'a = [> `read ] val sexp_of_s : ... val s_of_sexp : ... type 'a t = 'a s [@@deriving_inline sexp] include Sexpable.S1 with type 'a t := 'a t [@@@end] #+end_src will give an error looking like: #+begin_src Error: In this `with' constraint, the new definition of t does not match its original definition in the constrained signature: Type declarations do not match: type 'a t = 'a t constraint 'a = [> `read ] is not included in type 'a t File "sexpable.mli", line 8, characters 21-58: Expected declaration Their constraints differ. #+end_src To workaround that error, simply copy the constraint on the type which has the =[@@deriving]= annotation. This will force generating a list of value bindings. * Deprecated syntax Originally, ~ppx_sexp_conv~ used special types instead of attributes. Those types have been replaced with attributes. Here are the appropriate conversions to update from code using the old types to the new attributes. ** Opaque types Convert uses of ~sexp_opaque~ to uses of ~[@sexp.opaque]~. The ~[@sexp.opaque]~ attribute usually needs explicit parentheses to clarify what type it annotate. Before: #+begin_src ocaml type t = int sexp_opaque list [@@deriving sexp] #+end_src After: #+begin_src ocaml type t = (int [@sexp.opaque]) list [@@deriving sexp] #+end_src ** Record fields Convert uses of ~sexp_option~, ~sexp_list~, ~sexp_array~, and ~sexp_bool~ to uses of ~[@sexp.option]~, ~[@sexp.list]~, ~[@sexp.array]~, and ~[@sexp.bool]~ as appropriate. The attribute only specifies the modification, not the type, so you will need to use the regular types ~option~, ~list~, ~array~, and/or ~bool~ as well. Unlike ~[@sexp.opaque]~, these attributes do not need extra parentheses. Before: #+begin_src ocaml type t = { a : int sexp_option ; b : int sexp_list ; c : int sexp_array ; d : sexp_bool } [@@deriving sexp] #+end_src After: #+begin_src ocaml type t = { a : int option [@sexp.option] ; b : int list [@sexp.list] ; c : int array [@sexp.array] ; d : bool [@sexp.bool] } [@@deriving sexp] #+end_src ** Variant constructors Convert uses of ~sexp_list~ in variants and polymorphic variants to uses of ~[@sexp.list]~. You need to add the regular type ~list~ as well. Unlike ~[@sexp.opaque]~, this attribute does not need extra parentheses. Before: #+begin_src ocaml type t = A of int sexp_list [@@deriving sexp] type u = [`B of int sexp_list] [@@deriving sexp] #+end_src After: #+begin_src ocaml type t = A of int list [@sexp.list] [@@deriving sexp] type u = [`B of int list [@sexp.list]] [@@deriving sexp] #+end_src
Dependencies (5)
Dev Dependencies
None
-
amf
< "0.1.2"
-
async-zmq
>= "0.3.0"
-
awa
< "0.2.0"
- azblob
-
bin_prot
= "v0.16.0"
-
bio_io
>= "0.5.1"
-
biocaml
>= "0.11.0"
-
bistro
>= "0.6.0"
-
bookaml
>= "3.1"
- caldav
- charrua
-
charrua-core
>= "0.3"
- charrua-server
-
cohttp
>= "0.20.1"
-
cohttp-async
>= "2.5.2" & < "2.5.3" | >= "2.5.6" & < "3.0.0" | >= "5.0.0"
-
cohttp-lwt
>= "1.1.1"
-
cohttp-lwt-unix
>= "4.0.0"
-
cohttp-mirage
>= "4.0.0"
-
combinat
< "3.0"
-
conduit
>= "0.12.0" & != "3.0.0"
-
conduit-async
!= "3.0.0"
-
conduit-lwt
!= "3.0.0"
-
conduit-lwt-unix
>= "2.1.0"
-
conduit-mirage
!= "3.0.0"
- cookies
-
coq-lsp
>= "0.2.0+8.17"
-
coq-serapi
>= "8.10.0+0.7.1" & < "8.11.0+0.11.0" | >= "8.17.0+0.17.1"
-
core
>= "v0.16.0" & < "v0.17.0"
-
core_bench
= "v0.16.0"
- cppffigen
-
crc
>= "2.1.0"
- cudajit
-
current_web
>= "0.4"
- datakit-ci
-
diet
< "0.4"
- disml
- dns-forward
-
dockerfile
>= "1.3.0"
-
dockerfile-cmd
>= "6.0.0"
-
dockerfile-opam
>= "7.1.0"
- erlang
-
ezjsonm-lwt
>= "1.1.0"
-
frenetic
>= "5.0.0" & < "5.0.5"
-
gopcaml-mode-merlin
< "0.0.6"
-
graphql
< "0.4.0"
-
graphql-lwt
< "0.3.0"
-
graphql_parser
< "0.9.0"
- h1_parser
-
hardcaml
= "v0.16.0"
- hl_yaml
- http
-
ibx
>= "0.8.1"
-
ipaddr
>= "2.8.0"
- ipaddr-sexp
- jsonxt
-
lazy-trie
>= "1.2.0"
- ldp_tls
-
learn-ocaml
>= "0.13.0"
-
learn-ocaml-client
>= "0.13.0"
-
links
>= "0.9.2"
-
little_logger
< "0.3.0"
- lua_parser
- m_tree
- macaddr
- macaddr-sexp
- mecab
-
message-switch
>= "1.4.0"
-
mirage-block-unix
>= "2.10.0" & < "2.11.1"
-
mirage-conduit
< "2.0.0" | >= "3.0.0"
-
mirage-crypto-pk
< "0.10.4"
-
mirage-net-xen
>= "1.6.0" & < "1.7.1"
-
n_ary
< "v0.17.0"
-
nbd
>= "2.1.0" & < "2.1.3" | >= "3.0.0"
- netchannel
-
nocrypto
>= "0.5.4-1"
-
nsq
>= "0.2.5" & < "0.5.2"
- nuscr
-
obeam
>= "0.1.0"
- obuilder
- obuilder-spec
- ocaml-basics
-
ocaml-topexpect
>= "0.3"
- oci
- ocluster
- odep
- opam-check-npm-deps
-
opass
>= "1.0.6"
- opine
-
opium
>= "0.15.0" & < "0.19.0"
- opium_kernel
- p5scm
-
pa_ppx
< "0.07" | >= "0.10"
-
pgocaml
>= "4.2"
-
pgocaml_ppx
>= "4.2" & < "4.3.0"
-
pgx
>= "1.0"
-
planck
>= "2.2.0"
-
posixat
= "v0.16.0"
-
ppx_assert
= "v0.16.0"
-
ppx_bap
< "v0.14.0"
-
ppx_base
= "v0.16.0"
-
ppx_css
= "v0.16.0"
-
ppx_cstruct
>= "3.1.0"
-
ppx_custom_printf
= "v0.16.0"
-
ppx_hash
= "v0.16.0"
-
ppx_import
>= "1.9.0"
-
ppx_log
= "v0.16.0"
- ppx_minidebug
-
ppx_protocol_conv
>= "5.1.2"
-
ppx_protocol_conv_json
>= "3.1.0" & != "4.0.0"
-
ppx_protocol_conv_jsonm
!= "4.0.0"
-
ppx_protocol_conv_msgpack
>= "3.1.0" & != "4.0.0"
-
ppx_protocol_conv_xml_light
>= "3.1.0" & != "4.0.0"
- ppx_protocol_conv_xmlm
-
ppx_protocol_conv_yaml
>= "3.1.0" & != "4.0.0"
-
ppx_sexp_message
= "v0.16.0"
-
ppx_sexp_value
= "v0.16.0"
-
ppx_typed_fields
= "v0.16.0"
-
protocol-9p
>= "0.6.0" & < "2.0.2"
-
protocol-9p-unix
!= "2.0.0" & < "2.0.2"
-
provider
< "0.0.9"
- pyml_bindgen
- pyre-ast
-
qcow
>= "0.10.0"
-
qcow-format
>= "0.3"
-
reparse
>= "1.0.1" & < "2.0.0"
-
routes
>= "2.0.0"
-
SZXX
>= "4.1.0"
- sel
-
sexp_grammar
= "v0.16.0"
-
shared-block-ring
>= "2.3.0"
-
sihl
< "0.1.0"
- sihl-core
-
spin
< "0.6.0"
-
ssh-agent
< "0.4.0"
- sslconf
-
tdigest
>= "2.2.0"
-
tls
>= "0.9.2" & < "0.17.0"
-
torch
< "v0.16.0"
-
um-abt
>= "0.1.7"
-
uri
>= "1.9.2"
- uri-re
- uri-sexp
-
vchan
>= "2.1.0" & < "6.0.2"
-
vchan-unix
< "6.0.2"
-
vchan-xen
< "6.0.2"
- vendredi
-
vmnet
>= "1.1.0"
- vscoq-language-server
-
wamp
= "1.0"
-
x509
>= "0.6.2" & < "0.7.0"
- xapi-backtrace
- xapi-idl
-
yaml
>= "0.2.0" & < "3.0.0"
- yaml-sexp
Conflicts
None
sectionYPositions = computeSectionYPositions($el), 10)"
x-init="setTimeout(() => sectionYPositions = computeSectionYPositions($el), 10)"
>
On This Page