package ppx_diff
Install
Dune Dependency
Authors
Maintainers
Sources
sha256=5e817094edf127d384110227ecfdc3e23f0f130266d48d1f326a03f6f58a2609
Description
A PPX rewriter that generates the implementation of [Ldiffable.S]. Generates diffs and update functions for OCaml types.
Published: 23 May 2024
README
ppx_diff
Generation of diffs and update functions for ocaml types.
ppx_diff
is a ppx rewriter that generates the implementation of [Diffable.S]. The [Diff.t] type represents differences between two values. The [Diff.get] and [Diff.apply_exn] functions compute and apply the differences.
Users of ppx_diff
should add the [diffable] library to the dependencies in [jbuild].
Usage
Basic usage
If you define a type as follows:
type t [@@deriving diff]
then code will be generated matching the following signature:
module Diff : sig
type derived_on = t
type t
val get : from : derived_on > to_ : derived_on > local_ t Optional_diff.t
val apply_exn : derived_on > t > derived_on
val of_list_exn : t list > local_ t Optional_diff.t
end
Use of [@@deriving diff]
in an .mli will extend the signature with the above module. In an .ml, definitions will be generated.
Serialization and deserialization functions
You will likely want your diff type to derive some subset of bin_io
, sexp
, of_sexp
, sexp_of
.
By default [@@deriving diff]
will include the same subset of the above as those included on your original type.
So e.g. if you define a type as follows:
type t [@@deriving sexp, bin_io, diff]
then code will be generated matching the following signature:
module Diff : sig
type derived_on = t
type t [@@deriving sexp, bin_io]
val get : from : derived_on > to_ : derived_on > local_ t Optional_diff.t
val apply_exn : derived_on > t > derived_on
val of_list_exn : t list > local_ t Optional_diff.t
end
If you want to derive additional items on your diff type, you can achieve this using the extra_derive
argument.
For instance, you can write:
type t [@@deriving sexp_of, diff ~extra_derive:[of_sexp ; bin_io]]
Signature explanation
Optional Diff
Optional_diff
has the following accessor functions:
module Optional_diff : sig
type 'a t
val is_none : local_ _ t > bool
val unsafe_value : local_ 'a t > 'a
val to_option : local_ 'a t > 'a option
end
It is designed to improve allocations. Specifically, if you have a local_ 'a t
, that means that 'a
was allocated, but the pointer to 'a option
was not.
Exn functions
Why are the function called apply_exn
and of_list_exn
instead of apply
and of_list
?
Rest assured that calling
Option.iter (Diff.get ~from ~to_ > Optional_diff.to_option) ~f:(Diff.apply_exn from)
will never raise.
However, applying a handcrafted diff might raise for some types (in particular for variants when the diff is for an unexpected variant). Similarly, handcrafing diffs using of_list_exn
may raise if trying to combine diffs for different variants.
Nesting
By default, diffs will be "nested". This means if a type deriving diff
references other types, those types must also be diffable.
For example, for the following to work
type t =
{ x : X.t
; y : Y.t
; z : Z.t
} [@@deriving diff]
X.t
, Y.t
and Z.t
must also either derive diff
or otherwise implement Diffable.S
.
Some types are already supported, so it's ok to reference those. This includes:
basic types:
int
,float
,string
,bool
,char
,unit
andoption
Set
andMap
created using the variousComparable.Make
functors
Parametric types
ppx_diff
works also with parametric polymorphic types. In this case the [Diff.get] and [Diff.apply_exn] functions require diff functions for all the types in order of their appearance in the type definition. This is similar to what you may be used to with e.g. t_of_sexp
functions.
The [Diff.of_list_exn]
function requires [of_list_exn]
and [apply_exn]
of all the types in order of their appearance. (The [apply_exn]
is needed for variant types. If you combine something like [Set_to_variant_X of a]
with [Diff_variant_X of a_diff]
, the result should be [Set_to_variant_X (apply_a_exn a a_diff)]
)
If you define a type as follows:
type ('a, 'b) t [@@deriving diff]
then code will be generated matching following signature:
module Diff : sig
type ('a, 'b) derived_on = ('a, 'b) t
type ('a, 'b, 'a_diff, 'b_diff) t
val get
: (from:'a > to_:'a > local_ 'a_diff Optional_diff.t)
> (from:'b > to_:'b > local_ 'b_diff Optional_diff.t)
> from:('a, 'b) derived_on
> to_:('a, 'b) derived_on
> local_ ('a, 'b, 'a_diff, 'b_diff) t Optional_diff.t
val apply_exn
: ('a > 'a_diff > 'a)
> ('b > 'b_diff > 'b)
> ('a, 'b) derived_on
> ('a, 'b, 'a_diff, 'b_diff) t
> ('a, 'b) derived_on
val of_list_exn
: ('a_diff list > local_ 'a_diff Optional_diff.t)
> ('a > 'a_diff > 'a)
> ('b_diff list > local_ 'b_diff Optional_diff.t)
> ('b > 'b_diff > 'b)
> ('a, 'b, 'a_diff, 'b_diff) t list
> local_ ('a, 'b, 'a_diff, 'b_diff) t Optional_diff.t
end
Types with names other than t
ppx_diff
works also with types named something else than t
.
If you define a type as follows
type name [@@deriving diff]
then code will be generated matching the following signature:
module Diff_of_name : sig
type derived_on = name
type t
val get : from : derived_on > to_ : derived_on > local_ t Optional_diff.t
val apply_exn : derived_on > t > derived_on
val of_list_exn : t list > local_ t Optional_diff.t
end
Supported types
Below you can see some examples of what the diff types looks like for various supported types.
Records
The diff of a record is a list of diffs of its fields (and includes only the fields that changed)
type t =
{ x : X.t
; y : Y.t
; z : Z.t
} [@@deriving diff]
module Diff : sig
module Field_diff : sig
type t =  X of X.Diff.t  Y of Y.Diff.t  Z of Z.Diff.t
end
type t = private Field_diff.t list
val create : ?x:X.Diff.t > ?y:Y.Diff.t > ?z:Z.Diff.t > unit > t
val create_of_variants
: x:local_ (X.Diff.t, Field_diff.t) Of_variant.t
> y:local_ (Y.Diff.t, Field_diff.t) Of_variant.t
> z:local_ (Z.Diff.t, Field_diff.t) Of_variant.t
> t
val singleton : Field_diff.t > t
...
end
where
module Of_variant : sig
type ('a, 'diff) t = ('a > 'diff) Variantslib.Variant.t > local_ 'a Optional_diff.t
end
Invariant
The Diff.t
type is private, because the list satisfies two invariants:
(@) There is at most one item of each of X
, Y
and Z
variant of Field_diff.t
in t
(@) The items are in sorted order
Those two invariants allow for a more performant apply_exn
function.
How to create the diff
You can still create items of the Diff.t
type by calling Diff.singleton
, Diff.create
, Diff.create_of_variants
or Diff.of_list_exn
Note that Diff.of_list_exn
will correctly order the diffs and combine diffs for the same field in order to satisfy the invariants.
Record with one field
There is a special case for a record with just one field. We don't return a list, just the (optional) single field diff:
type t = { x : X.t } [@@deriving diff]
module Diff : sig
type t =  X of X.Diff.t
...
end
Tuples
Very similar to records
type t = X.t * Y.t [@@deriving diff]
module Diff : sig
type t = (X.t, Y.t, X.Diff.t, Y.Diff.t) Diffable.Tuples.Tuple2.Diff.t
...
end
where
module Tuple2 : sig
type ('a1, 'a2) t = 'a1 * 'a2 [@@deriving sexp, bin_io]
module Diff : sig
type ('a1, 'a2) derived_on = ('a1, 'a2) t
module Entry_diff : sig
type ('a1, 'a2, 'a1_diff, 'a2_diff) t =
 T1 of 'a1_diff
 T2 of 'a2_diff
[@@deriving variants, sexp, bin_io]
end
type ('a1, 'a2, 'a1_diff, 'a2_diff) t =
private
('a1, 'a2, 'a1_diff, 'a2_diff) Entry_diff.t list
[@@deriving sexp, bin_io]
val create : ?t1:'a1_diff > ?t2:'a2_diff > unit > ('a1, 'a2, 'a1_diff, 'a2_diff) t
val create_of_variants
: t1:local_ ('a1_diff, ('a1, 'a2, 'a1_diff, 'a2_diff) Entry_diff.t) Of_variant.t
> t2:local_ ('a2_diff, ('a1, 'a2, 'a1_diff, 'a2_diff) Entry_diff.t) Of_variant.t
> ('a1, 'a2, 'a1_diff, 'a2_diff) t
val singleton
: ('a1, 'a2, 'a1_diff, 'a2_diff) Entry_diff.t
> ('a1, 'a2, 'a1_diff, 'a2_diff) t
...
end
end
Again the Entry_diff.t
entries in Diff.t
must be sorted and unique, and again create
, create_of_variants
, singleton
functions are provided for convenience.
Tuples of size greater than 6
are not supported, but this can be trivially extended as the Tuple
modules are generated using cinaps.
Constructors
type t = X.t [@@deriving diff]
module Diff : sig
type t = X.Diff.t
...
end
including parametrized ones
type t = Y.t X1.t [@@deriving diff]
module Diff : sig
type t = (Y.t, Y.Diff.t) X.Diff.t
...
end
Vars
type 'a t = 'a [@@deriving diff]
module Diff : sig
type ('a, 'a_diff) t = 'a_diff
...
end
Variants
type t =
 A
 B of B.t
[@@deriving diff]
module Diff : sig
type t =  Set_to_a  Set_to_b of B.t  Diff_b of B.Diff.t
...
end
Inlined tuples and records
Inlined tuples and records are also supported. Their diff types look exactly like those for regular tuples / records.
type t =
 C of C1.t * C2.t
 D of { d1 : D1.t ; d2 : D2.t}
[@@deriving diff]
module Diff : sig
module D_record : sig
type t =
{ global_ d1 : D1.t
; global_ d2 : D2.t
}
module Diff : sig
type derived_on = t
module Field_diff : sig
type t =
 D1 of D1.Diff.t
 D2 of D2.Diff.t
end
type t = private Field_diff.t list
val get
: from:local_ derived_on
> to_:local_ derived_on
> local_ t Optional_diff.t
val apply_exn : local_ derived_on > t > local_ derived_on
val of_list_exn : t list > local_ t Optional_diff.t
val create : ?d1:D1.Diff.t > ?d2:D2.Diff.t > unit > t
val create_of_variants
: d1:local_ (D1.Diff.t, Field_diff.t) Of_variant.t
> d2:local_ (D2.Diff.t, Field_diff.t) Of_variant.t
> t
val singleton : Field_diff.t > t
end
type t =
 Set_to_c of C1.t * C2.t
 Set_to_d of
{ d1 : D1.t
; d2 : D2.t
}
 Diff_c of Diffable.Tuples.Tuple2.For_inlined_tuple.Diff.t
 Diff_d of D_record.Diff.t
...
end
where
type Diffable.Tuples.Tuple2.For_inlined_tuple.Diff.t = Diffable.Tuples.Tuple2.Diff.t
Notice that for inlined records the ppx generates a helper module which looks the same as a regular record module with [@@deriving diff]
, but can be calculated from / applied to local_
values of derived_on
.
Similar helper modules exist for tuples in Diffable.Tuples.TupleN.For_inlined_tuple
Referencing other types
Since diffs are by default nested, any type you reference must also derive diff
or otherwise implement Diffable.S
For example, if Some_type
doesn't implement Diffable.S
, then the following:
type t = Some_type.t * Some_type.t
[@@deriving diff]
Will give you an error like:
Error: Unbound module Some_type.Diff
Often it's enough to just add [@@deriving diff]
to the referenced type.
But occasionally that doesn't quite work. This section covers some special cases and how to handle them.
Atomic diffs
Some types, e.g. any Identifiable
, or indeed Some_type.t
, are super simple, and what we really want is to have type Diff.t = t
and only return a diff if the two values aren't equal. We call such diffs atomic
.
This section explains how you can mark your types as atomically diffable
[@diff.atomic]
You can annotate any part of your type to use atomic diffs using the [@diff.atomic]
attribute.
For instance, the error from the example above will go away if you write
type t =
(Some_type.t [@diff.atomic]) * (Some_type.t [@diff.atomic])
[@@deriving diff]
The annotation also works for record fields:
type t =
{ start : Some_type.t [@diff.atomic]
; stop : Some_type.t [@diff.atomic]
}
[@@deriving diff]
and for variants:
type t =
 Start of Some_type.t [@diff.atomic]
 Stop of Some_type.t [@diff.atomic]
[@@deriving diff]
The two examples above are in fact equivalent to
type t =
{ start : (Some_type.t [@diff.atomic])
; stop : (Some_type.t [@diff.atomic])
}
[@@deriving diff]
and
type t =
 Start of (Some_type.t [@diff.atomic])
 Stop of (Some_type.t [@diff.atomic])
[@@deriving diff]
but slightly nicer to write.
~how:"atomic"
Finally, you can mark your whole type atomic by using the how
parameter:
type t = Some_type.t [@@deriving equal, diff ~how:"atomic"]
Any type that you mark atomic
, must also derive / otherwise implement equal
.
Atomic types
To avoid having to add the annotations everywhere, you can also make the referenced type implement Diffable.S
in an atomic fashion.
To do this, use Diffable.Make_atomic
, e.g.
module Id : sig
type t [@@deriving sexp, bin_io]
include Diffable.S with type t := t and type Diff.t = t
end = struct
type t = string [@@deriving sexp, bin_io, equal]
include functor Diffable.Atomic.Make
end
Note that Diffable.Atomic.Make
requires that your type derives equal
, sexp
and bin_io
.
If you don't derive all of those (but do derive, say, sexp_of
), Diffable.Atomic
will still work but you will need to be more verbose, e.g.
module Id : sig
type t [@@deriving sexp_of]
module Diff : sig
type derived_on = t [@@deriving sexp_of]
type t = derived_on [@@deriving sexp_of]
include Diffable.Diff.S_plain with type t := t and type derived_on := derived_on
end
end = struct
type t = some_type [@@deriving equal, sexp_of]
module Diff = struct
type derived_on = t [@@deriving equal, sexp_of]
type t = derived_on [@@deriving equal, sexp_of]
include functor Diffable.Atomic.Make_diff_plain
end
end
Atomic using compare
If a type doesn't implement equal
, but does implement compare
, you can also use the atomic_using_compare
attribute, which works in exactly the same way as atomic
.
E.g. the following will work:
type t = Some_type.t [@@deriving compare, diff ~how:"atomic_using_compare"]
type t =
{ start : Some_type.t [@diff.atomic_using_compare]
; stop : Some_type.t [@diff.atomic_using_compare]
}
[@@deriving diff]
atomic_using_compare
does not work in mlis/signatures, just continue using atomic
there  the generated signature is exactly the same
A word of warning about NANs
It may come as a surprise, but in Core the following holds
Float.equal Float.nan Float.nan = false
Float.compare Float.nan Float.nan = 0
If you want to avoid spurious NANs in your diffs, e.g. because you want to manually inspect your diffs and don't want them too be noisy, or because you expect there to be a lot of NANs in your values, you need to make sure you use compare
and not equal
In practice this means you'll want to:
Use
atomic_using_compare
and notatomic
If using
Diffable.Atomic.Make
add an overridelet equal = [%compare.equal]
The primitive float
type is already handled using compare
Sets and maps
Sets and maps implemented using the various Make
modules in Comparable
and Comparable.Stable
already implement Diffable.S
and Diffable.S1
respectively, so you can use them out of the box.
All of the following will work:
type t = My_id.Set.t [@@deriving diff]
type t = My_id.Stable.V1.Set.t [@@deriving diff]
type t = { value : float My_id.Map.t } [@@deriving diff]
type t = { value : float My_id.Stable.V1.Map.t } [@@deriving diff]
type 'a t =  Value of 'a My_id.Map.t [@@deriving diff]
type 'a t =  Value of 'a My_id.Stable.V1.Map.t [@@deriving diff]
The diff for sets looks like:
module Diff : sig
type derived_on = t
type t = My_id.t Diffable.Set_diff.t
...
end
where
module Diffable.Set_diff : sig
module Change : sig
type 'a t =  Add of 'a  Remove of 'a
end
type 'a t = 'a Change.t list
end
and the diff for maps looks like:
module Diff : sig
type 'v derived_on = 'v t
type ('v, 'v_diff) t = (My_id.t, 'v, 'v_diff) Diffable.Map_diff.t
...
end
where
module Diffable.Map_diff : sig
module Change : sig
type ('key, 'v, 'v_diff) t =
 Remove of 'key
 Add of 'key * 'v
 Diff of 'key * 'v_diff
[@@deriving sexp, bin_io]
end
type ('key, 'v, 'v_diff) t = ('key, 'v, 'v_diff) Change.t list [@@deriving sexp, bin_io]
end
Map value diffs
By default the values in a map are also diffed.
So if the following:
type t = Some_type.t My_id.Map.t
gives you an error
Error: Unbound module Some_type.Diff
you can fix it by writing
type t = (Some_type.t [@diff.atomic]) My_id.Map.t
Sets and maps with applicative functors
Applicative functors are not supported by diff in general, so none of the following will work:
type t = Set.M(Int).t [@@deriving diff]
type t = float Map.M(Int).t [@@deriving diff]
type t = Set.Stable.V1.M(Int).t [@@deriving diff]
type t = float Map.Stable.V1.M(Int).t [@@deriving diff]
However, there is specific support for applicative functors for sets and maps: you can use set
and map
annotations similar to the atomic
annotation.
Here are some examples of what will work:
type t = Set.M(Int).t [@@deriving diff ~how:"set"]
type t = (float Map.M(Int).t [@diff.map]) [@@deriving diff]
type t = { value : Set.Stable.V1.M(Int).t [@diff.set] } [@@deriving diff]
type t =  Value of float Map.Stable.V1.M(Int).t [@diff.map] [@@deriving diff]
Comparator witness
The following are not supported by default:
type t = (int, Int.comparator_witness) Set.t [@@deriving diff]
type t = (int, float, Int.comparator_witness) Map.t [@@deriving diff]
But they do work with the set
and map
annotations.
Here are some examples of what will work:
type t = (int, Int.comparator_witness) Set.t [@@deriving diff ~how:"set"]
type t = (int, float, Int.comparator_witness) Map.t [@@deriving diff ~how:"map"]
type t = { value : (int, Int.comparator_witness) Set.t [@diff.set] } [@@deriving diff]
type t =
 Value of (int, float, Int.comparator_witness) Map.t [@diff.map]
[@@deriving diff]
General set/map diff
For the set/map annotations to work in general, they need to be able to determine the element type of the set / the key type of the map.
Default
All of the examples so far use a heuristic to determine the key/elt. By default we assume that the key/elt is X.t
for:
any applicative functor with
X
as an argument (e.g.Set.M(X).t
,Map.Stable.V1.M(X).t
)any type called
X.Set.t
orX.Map.t
any type where
X.t
is the first of two parameters for set or first of three for map (so that it works for(X.t, X.comparator_witness) Set.t
or(X.t, Y.t, X.comparator_witness) Map.t
))
Overrides
If none of the above heuristics apply (or if you want to override the default for whatever reason), you will need to provide the key/elt explicitly.
Any of the following would work:
type t = My_int_set.t [@@deriving diff ~how:"set" ~elt:int]
type t = float My_int_map.t [@@deriving diff ~how:"map" ~key:int]
type t = { value : My_int_set.t [@diff.set (elt : int)] } [@@deriving diff]
type t =  Value of float My_int_map.t [@diff.map (key : int)] [@@deriving diff]
Abstract diffs
One problem that arises from using atomic
, atomic_using_compare
, set
and map
attributes is that your type definition may look a bit polluted, since you need to use the attributes in both the mli and the ml
E.g. the following is rather verbose
module Range : sig
type t =
{ start : Some_type.t [@diff.atomic]
; stop : Some_type.t [@diff.atomic]
}
[@@deriving diff]
end = struct
type t =
{ start : Some_type.t [@diff.atomic]
; stop : Some_type.t [@diff.atomic]
}
[@@deriving diff]
end
If you don't care about exposing what the diff type actually is, you can make the diff abstract.
You can do this either directly by writing:
module Range : sig
type t =
{ start : Some_type.t
; stop : Some_type.t
}
include Diffable.S with type t := t
end
or, equivalently, by using ~how:"abstract"
:
module Range : sig
type t =
{ start : Some_type.t
; stop : Some_type.t
}
[@@deriving diff ~how:"abstract"]
end
Stability
Want to use [@@deriving diff]
with stable types and guarantee that [Diff.t]
is also stable? You can use the [stable_version]
annotation, e.g.
type t = ... [@@deriving diff ~stable_version:1]
The resulting diff type won't change as long as it only references other types that are both stable themselves and use the diff [stable_version]
annotation (or atomic diffs). Note that the compiler does NOT check that you got this right, so we suggest adding a bin digest test for peace of mind.
Only version 1
exists at the moment. There are also currently no plans to change any the types generated by diff, but the stable_version
annotation will protect you in case we ever do choose to make changes.
Polling state RPC
ppx_diff
can be used with Polling_state_rpc
by calling Diffable_polling_state_rpc_response.Polling_state_rpc_response.Make
For example, the following will work:
module Query = Unit
module Response = struct
type t =
{ foo : int
; bar : float
; baz : char
} [@@deriving sexp, bin_io, diff]
end
let polling_state_rpc =
Polling_state_rpc.create
~name:"getresponse"
~version:0
~query_equal:Query.equal
~bin_query:Query.bin_t
(module Diffable_polling_state_rpc_response.Polling_state_rpc_response.Make
(Response))
Dependencies (9)

ppxlib
>= "0.28.0"

dune
>= "3.11.0"

ppxlib_jane
>= "v0.17" & < "v0.18"

ppx_jane
>= "v0.17" & < "v0.18"

ppx_enumerate
>= "v0.17" & < "v0.18"

ppx_compare
>= "v0.17" & < "v0.18"

gel
>= "v0.17" & < "v0.18"

base
>= "v0.17" & < "v0.18"

ocaml
>= "5.1.0"
Dev Dependencies
None
Conflicts
None