package git-kv

  1. Overview
  2. Docs

A Git key-value store.

This module implements the ability to manipulate a Git repository as a Key-Value store. It allows you to create a local (in-memory) Git repository that can come from either:

  • a remote Git repository (with connect)
  • a state serialized by the to_octets function

The first case is interesting if you want to be synchronised with the remote repository. The second case can be interesting if we don't want to create a connection at the beginning and desynchronisation between our local and remote repositories is not a problem.

In the second case, the synchronisation can be done later with pull.

Pushing and synchronisation.

The user can modify the repository (add files, remove files, etc.). Each change produces a commit and after each change we try to transfer them to the remote Git repository. If you want to make multiple changes but contain them in a single commit and only transfer those changes once, you should use the Make.change_and_push function.

Serialization of the Git repository.

Finally, the KV-store tries to keep the minimal set of commits required between you and the remote repository. Only unpushed changes are kept by the KV-store. However, if these changes are not pushed, they will be stored into the final state produced by to_octets. In other words, the more changes you make out of sync with the remote repository (without pushing them), the bigger the state serialization will be.

include Mirage_kv.RW with type write_error = [ `Msg of string | `Hash_not_found of Digestif.SHA1.t | `Reference_not_found of Git.Reference.t | Mirage_kv.write_error ] and type error = [ `Msg of string | Mirage_kv.error ]

Read-write Stores

The functions set and remove will cause a flush in the underlying storage layer every time, which can degrade performance.

include Mirage_kv.RO with type error = [ `Msg of string | Mirage_kv.error ]

Read-only key-value stores

type nonrec error = [
  1. | `Msg of string
  2. | Mirage_kv.error
]

The type for errors.

val pp_error : error Fmt.t

pp_error is the pretty-printer for errors.

type t

The type representing the internal state of the key-value store.

val disconnect : t -> unit Lwt.t

Disconnect from the key-value store. While this might take some time to complete, it can never result in an error.

type key = Mirage_kv.Key.t

The type for keys.

val exists : t -> key -> ([ `Value | `Dictionary ] option, error) result Lwt.t

exists t k is Some `Value if k is bound to a value in t, Some `Dictionary if k is a prefix of a valid key in t and None if no key with that prefix exists in t.

exists answers two questions: does the key exist and is it referring to a value or a dictionary.

An error occurs when the underlying storage layer fails.

val get : t -> key -> (string, error) result Lwt.t

get t k is the value bound to k in t.

The result is Error (`Value_expected k) if k refers to a dictionary in t.

val get_partial : t -> key -> offset:Optint.Int63.t -> length:int -> (string, error) result Lwt.t

get_partial t k ~offset ~length is the length bytes wide value bound at offset of k in t.

If the size of k is less than offset, get_partial returns an empty string. If the size of k is less than offset+length, get_partial returns a short string. The result is Error (`Value_expected k) if k refers to a dictionary in t.

val list : t -> key -> ((key * [ `Value | `Dictionary ]) list, error) result Lwt.t

list t k is the list of entries and their types in the dictionary referenced by k in t. The returned keys are all absolute (i.e. Key.add k entry).

The result is Error (`Dictionary_expected k) if k refers to a value in t.

val last_modified : t -> key -> (Ptime.t, error) result Lwt.t

last_modified t k is the last time the value bound to k in t has been modified.

When the value bound to k is a dictionary, the implementation is free to decide how to compute a last modified timestamp, or return Error (`Value_expected _).

val digest : t -> key -> (string, error) result Lwt.t

digest t k is the unique digest of the value bound to k in t.

When the value bound to k is a dictionary, the implementation is allowed to return Error (`Value_expected _). Otherwise, the digest is a unique and deterministic digest of its entries.

val size : t -> key -> (Optint.Int63.t, error) result Lwt.t

size t k is the size of k in t.

type nonrec write_error = [
  1. | `Msg of string
  2. | `Hash_not_found of Digestif.SHA1.t
  3. | `Reference_not_found of Git.Reference.t
  4. | Mirage_kv.write_error
]

The type for write errors.

val pp_write_error : write_error Fmt.t

The pretty-printer for pp_write_error.

val allocate : t -> key -> ?last_modified:Ptime.t -> Optint.Int63.t -> (unit, write_error) result Lwt.t

allocate t key ~last_modified size allocates space for key in t with the provided size and last_modified. This is useful for e.g. append-only backends that could still use set_partial. The data will be filled with 0. If key already exists, Error (`Already_present key) is returned. If there's not enough space, Error `No_space is returned.

val set : t -> key -> string -> (unit, write_error) result Lwt.t

set t k v replaces the binding k -> v in t.

Durability is guaranteed.

val set_partial : t -> key -> offset:Optint.Int63.t -> string -> (unit, write_error) result Lwt.t

set_partial t k offset v attempts to write v at offset in the value bound to k in t. If k contains directories that do not exist, set_partial will attempt to create them. If the size of k is less than offset, set_partial appends v at the end of k. If the size of k is greater than offset+length of v, set_partial leaves the last bytes of k unchanged.

The result is Error (`Value_expected k) if k refers to a dictionary in t.

val remove : t -> key -> (unit, write_error) result Lwt.t

remove t k removes any binding of k in t. If k was bound to a dictionary, the full dictionary will be removed.

Durability is guaranteed.

val rename : t -> source:key -> dest:key -> (unit, write_error) result Lwt.t

rename t source dest rename source to dest in t. If source and dest are both bound to values in t, dest is removed and the binding of source is moved to dest. If dest is bound to a dictionary in t, source is moved inside dest. If source is bound to a dictionary, the full dictionary is moved.

The result is Error (`Not_found source) if source does not exists in t. The result is Error (`Value_expected source) if source is bound to a dictionary in t and dest is bound to a value in t. The result id Error (`Rename_source_prefix (source, dest)) if source is a prefix of dest, and source is a directory.

val connect : Mimic.ctx -> string -> t Lwt.t

connect ctx remote creates a new Git store which synchronises with remote via protocols available into the given ctx.

  • raises [Invalid_argument

    _] if we can not initialize the store, or if we can not fetch the given remote.

val branch : t -> Git.Reference.t

branch t returns the branch used by the given t.

val commit : t -> [ `Clean of Digestif.SHA1.t | `Dirty of Digestif.SHA1.t ] option

commit t returns the commit used by the given t. The commit is either marked `Dirty _ if we're inside a change_and_push or `Clean _ otherwise.

val to_octets : ?level:int -> t -> string Lwt_stream.t

to_octets ?level store returns a serialized version of the given store. level is the zlib level compression used for Git object (between 0 and 9 including), defaults to 4.

val of_octets : Mimic.ctx -> remote:string -> string Lwt_stream.t -> (t, [> `Msg of string ]) result Lwt.t

of_octets ctx ~remote contents tries to re-create a t from its serialized version contents. This function does not do I/O and the returned t can be out of sync with the given remote. We advise to call pull to be in-sync with remote.

type change = [
  1. | `Add of Mirage_kv.Key.t
  2. | `Remove of Mirage_kv.Key.t
  3. | `Change of Mirage_kv.Key.t
]
val pull : t -> (change list, [> `Msg of string ]) result Lwt.t

pull store tries to synchronise the remote Git repository with your local store Git repository. It returns a list of changes between the old state of your store and what you have remotely.

val change_and_push : t -> ?author:string -> ?author_email:string -> ?message:string -> (t -> 'a Lwt.t) -> ('a, [> `Msg of string ]) result Lwt.t

change_and_push store ~author ~author_email ~message f applies the changes of f to store, and creates a commit using author, author_email, and message (committer will be the same as author), and pushes that commit to the remote.

OCaml

Innovation. Community. Security.