package index

  1. Overview
  2. Docs
A platform-agnostic multi-level index for OCaml

Install 

dune-project
 Dependency

github.com Readme Changelog Edit opam file Versions (16)

Authors

  1. C
    Craig Ferguson <craig@tarides.com>
  2. T
    Thomas Gazagnaire <thomas@tarides.com>
  3. I
    Ioana Cristescu <ioana@tarides.com>
  4. C
    Clément Pascutto <clement@tarides.com>

Maintainers

  1. C
    Clement Pascutto

Sources

index-1.3.3.tbz
sha256=6051cf8cad4ab425d8501063110c1fc91fb86fa90426b094dce34cdea5588a91
sha512=8fda03c25fef27d6744dd3aabc5928184cd62f32d0ab80c7b819b07f046988c83cd70527b87ec44d537d4650ea8b1fb949dc10b2e2acf66def219a4bcf3da022

doc/index.unix/Index_unix/Private/Make/index.html

Module Private.MakeSource

Parameters

module K : Index.Key.S
module V : Index.Value.S
module C : Index.Cache.S

Signature

Sourcetype t

The type for indexes.

Sourcetype key = K.t

The type for keys.

Sourcetype value = V.t

The type for values.

Sourcetype cache

The type for caches of index instances.

Sourceval empty_cache : unit -> cache

Construct a new empty cache of index instances.

Sourceval v : ?flush_callback:(unit -> unit) -> ?cache:cache -> ?fresh:bool -> ?readonly:bool -> ?throttle:[ `Overcommit_memory | `Block_writes ] -> log_size:int -> string -> t

The constructor for indexes.

  • parameter flush_callback

    A function to be called before any new bindings are persisted to disk (including both automatic flushing and explicit calls to flush or close).

This can be used to ensure certain pre-conditions are met before bindings are persisted to disk. (For instance, if the index bindings are pointers into another data-structure d, it may be necessary to flush d first to avoid creating dangling pointers.)

  • parameter cache

    used for instance sharing.

  • parameter fresh

    whether an existing index should be overwritten.

  • parameter read_only

    whether read-only mode is enabled for this index.

  • parameter throttle

    the strategy to use when the cache are full and and async in already in progress. Block_writes (the default) blocks any new writes until the merge is completed. Overcommit_memory does not block but continues to fill the (already full) cache.

  • parameter log_size

    the maximum number of bindings in the `log` IO.

Sourceval clear : t -> unit

clear t clears t so that there are no more bindings in it.

Sourceval find : t -> key -> value

find t k is the binding of k in t.

Sourceval mem : t -> key -> bool

mem t k is true iff k is bound in t.

Sourceval replace : ?overcommit:bool -> t -> key -> value -> unit

replace t k v binds k to v in t, replacing any existing binding of k.

If overcommit is true, the operation does not triger a merge, even if the caches are full. By default overcommit is false.

Sourceval filter : t -> ((key * value) -> bool) -> unit

filter t p removes all the bindings (k, v) that do not satisfy p. This operation is costly and blocking.

Sourceval iter : (key -> value -> unit) -> t -> unit

Iterates over the index bindings. Limitations:

  • Order is not specified.
  • In case of recent replacements of existing values (since the last merge), this will hit both the new and old bindings.
  • May not observe recent concurrent updates to the index by other processes.
Sourceval flush : ?no_callback:unit -> ?with_fsync:bool -> t -> unit

Flushes all internal buffers of the IO instances.

  • Passing ~no_callback:() disables calling the flush_callback passed to v.
  • If with_fsync is true, this also flushes the OS caches for each IO instance.
Sourceval close : ?immediately:unit -> t -> unit

Closes all resources used by t, flushing any internal buffers in the instance.

If immediately is passed, this operation will abort any ongoing background processes. This guarantees not to corrupt the store, but may require additional work to be done on the next startup.

Sourceval sync : t -> unit

sync t syncs a read-only index with the files on disk. Raises RW_not_allowed if called by a read-write index.

Sourceval is_merging : t -> bool

is_merging t returns true if t is running a merge. Raises RO_not_allowed if called by a read-only index.

Sourceval merge : t -> unit

merge t forces a merge for t.

If there is no merge running, this operation is non-blocking, i.e. it returns immediately, with the merge running concurrently.

If a merge is running already, this operation blocks until the previous merge is complete. It then launches a merge (which runs concurrently) and returns.

Sourceval try_merge : t -> unit

try_merge is like merge but is a no-op if the number of entries in the write-ahead log is smaller than log_size.

Sourcemodule Checks : sig ... end

Offline fsck-like utility for checking the integrity of Index stores built using this module.

Sourcetype merge_stages = [
  1. | `After
  2. | `After_clear
  3. | `After_first_entry
  4. | `Before
]

Some operations that trigger a merge can have hooks inserted at the following stages:

  • `Before: immediately before merging (while holding the merge lock);
  • `After_clear: immediately after clearing the log, at the end of a merge;
  • `After_first_entry: immediately after adding the first entry in the merge file, if the data file contains at least one entry;
  • `After: immediately after merging (while holding the merge lock).
Sourcetype merge_result = [
  1. | `Completed
  2. | `Aborted
]
Sourcetype 'a async

The type of asynchronous computation.

Sourceval replace' : ?hook:[ `Merge of merge_stages ] Index.Private.Hook.t -> ?overcommit:bool -> t -> key -> value -> merge_result async option

replace' t k v is like replacetkv but returns a promise of a merge result if the replace call triggered one.

Sourceval close' : hook:[ `Abort_signalled ] Index.Private.Hook.t -> ?immediately:unit -> t -> unit

`Abort_signalled: after the cancellation signal has been sent to any concurrent merge operations, but before blocking on those cancellations having completed.

Sourceval clear' : hook:[ `Abort_signalled | `IO_clear ] Index.Private.Hook.t -> t -> unit
Sourceval try_merge_aux : ?hook:merge_stages Index.Private.Hook.t -> ?force:bool -> t -> merge_result async

try_merge_aux t tries to merge t. If force is false (the default), a merge is performed only if there is more entries in the write-ahead log than the configured limits. If force is set, the merge is performed no matter what.

Sourceval await : 'a async -> ('a, [ `Async_exn of exn ]) result

Wait for an asynchronous computation to finish.

Sourceval replace_with_timer : ?sampling_interval:int -> t -> key -> value -> unit

Time replace operations. The reported time is an average on an number of consecutive operations, which can be specified by sampling_interval. If sampling_interval is not set, no operation is timed.

Sourceval sync' : ?hook: [ `Before_offset_read | `After_offset_read | `Reload_log | `Reload_log_async ] Index.Private.Hook.t -> t -> unit

Hooks:

  • `Before_offset_read: before reading the generation number and the offset.
  • `After_offset_read: after reading the generation number and offset.
Sourceval log : t -> (key * value) list option
Sourceval log_async : t -> (key * value) list option