package plebeia

  1. Overview
  2. Docs
package plebeia
Legend:
Library
Module
Module type
Parameter
Class
Class type

Generic cell based storage

type mode =
  1. | Reader
  2. | Writer

Mode to access the storage.

  • Reader: Read only mode. Multiple readers can exist.
  • Writer: Read write mode. Only 1 writer can exist for a storage.
type config = {
  1. head_string : string;
    (*

    exactly 20 bytes

    *)
  2. version : int;
  3. bytes_per_cell : int;
  4. max_index : Index.t;
    (*

    Maximal possible index. Requesting more indicies than max_index fails.

    *)
}
val pp_config : Format.formatter -> config -> unit
type storage
type t = storage

The type

type Error.t +=
  1. | No_such_file of string
  2. | Not_reader
  3. | Not_writer
  4. | Index_overflow of t
    (*

    Error when Index.t exceeds the max_index or overflows

    *)
  5. | Config_mismatch of {
    1. actual : config;
    2. expected : config;
    }
type writer_key = private string

Key required to enable fast process communcation between the writer process and readers

val writer_key_encoding : writer_key Data_encoding.t
val make_new_writer_key : t list -> writer_key Lwt.t

Make a writer key which is new to all the given ts

2 Open, close, and commit

val create : ?length:Index.t -> ?resize_step_bytes:int -> config:config -> key:writer_key -> string -> t Lwt.t

Create a new storage

* length: The initial size of the storage in cells * resize_step: How many cells allocated for each resize * config : Configuration * key : The writer key * string : The path name

val open_existing_for_read : string -> (config * t, Error.t) result Lwt.t

Open an existing storage

val open_for_write : ?resize_step_bytes:int -> config:config -> key:writer_key -> string -> t Lwt.t

Open a file for writing. If the file does not exist, create it.

val null : t

Dummy storage

val is_null : t -> bool

is_null t returns true if t is null

val truncate : ?length:int -> t -> unit Lwt.t

Truncate the data file and reinitialize it. All the contents are lost. length is the initial reserved size of the reinitialized file.

val update_reader : t -> unit Lwt.t

For reader to update the storage to catch up the update by the writer process. For Writer and Private, it does nothing.

val close : t -> unit Lwt.t

Close the storage. For the Writer, it calls commit before closing.

val commit : t -> unit Lwt.t

Write the current state of the storage. Once called, the state is shared with the reader processes with a writer key.

Note that commit does NOT assure the persistence of the data after a system crash. Use flush to make it persistent even after a crash.

val flush : t -> unit Lwt.t

Flushes all the added data to the disk. Once called, the state is shared with all the readers, even without writer key.

The data added to the storage before the call of flush are persisted even after a system crash. Data added after the last call of flush may be lost after a system crash.

flush may trigger huge disk writes. Too frequent call of flush may degrates the performance.

val enable_process_sync : t -> writer_key -> (unit, Error.t) result Lwt.t

For readers. Enable the process synchronization between the writer process. Once enabled, update_reader checks the header for the process sync instead of the disk sync, and they do not need to wait a call of flush in the writer.

A reader process obtains the latest state of the storage calling update_reader function. By default, it can detect the updates only after the writer calls flush. flush synchronizes the disk and therefore is a heavy operation.

enable_process_sync changes this detection of a reader process from via file to via shared memory, so that it can detect the updates without writer's call of flush.

Typical usage:

1. The writer process creates a new writer key. 2. The writer opens storages (context and commit_tree) with the writer key. (Vc.create and Vc.open_for_write do this internally.) 3. The writer sends the writer key to reader processes. 4. The reader processes call enable_process_sync to enable the faster process synchronization.

The writer and a reader process must run on the same machine to use enable_process_sync correcly. (In any way, it is not recommended to use Plebeia via network file systems.)

Call of this function with an invalid writer key returns an Error. Call of this function in the writer process returns an Error.

2 Accessor

val filename : t -> string

Return the file name

val mode : t -> mode

Return the opening mode

val get_last_root_index : t -> Index.t option
val get_current_length : t -> Index.t

Get the status of the storage

For Reader, it only returns the lastest information it knows in memory. Writer may already update this information on the disk.

val size : t -> int64

Used cells in bytes

val version : t -> int

Storage version

val start_index : t -> Index.t
val writer_key : t -> writer_key option
val override_version : t -> int -> unit

Override the header version of the file

val set_last_root_index : t -> Index.t option -> unit

Set the last index of root hash. Note that the data are only saved to the file when Header.commit is called.

2 Read and write

val get_cell : t -> Index.t -> Mmap.Buffer.t

Get the content of the cell specified by the index

val get_cell2 : t -> Index.t -> Mmap.Buffer.t

make a 2-cell-wide writable buffer from the beginning of the cell of the given index

val get_bytes : t -> Index.t -> int -> Mmap.Buffer.t

Get the contiguous bytes from the head of the index

val new_index : t -> (Index.t, Error.t) result

Allocate a cell and returns its index

val new_indices : t -> int -> (Index.t, Error.t) result

Allocate cells and return the first index

module Chunk : sig ... end

Bigger data than a cell

2 Statistics

module Stat : sig ... end
val stat : t -> Stat.t

2 Internal use

module Internal : sig ... end