package plebeia

  1. Overview
  2. Docs
package plebeia
Legend:
Page
Library
Module
Module type
Parameter
Class
Class type
Source

Module Plebeia.StorageSource

Generic cell based storage

Sourcetype mode =
  1. | Reader
  2. | Writer

Mode to access the storage.

  • Reader: Read only mode. Multiple readers can exist.
  • Writer: Read write mode. Only 1 writer can exist for a storage.
Sourcetype config = {
  1. head_string : string;
    (*

    exactly 20 bytes

    *)
  2. version : int;
  3. bytes_per_cell : int;
  4. max_index : Index.t;
    (*

    Maximal possible index. Requesting more indicies than max_index fails.

    *)
}
Sourceval pp_config : Format.formatter -> config -> unit
Sourcetype storage
Sourcetype t = storage

The type

Sourcetype Error.t +=
  1. | No_such_file of string
  2. | Not_reader
  3. | Not_writer
  4. | Index_overflow of t
    (*

    Error when Index.t exceeds the max_index or overflows

    *)
  5. | Config_mismatch of {
    1. actual : config;
    2. expected : config;
    }
Sourcetype writer_key = private string

Key required to enable fast process communcation between the writer process and readers

Sourceval writer_key_encoding : writer_key Data_encoding.t
Sourceval make_new_writer_key : t list -> writer_key Lwt.t

Make a writer key which is new to all the given ts

2 Open, close, and commit

Sourceval create : ?length:Index.t -> ?resize_step_bytes:int -> config:config -> key:writer_key -> string -> t Lwt.t

Create a new storage

* length: The initial size of the storage in cells * resize_step: How many cells allocated for each resize * config : Configuration * key : The writer key * string : The path name

Sourceval open_existing_for_read : string -> (config * t, Error.t) result Lwt.t

Open an existing storage

Sourceval open_for_write : ?resize_step_bytes:int -> config:config -> key:writer_key -> string -> t Lwt.t

Open a file for writing. If the file does not exist, create it.

Sourceval null : t

Dummy storage

Sourceval is_null : t -> bool

is_null t returns true if t is null

Sourceval truncate : ?length:int -> t -> unit Lwt.t

Truncate the data file and reinitialize it. All the contents are lost. length is the initial reserved size of the reinitialized file.

Sourceval update_reader : t -> unit Lwt.t

For reader to update the storage to catch up the update by the writer process. For Writer and Private, it does nothing.

Sourceval close : t -> unit Lwt.t

Close the storage. For the Writer, it calls commit before closing.

Sourceval commit : t -> unit Lwt.t

Write the current state of the storage. Once called, the state is shared with the reader processes with a writer key.

Note that commit does NOT assure the persistence of the data after a system crash. Use flush to make it persistent even after a crash.

Sourceval flush : t -> unit Lwt.t

Flushes all the added data to the disk. Once called, the state is shared with all the readers, even without writer key.

The data added to the storage before the call of flush are persisted even after a system crash. Data added after the last call of flush may be lost after a system crash.

flush may trigger huge disk writes. Too frequent call of flush may degrates the performance.

Sourceval enable_process_sync : t -> writer_key -> (unit, Error.t) result Lwt.t

For readers. Enable the process synchronization between the writer process. Once enabled, update_reader checks the header for the process sync instead of the disk sync, and they do not need to wait a call of flush in the writer.

A reader process obtains the latest state of the storage calling update_reader function. By default, it can detect the updates only after the writer calls flush. flush synchronizes the disk and therefore is a heavy operation.

enable_process_sync changes this detection of a reader process from via file to via shared memory, so that it can detect the updates without writer's call of flush.

Typical usage:

1. The writer process creates a new writer key. 2. The writer opens storages (context and commit_tree) with the writer key. (Vc.create and Vc.open_for_write do this internally.) 3. The writer sends the writer key to reader processes. 4. The reader processes call enable_process_sync to enable the faster process synchronization.

The writer and a reader process must run on the same machine to use enable_process_sync correcly. (In any way, it is not recommended to use Plebeia via network file systems.)

Call of this function with an invalid writer key returns an Error. Call of this function in the writer process returns an Error.

2 Accessor

Sourceval filename : t -> string

Return the file name

Sourceval mode : t -> mode

Return the opening mode

Sourceval get_last_root_index : t -> Index.t option
Sourceval get_current_length : t -> Index.t

Get the status of the storage

For Reader, it only returns the lastest information it knows in memory. Writer may already update this information on the disk.

Sourceval size : t -> int64

Used cells in bytes

Sourceval version : t -> int

Storage version

Sourceval start_index : t -> Index.t
Sourceval writer_key : t -> writer_key option
Sourceval override_version : t -> int -> unit

Override the header version of the file

Sourceval set_last_root_index : t -> Index.t option -> unit

Set the last index of root hash. Note that the data are only saved to the file when Header.commit is called.

2 Read and write

Sourceval get_cell : t -> Index.t -> Mmap.Buffer.t

Get the content of the cell specified by the index

Sourceval get_cell2 : t -> Index.t -> Mmap.Buffer.t

make a 2-cell-wide writable buffer from the beginning of the cell of the given index

Sourceval get_bytes : t -> Index.t -> int -> Mmap.Buffer.t

Get the contiguous bytes from the head of the index

Sourceval new_index : t -> (Index.t, Error.t) result

Allocate a cell and returns its index

Sourceval new_indices : t -> int -> (Index.t, Error.t) result

Allocate cells and return the first index

Sourcemodule Chunk : sig ... end

Bigger data than a cell

2 Statistics

Sourcemodule Stat : sig ... end
Sourceval stat : t -> Stat.t

2 Internal use

Sourcemodule Internal : sig ... end