package fehu

  1. Overview
  2. Docs

Module Buffer.ReplaySource

Replay buffer for off-policy algorithms.

Implements a fixed-capacity circular buffer storing complete transitions. When capacity is reached, oldest transitions are overwritten. Supports uniform random sampling for breaking temporal correlations in training data.

All transitions are stored contiguously in memory. Observation and action arrays are lazily initialized on the first call to add.

Sourcetype ('obs, 'act) t

Replay buffer storing transitions of observations and actions.

Sourceval create : capacity:int -> ('obs, 'act) t

create ~capacity creates an empty replay buffer.

The buffer stores up to capacity transitions. When full, adding new transitions overwrites the oldest ones in circular fashion.

Sourceval add : ('obs, 'act) t -> ('obs, 'act) transition -> unit

add buffer transition stores a transition in the buffer.

Appends transition to the buffer, overwriting the oldest transition if at capacity. The first call initializes internal storage arrays based on the observation and action types.

Time complexity: O(1).

Sourceval sample : ('obs, 'act) t -> rng:Rune.Rng.key -> batch_size:int -> ('obs, 'act) transition array

sample buffer ~rng ~batch_size returns uniformly sampled transitions.

Samples batch_size transitions uniformly at random from the buffer. If batch_size exceeds the current buffer size, samples min(batch_size, size) transitions instead. Sampling is with replacement.

Time complexity: O(batch_size).

Sourceval size : ('obs, 'act) t -> int

size buffer returns the current number of transitions stored.

Returns values between 0 and capacity.

Sourceval is_full : ('obs, 'act) t -> bool

is_full buffer checks whether the buffer has reached capacity.

Sourceval clear : ('obs, 'act) t -> unit

clear buffer removes all transitions from the buffer.

Resets size to 0 and write position to 0. Does not deallocate internal storage arrays.