package fehu

  1. Overview
  2. Docs
Reinforcement learning for OCaml

Install 

dune-project
 Dependency

github.com Readme Changelog Edit opam file Versions (3)

Authors

  1. Thibaut Mattio

Maintainers

  1. Thibaut Mattio

Sources

raven-1.0.0.alpha3.tbz
sha256=96d35ce03dfbebd2313657273e24c2e2d20f9e6c7825b8518b69bd1d6ed5870f
sha512=90c5053731d4108f37c19430e45456063e872b04b8a1bbad064c356e1b18e69222de8bfcf4ec14757e71f18164ec6e4630ba770dbcb1291665de5418827d1465

doc/fehu/Fehu/Space/index.html

Module Fehu.SpaceSource

Observation and action spaces.

Spaces define valid observations and actions for reinforcement learning environments. They specify shapes, constraints, and provide methods to validate, sample, and serialize values.

Each space type corresponds to a common RL scenario: discrete choices, continuous vectors, binary indicators, composite structures, and variable-length sequences.

Structural description

Sourcetype spec =
  1. | Discrete of {
    1. start : int;
    2. n : int;
    }
    (*

    Integer choices in [start; start + n - 1].

    *)
  2. | Box of {
    1. low : float array;
    2. high : float array;
    }
    (*

    Continuous vector bounded per dimension.

    *)
  3. | Multi_binary of {
    1. n : int;
    }
    (*

    Binary vector of length n.

    *)
  4. | Multi_discrete of {
    1. nvec : int array;
    }
    (*

    Multiple discrete axes with per-axis cardinalities.

    *)
  5. | Tuple of spec list
    (*

    Fixed-length heterogeneous sequence.

    *)
  6. | Dict of (string * spec) list
    (*

    Named fields with different types.

    *)
  7. | Sequence of {
    1. min_length : int;
    2. max_length : int option;
    3. base : spec;
    }
    (*

    Variable-length homogeneous sequence.

    *)
  8. | Text of {
    1. charset : string;
    2. max_length : int;
    }
    (*

    Character strings from a fixed alphabet.

    *)

Structural description of a space. Two spaces are compatible when their specs are equal.

Sourceval equal_spec : spec -> spec -> bool

equal_spec a b is true iff a and b describe structurally identical spaces.

Spaces

Sourcetype 'a t

The type for spaces over values of type 'a. A space is self-contained: all bounds, constraints, and serialization logic are stored in the value itself.

Sourcetype packed =
  1. | Pack : 'a t -> packed
    (*

    Type-erased space for heterogeneous collections.

    *)

Operations

Sourceval spec : 'a t -> spec

spec s is the structural description of s.

Sourceval shape : 'a t -> int array option

shape s is the dimensionality of s, if defined. None for scalar or variable-length spaces.

Sourceval contains : 'a t -> 'a -> bool

contains s v is true iff v is valid in s.

Sourceval sample : 'a t -> 'a

sample s is a uniformly sampled value from s.

Random keys are drawn from the implicit RNG scope.

Sourceval pack : 'a t -> 'a -> Value.t

pack s v is v converted to the universal Value.t representation.

Sourceval unpack : 'a t -> Value.t -> ('a, string) result

unpack s v is Ok x if v can be converted to a valid element of s, or Error msg otherwise.

Sourceval boundary_values : 'a t -> Value.t list

boundary_values s is a list of representative edge-case values for s. Includes lower/upper bounds or canonical sentinels when known. The empty list when no boundary values apply.

Space types

Sourcemodule Discrete : sig ... end
Sourcemodule Box : sig ... end
Sourcemodule Multi_binary : sig ... end
Sourcemodule Multi_discrete : sig ... end
Sourcemodule Tuple : sig ... end
Sourcemodule Dict : sig ... end
Sourcemodule Sequence : sig ... end
Sourcemodule Text : sig ... end