package modelkit

  1. Overview
  2. Docs
Portable classical machine learning workflows for OCaml

Install 

dune-project
 Dependency

github.com Readme Changelog Edit opam file Versions (4)

Authors

  1. A
    Asara

Maintainers

  1. A
    Asara developers <devs@asara.io>

Sources

modelkit-0.5.0.tbz
sha256=1fe8fa7c7f904dd098a21a2ca30fd69230750b8cf8aa2ae481b97a16531b47b4
sha512=c946cd1ac014726f4d21791e14d806a205680e6edfa2f80ed8d3680f24f48ca3c5a2f89d5afa68c11eac4053ead448b6381fb3d06dc8ff129097af6c69b262fb

doc/modelkit/Modelkit/Cross_validation/index.html

Module Modelkit.Cross_validation

Deterministic cross-validation over immutable pipelines.

Split membership is planned from seed before fitting. Each fold receives a child seed derived from its logical index and fit_seed, which defaults to seed; meta-estimators can therefore vary fit randomness without changing split membership. Training and test partitions are materialized explicitly, so every preprocessing stage is fitted only from training rows. Fold and scorer arrays retain splitter and caller order.

fit_time and score_time are portable process CPU seconds measured with Sys.time; intervals can overlap under parallel execution, so their sum is not elapsed wall time. Abort returns the lowest-index failure; Record retains typed failures in the report and continues with later folds. Models and indices are retained only when requested. execution defaults to Execution.sequential; every backend must return outputs and the lowest-index failure in logical fold order.

Binary_classification scores with Binary_classification_scorer and requires exactly two declared classes for probability scorers. Multiclass_classification scores with Multiclass_classification_scorer, accepts any pipeline that declares two or more distinct classes, and passes the full probability matrix in declared class order to log-loss scorers. Both request predicted labels and probabilities only when a scorer needs them; a pipeline without the requested capability records a typed prediction failure for the fold.

Out-of-fold prediction requires test folds to contain every source row exactly once and restores successful responses to source row order. Classification callers select labels or probabilities. Probabilities use the complete dataset's ascending class order; missing fitted-fold classes receive zero columns, while unknown or duplicate classes are typed compatibility failures.

metadata defaults to Metadata.of_dataset: dataset weights and groups are selected with each fold's exact training/test row views, including inference. An explicit carrier replaces that default without merging; its fields must match the complete dataset's row count. Splitters still use dataset groups and scorers still use dataset weights. Search refit receives the complete carrier. These inputs are never inferred from a previously fitted model.

A supplied callback receives evaluation lifecycle events and is delivered to nested consumers only when their per-method request opts in. Fold events are buffered and dispatched on the caller domain in logical order; see Callback for bounds, cancellation, and failure semantics.

Each task-specific cross_validate accepts built-in scorers plus optional first-class custom_scorers. Names must be nonblank and unique across both arrays. A custom scorer's Capability.prediction is checked against the task before fitting, and its declared sample-weight support is enforced while scoring.

type failure_policy =
  1. | Abort
  2. | Record
type partition =
  1. | Train
  2. | Test
type failure_phase =
  1. | Materialization
  2. | Fitting
  3. | Prediction of partition
  4. | Scoring of {
    1. partition : partition;
    2. scorer : string;
    }
type failure = {
  1. phase : failure_phase;
  2. error : Error.t;
}
type score = {
  1. name : string;
  2. train_score : (float, Error.t) result option;
  3. test_score : (float, Error.t) result option;
}
type 'model fold = {
  1. fold_index : int;
  2. fit_time : float;
  3. score_time : float;
  4. scores : score array;
  5. model : 'model option;
  6. train_indices : int array option;
  7. test_indices : int array option;
  8. failures : failure array;
}
type 'model report
type classification_response =
  1. | Labels
  2. | Probabilities
type 'prediction prediction_fold = {
  1. prediction_fold_index : int;
  2. prediction_fit_time : float;
  3. predict_time : float;
  4. prediction_test_indices : int array;
  5. prediction_result : ('prediction, failure) result;
}
type 'prediction prediction_report
type 'target splitter
val target_independent_splitter : (module SPLITTER with type rng = Rng.t and type t = 'specification and type target = unit) -> 'specification -> 'target splitter

Adapts a target-independent splitter such as K_fold.

val target_aware_splitter : (module SPLITTER with type rng = Rng.t and type t = 'specification and type target = 'target) -> 'specification -> 'target splitter

Adapts a target-aware splitter such as Stratified_k_fold.

val folds : 'model report -> 'model fold array
val successful_fold_count : 'model report -> int
val prediction_folds : 'prediction prediction_report -> 'prediction prediction_fold array
val successful_prediction_fold_count : 'prediction prediction_report -> int
val out_of_fold_predictions : 'prediction prediction_report -> ('prediction, failure array) result

Returns predictions restored to source row order. Under Record, any failed folds make the assembled value unavailable; their successful peers remain inspectable through prediction_folds.

module Regression : sig ... end
module Binary_classification : sig ... end
module Multiclass_classification : sig ... end