package octez-libs

  1. Overview
  2. Docs
package octez-libs
Legend:
Library
Module
Module type
Parameter
Class
Class type

Wrapper around Tezt to declare test dependencies.

include module type of Tezt

Tezt (pronounced "tezty", as in "tasty" with a "z") is a test framework for OCaml. It is well suited for writing and executing unit, integration and regression tests. It integrates well with continuous integration (CI).

Tezt provides a function (Test.register) to register tests. Tests have a title, some tags, and an implementation (a function). Titles and tags can be used on the command-line to select which tests to run. The name of the file in which the test was registered can also be used to do so. The implementation of the test is ran if the test is selected. If this implementation does not raise an exception (for instance by calling Test.fail), the test is considered to be successful. Along with modules Base, which is supposed to be opened a la Pervasives and which provides a few useful generic functions, and Check, which provides ways to perform assertions with nice error messages, the result is a framework that is suitable for unit tests.

Tests can be ran in a CI, optionally producing JUnit reports. Tezt can automatically compute a partition of the set of tests where subsets are balanced to take roughly the same amount of time to run. The intent is that each of those subset can be one CI job, resulting in automatic balanced parallelisation.

Specific features supporting integration tests include:

  • running external processes;
  • invoking distant runners through SSH;
  • decoding JSON values (e.g. to test REST APIs);
  • cleaning up temporary files automatically.

Specific features supporting regression tests include:

  • capturing output from local or external processes;
  • applying regular-expression-based substitutions to those outputs;
  • comparing with previous captured outputs.

Tezt provides a flexible user interface:

  • using colors in logs (e.g. to distinguish external processes or to make error messages easy to see);
  • adjusting automatically the verbosity of logs around errors;
  • making it easy to copy-paste the invoked commands to reproduce errors;
  • giving extensive information to precisely locate errors;
  • supporting flaky (randomly failing) tests, by running them repeatedly.

To get started, register tests with Test.register, then call the main function Test.run. Tests are run from the command-line, for instance with dune runtest or dune exec. The latter gives access to a large list of command-line options to select which tests to run and how. Execute your program with --help to get the list. See also the Tezt mini-tutorial.

Backends

Tezt executables can be compiled to bytecode, native code, or JavaScript. This is reflected by three Dune libraries:

  • code which is common to all backends is provided by tezt.core;
  • the bytecode and native code backends are provided by tezt;
  • the JavaScript backend is provided by tezt.js.

Function Test.run is only available in tezt and tezt.js. In other words, to actually run the tests you have to decide whether to use the JavaScript backend or not.

The difference between tezt and tezt.js is that tezt.js does not provide:

  • the Process module;
  • the Temp module;
  • the Runner module. So the JavaScript backend is not well suited for integration tests and regression tests. But it does support unit tests.

If you want to run your tests on multiple backends you have to write two executables: one linked with tezt and one linked with tezt.js. To share the code of your tests, you can write your calls to Test.register in a library that depends on tezt.core. Here is an example of dune files:

; Dune file for the library that calls [Test.register].
(library
 (name tezts)
 (libraries tezt.core)
 (library_flags (:standard -linkall))
 (flags (:standard) -open Tezt_core -open Tezt_core.Base))

; Dune file for the executable used to run tests in native or bytecode mode.
(executable (name main) (libraries tezts tezt))

; Dune file for the executable used to run tests using nodejs.
(executable (name main_js) (modes js) (libraries tezts tezt.js))

Modules

module Background = Tezt_core.Background

Support for running promises in the background.

module Base = Tezt_core.Base

Base primitives useful in writing tests.

module Check = Tezt_core.Check

Support for expressing assertions.

module Cli = Tezt_core.Cli

Command-line interface options.

module Diff = Tezt_core.Diff

Compute the difference between two sequences of items.

module JSON = JSON

JSON handling (encoding/decoding, accessors, error management, etc).

module Log = Tezt_core.Log

Functions for logging messages.

module Process = Tezt.Process

Managing external processes.

module Process_hooks = Tezt_core.Process_hooks

Process hooks, in particular for use with regression tests.

module Runner = Tezt.Runner

Runner specifications for processes spawned remotely using SSH.

module Temp = Tezt.Temp

Temporary file management.

module Uses : sig ... end

Test dependencies.

module Test : sig ... end
module Regression : sig ... end
type error_mode =
  1. | Ignore
  2. | Warn
  3. | Fail

Error modes.

  • Ignore: do not warn, do not fail.
  • Warn: warn, but do not fail.
  • Fail: output an error and fail the test.
val error_mode_for_missing_use : error_mode ref

What to do if a test uses something without having it declared in its ~uses.

Recommended setting for tests that are intended to run in the CI is Fail, which is the default.

val error_mode_for_useless_use : error_mode ref

What to do if a test declares something in its ~uses and does not use it.

Recommended setting for tests that are intended to run in the CI is Warn, which is the default.

Using Fail is possible but:

  • if a test is non-deterministic and only sometimes uses something, one needs to make sure Uses.path is always called anyway;
  • if a test actually uses something but not through Uses.path, the wrapper will not be able to detect it, so one will have to call Uses.path just to suppress the error.
val error_mode_for_non_existing_use : error_mode ref

What to do if a test declares a ~uses with a path that does not exist.

Recommended setting is Fail, which is the default.

OCaml

Innovation. Community. Security.

GitHub Discord Twitter Peertube RSS
About
Changelog Releases Industrial Users Academic Users Why OCaml
Ecosystem
Packages Community Events OCaml Planet Jobs
Resources
Install OCaml Get Started Platform Tools Language Manual Standard Library API Books Exercises Papers OCaml Playground Logo
Policies
Carbon Footprint Governance Privacy Code of Conduct