package alcotest

Build not yet checked
  1. Overview
  2. Docs
Alcotest is a lightweight and colourful test framework

Install

dune-project
 Dependency

Authors

Maintainers

Sources

alcotest-1.10.1.tbz
sha256=62c095deda5cdd66c30684f7b1f25b5775d1d3d3ded927c7b893b4a6834ddb47
sha512=6f779fa96faf96b5b462ddc428aeaaa73a6be96771dbbad6a05b7bb00dae246648891c211e26174729c1f40128410ad96f2512101ae8b7d61f968ee61534b3d0

doc/alcotest/Alcotest/index.html

Module AlcotestSource

A lightweight and colourful test framework.

Alcotest provides a simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

From these descriptions, Alcotest builds a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect).

Release 1.10.1

include Alcotest_engine.V1.Cli.S with type return = unit
Sourcetype return = unit

The return type of each test case run by Alcotest. For the standard Alcotest module, return = unit. The concurrent backends Alcotest_lwt and Alcotest_async set return = unit Lwt.t and return = unit Async.Deferred.t respectively.

Sourcetype tag = [
  1. | `Quick
  2. | `Slow
  3. | `Always
  4. | `Tag of string
  5. | `And of tag list
  6. | `Or of tag list
  7. | `Not of tag
]

When a test runs. The command line sets up a context, a set of tags that starts with quick and slow: -q removes slow, and --tags adds or removes tags. `Quick, `Slow and `Tag s hold when quick, slow and s are in the context, `And, `Or and `Not combine them, and `Always always holds.

Sourcetype speed_level = tag

Deprecated alias of tag.

Sourcemodule Tag : sig ... end

Combinators for tag, as in Tag.(slow && v "release").

Sourcetype 'a test_case = string * tag * ('a -> return)

A test case is a UTF-8 encoded documentation string, when it runs and a function to execute. Typically, the testing function calls the helper functions provided below (such as check and fail).

Sourceexception Test_error

The exception return by run in case of errors.

Sourceval test_case : string -> tag -> ('a -> return) -> 'a test_case

test_case n t f is the test case n using the function f, which runs when t holds.

Sourceval test : ?tags:tag -> string -> ('a -> return) -> 'a test_case

test name f is the test case name using the function f. It only runs when tags hold, which defaults to `Always.

Sourceval group : ?tags:tag -> string -> 'a test_case list -> 'a test_case

group name cases is a test case that stands for cases, which run in place under the test name test.name, so that they can be selected together. Groups nest. name must not be empty. cases only run when tags hold as well as their own, and tags defaults to `Always.

A group is recognised by physical equality on the returned tuple, so it can appear several times and under different parents. A new tuple built from it is a plain test case, whose function runs all of cases. Functions on a list of test cases, like List.filter, see the group and not its cases.

The test subcommand matches names as a regular expression, so test top also selects top.name. Use test '^top$' to select only the cases outside of groups. A test called top.name next to a group name in top is a duplicate test path if their indices overlap.

Sourcetype 'a test = string * 'a test_case list

A test is a UTF-8 encoded name and a list of test cases. The name can be used for filtering which tests to run on the CLI.

Sourceval suite : string -> 'a test_case list -> 'a test

suite name cases is the test name with the test cases cases.

Sourcetype 'a with_options = ?stdout:Alcotest_engine.Formatters.stdout -> ?stderr:Alcotest_engine.Formatters.stderr -> ?and_exit:bool -> ?verbose:bool -> ?compact:bool -> ?tail_errors:[ `Unlimited | `Limit of int ] -> ?quick_only:bool -> ?show_errors:bool -> ?json:bool -> ?filter:(name:string -> index:int -> [ `Run | `Skip ]) -> ?log_dir:string -> ?bail:bool -> ?record_backtrace:bool -> ?ci:[ `Github_actions | `Unknown | `Disabled ] -> 'a

The various options taken by the tests runners run and run_with_args:

  • stdout (default to Fmt.stdout). The formatter used to print on the standard output.
  • stderr (default to Fmt.stderr). The formatter used to print ont the standard error output.
  • and_exit (default true). Once the tests have completed, exit with return code 0 if all tests passed, otherwise 1.
  • verbose (default false). Display the test std.out and std.err (rather than redirecting to a log file).
  • compact (default false). Compact the output of the tests.
  • tail_errors (default unlimited). Show only the last N lines of output of failed tests.
  • quick_only (default false). Do not run the `Slow tests, unless --tags selects them.
  • show_errors (default false, or true when a CI is detected). Display the test errors.
  • json (default false). Print test results in a JSON-compatible format.
  • filter. Filter tests according to ~name, the name of the test, and ~index, the number of the test case.
  • log_dir (default "$PWD/_build/_tests/", or "$DUNE_SOURCEROOT/_build/_tests/" under dune). The directory in which to log the output of the tests. Each run also writes a JUnit XML report in its junit directory, where a failed check or fail is a <failure> and any other exception is an <error>.
  • bail (default false). If true, stop running the tests after the first failure.
  • record_backtrace (default true). Enable backtrace recording before beginning testing.
  • ci (default auto-detected). Whether to enable specific logging for a CI system.
Sourceval run : (?argv:string array -> string -> unit test list -> return) with_options

run n t runs the test suite t. n is the name of the tested library.

The optional argument and_exit controls what happens when the function ends. By default, and_exit is set, which makes the function exit with 0 if everything is fine or 1 if there is an issue. If and_exit is false, then the function raises Test_error on error.

The optional argument argv specifies command line arguments sent to alcotest like "--json", "--verbose", etc. Note that this array will be treated like a regular Sys.argv, so the array must have at least one element, and the first element will be treated as if it was the command name and thus ignored for the purposes of option processing. So ~argv:[||] is an error, ~argv:[| "--verbose" |] will have no effect, and ~argv:[| "ignored"; "--verbose" |] will successfully pass the verbose option.

Sourceval run_with_args : (?argv:string array -> string -> 'a Cmdliner.Term.t -> 'a test list -> return) with_options

run_with_args n a t Similar to run a t but take an extra argument a. Every test function will receive as argument the evaluation of the Cmdliner term a: this is useful to configure the test behaviors using the CLI.

Testable values

The following combinators represent types that can be used with the check functions below.

Sourcemodule type TESTABLE = sig ... end

TESTABLE provides an abstract description for testable values.

Sourcetype 'a testable = (module TESTABLE with type t = 'a)

The type for testable values.

Sourceval testable : 'a Fmt.t -> ('a -> 'a -> bool) -> 'a testable

testable pp eq is a new testable with the pretty-printer pp and equality eq.

Sourceval pp : 'a testable -> 'a Fmt.t

pp t is t's pretty-printer.

Sourceval equal : 'a testable -> 'a -> 'a -> bool

equal t is t's equality.

Sourceval bool : bool testable

bool tests booleans.

Sourceval int : int testable

int tests integers.

Sourceval int32 : int32 testable

int32 tests 32-bit integers.

Sourceval int64 : int64 testable

int64 tests 64-bit integers.

Sourceval float : float -> float testable

float tests floats with specified absolute error.

Sourceval char : char testable

char tests characters.

uchar tests Unicode characters.

Sourceval string : string testable

string tests OCaml strings. When the output supports UTF-8, a valid UTF-8 string prints as is, apart from escaping ASCII as %S does.

Sourceval bytes : bytes testable

bytes tests OCaml bytes.

Sourceval unit : unit testable

unit tests unit values (useful for functions with side-effects).

Sourceval list : 'a testable -> 'a list testable

list t tests lists of ts.

Sourceval seq : 'a testable -> 'a Seq.t testable

seq t tests sequence of ts.

Sourceval slist : 'a testable -> ('a -> 'a -> int) -> 'a list testable

slist t comp tests sorted lists of ts. The list are sorted using comp.

Sourceval array : 'a testable -> 'a array testable

array t tests arrays of ts.

Sourceval iarray : 'a testable -> 'a Iarray.t testable

iarray t tests immutable arrays of ts. Available from OCaml 5.4.

Sourceval option : 'a testable -> 'a option testable

option t tests optional ts.

Sourceval result : 'a testable -> 'e testable -> ('a, 'e) result testable

result t e tests ts on success and es on failure.

Sourceval pair : 'a testable -> 'b testable -> ('a * 'b) testable

pair a b tests pairs of as and bs.

Sourceval triple : 'a testable -> 'b testable -> 'c testable -> ('a * 'b * 'c) testable

triple a b c tests triples of as, bs and cs.

Sourceval of_pp : 'a Fmt.t -> 'a testable

of_pp pp tests values which can be printed using pp and compared using Stdlib.compare

Sourceval pass : 'a testable

pass tests values of any type and always succeeds.

Sourceval reject : 'a testable

reject tests values of any type and always fails.

Sourceval neg : 'a testable -> 'a testable

neg t is t's negation: it is true when t is false and it is false when t is true.

Sourceval map : ('b -> 'a) -> 'a testable -> 'b testable

map f t lift a 'a testable to a 'b testable, converting 'b to 'a.

Assertion functions

Functions for asserting various properties within unit-tests. A failing assertion will cause the testcase to fail immediately.

Sourcemodule Source_code_position : sig ... end
Sourcetype 'a extra_info = ?here:Source_code_position.here -> ?pos:Source_code_position.pos -> 'a

The assertion functions optionally take information about the location at which they are called in the source code. This is used for giving more descriptive error messages in the case of failure.

Sourceval check : ('a testable -> string -> 'a -> 'a -> unit) extra_info

check testable msg expected actual checks that two values are equal.

msg is printed if the check fails.

If check isn't in a tail-call position, Alcotest may guess the location of the check. Otherwise, use extra_info to report the location.

Sourceval check' : ('a testable -> msg:string -> expected:'a -> actual:'a -> unit) extra_info

Check that two values are equal (labeled variant of check).

Sourceval fail : (string -> 'a) extra_info

Simply fail.

Sourceval failf : (('a, Format.formatter, unit, 'b) format4 -> 'a) extra_info

Simply fail with a formatted message.

Sourceval check_raises : (string -> exn -> (unit -> unit) -> unit) extra_info

Check that an exception is raised.

Sourceval match_raises : (string -> (exn -> bool) -> (unit -> unit) -> unit) extra_info

match_raises msg exception_is_expected f Runs f (), and passes the raised exception to exception_is_expected. The check fails when no exception is raised, or exception_is_expected returns false.

Sourceval skip : unit -> 'a

Skip the current test case.

  • since 1.7.0

Versioned APIs

Sourcemodule V1 : sig ... end

An alias of the above API that provides a stability guarantees over major version changes.

Unix-specific engine constructors

The Alcotest_engine package provides the most general form of the Alcotest API, parameterised over the thread implementation and the platform. This package provides the Unix platform implementation.