package qcheck

You can search for identifiers within the package.

in-package search v0.2.0

doc
- CHANGELOG
- Library qcheck
  - QCheck_runner
    
    Color
    
    Raw
- Sources
  - qcheck
    
    QCheck_runner.ml

Legend:
Page
Library
Module
Module type
Parameter
Class
Class type
Source

Module `QCheck_runner`Source

include module type of struct include QCheck_base_runner end

Runners for Tests

Once you have built tests using QCheck.Test.make or QCheck2.Test.make, you need to run them. This module contains several runners, which are designed to run every test and report the result.

By default, you can use run_tests in a test program as follows:

  let testsuite = [
    Test.make ...;
    Test.make ...;
  ]

  let () =
    let errcode = QCheck_base_runner.run_tests ~verbose:true testsuite in
    exit errcode

which will run the tests, and exit the program. The error code will be 0 if all tests pass, 1 otherwise.

run_tests_main can be used as a shortcut for that, also featuring command-line parsing (using Arg) to activate verbose mode and others.

State

Sourceval random_state : unit -> Random.State.t

Access the current random state

Sourceval verbose : unit -> bool

Is the default mode verbose or quiet?

Sourceval long_tests : unit -> bool

Is the default mode to run long tests or nor?

Sourceval set_seed : int -> unit

Change the random_state by creating a new one, initialized with the given seed.

Sourceval set_verbose : bool -> unit

Change the value of verbose ()

Sourceval set_long_tests : bool -> unit

Change the value of long_tests ()

Console message printing

In verbose mode, by default QCheck_base_runner prints frequent sub-second messages suitable for an interactive console test run. This behaviour can be changed by the environment variable QCHECK_MSG_INTERVAL. Intervals are given in seconds and can also be decimal numbers. For example, setting

   QCHECK_MSG_INTERVAL=7.5

will only print a console message every 7.5 seconds. This feature can be useful in a CI context, where updates are printed on consecutive lines and one may want to avoid overflowing the CI log files with too many lines.

Note: The start and finishing message for each test is printed eagerly in verbose mode regardless of the specified message interval.

Sourceval get_time_between_msg : unit -> float

Get the minimum time (in seconds) to wait between printing messages.

since 0.9

Sourceval set_time_between_msg : float -> unit

Set the minimum time (in seconds) between messages.

since 0.9

Event handlers

Sourcetype counter = private QCheck_base_runner.counter = {

start : float;
expected : int;
mutable gen : int;
mutable passed : int;
mutable failed : int;
mutable errored : int;

}

The type of counter used to keep tracks of the events received for a given test cell.

Sourcetype handler = QCheck_base_runner.handler = {

handler : 'a. 'a QCheck2.Test.handler;

}

A type to represent polymorphic-enough handlers for test cells.

Source

type handler_gen =
  colors:bool ->
  debug_shrink:out_channel option ->
  debug_shrink_list:string list ->
  size:int ->
  out:out_channel ->
  verbose:bool ->
  counter ->
  handler

An alias type to a generator of handlers for test cells.

Sourceval default_handler : handler_gen

The default handler used.

Source

val debug_shrinking_choices : 
  colors:bool ->
  out:out_channel ->
  name:string ->
  'a QCheck2.Test.cell ->
  step:int ->
  'a ->
  unit

The function used by the default handler to debug shrinking choices. This can be useful to outside users trying to reproduce some of the base-runner behavior.

since 0.19

Run a Suite of Tests and Get Results

Source

val run_tests : 
  ?handler:handler_gen ->
  ?colors:bool ->
  ?verbose:bool ->
  ?long:bool ->
  ?debug_shrink:out_channel option ->
  ?debug_shrink_list:string list ->
  ?out:out_channel ->
  ?rand:Random.State.t ->
  QCheck2.Test.t list ->
  int

Run a suite of tests, and print its results. This is an heritage from the "qcheck" library.

returns
an error code, 0 if all tests passed, 1 otherwise.

parameter colors
if true (default), colorful output

parameter verbose
if true, prints more information about test cases (default: false)

parameter long
if true, runs the long versions of the tests (default: false)

parameter debug_shrink
debug_shrink:(Some ch) writes a log of successful shrink attempts to channel ch, for example ~debug_shrink:(Some (open_out "mylog.txt")). Use together with a non-empty list in ~debug_shrink_list.

parameter debug_shrink_list
the test names to log successful shrink attempts for, for example ~debug_shrink_list:["list_rev_is_involutive"]. Requires ~debug_shrink to be Some ch.

parameter out
print output to the provided channel (default: stdout)

parameter rand
start the test runner in the provided RNG state

Sourceval run_tests_main : ?argv:string array -> QCheck2.Test.t list -> 'a

Can be used as the main function of a test file. Exits with a non-0 code if the tests fail. It refers to run_tests for actually running tests after CLI options have been parsed.

The available options are:

"--verbose" (or "-v") for activating verbose tests
"--seed <n>" (or "-s <n>") for repeating a previous run by setting the random seed
"--long" for running the long versions of the tests

Below is an example of the output of the run_tests and run_tests_main function:

random seed: 438308050
generated  error;  fail; pass / total -     time -- test name
[✓] (1000)    0 ;    0 ; 1000 / 1000 --     0.5s -- list_rev_is_involutive
[✗] (   1)    0 ;    1 ;    0 /   10 --     0.0s -- should_fail_sort_id
[✗] (   1)    1 ;    0 ;    0 /   10 --     0.0s -- should_error_raise_exn
[✓] (1000)    0 ;    0 ; 1000 / 1000 --     0.0s -- collect_results

--- Failure --------------------------------------------------------------------

Test should_fail_sort_id failed (11 shrink steps):

[1; 0]

=== Error ======================================================================

Test should_error_raise_exn errored on (62 shrink steps):

0

exception QCheck_runner_test.Error
Raised at file "example/QCheck_runner_test.ml", line 20, characters 20-25
Called from file "src/QCheck.ml", line 839, characters 13-33


+++ Collect ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Collect results for test collect_results:

 4:     207 cases (20.7%)
 3:     190 cases (19.0%)
 2:     219 cases (21.9%)
 1:     196 cases (19.6%)
 0:     188 cases (18.8%)

================================================================================
failure (1 tests failed, 1 tests errored, ran 4 tests)

Sourcemodule Color = QCheck_base_runner.Color

Sourcemodule Raw = QCheck_base_runner.Raw

include module type of struct include QCheck_ounit end

Conversion of tests to OUnit Tests

since 0.9

Source

val to_ounit_test : 
  ?verbose:bool ->
  ?long:bool ->
  ?rand:Random.State.t ->
  QCheck2.Test.t ->
  OUnit.test

to_ounit_test ~rand t wraps t into a OUnit test

parameter verbose
used to print information on stdout (default: verbose())

parameter rand
the random generator to use (default: random_state ())

Source

val to_ounit_test_cell : 
  ?verbose:bool ->
  ?long:bool ->
  ?rand:Random.State.t ->
  _ QCheck2.Test.cell ->
  OUnit.test

Same as to_ounit_test but with a polymorphic test cell

Sourceval (>:::) : string -> QCheck2.Test.t list -> OUnit.test

Same as OUnit.(>:::) but with a list of QCheck2 tests

Sourceval to_ounit2_test : ?rand:Random.State.t -> QCheck2.Test.t -> OUnit2.test

to_ounit2_test ?rand t wraps t into a OUnit2 test

parameter rand
the random generator to use (default: a static seed for reproducibility), can be overridden with "-seed" on the command-line

Source

val to_ounit2_test_list : 
  ?rand:Random.State.t ->
  QCheck2.Test.t list ->
  OUnit2.test list

to_ounit2_test_list ?rand t like to_ounit2_test but for a list of tests

OUnit runners

QCheck provides some custom runners for OUnit tests.

run is used by qtest.
run_tap should be compatible with TAP.

Note that OUnit.run_test_tt or OUnit.run_test_tt_main can be used as well, in particular when QCheck tests are mixed with normal unit tests.

For OUnit2 you can use OUnit2.run_test_tt_main.

Sourceval run : ?argv:string array -> OUnit.test -> int

run test runs the test, and returns an error code that is 0 if all tests passed, 1 otherwise. This is the default runner used by the comment-to-test generator.

parameter argv
the command line arguments to parse parameters from (default Sys.argv)

raises Arg.Bad
in case argv contains unknown arguments

raises Arg.Help
in case argv contains "--help"

This test runner displays execution in a compact way, making it good for suites that have lots of tests.

Output example:

random seed: 101121210
random seed: 101121210
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
Error: tests>error_raise_exn

test `error_raise_exn` raised exception `QCheck_ounit_test.Error`
on `0 (after 62 shrink steps)`
Raised at file "example/QCheck_ounit_test.ml", line 19, characters 20-25
Called from file "src/QCheck.ml", line 846, characters 13-33

///////////////////////////////////////////////////////////////////////////////
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
Failure: tests>fail_sort_id

fail_sort_id
///////////////////////////////////////////////////////////////////////////////
Ran: 4 tests in: 0.74 seconds.
WARNING! SOME TESTS ARE NEITHER SUCCESSES NOR FAILURES!

Sourceval run_tap : OUnit.test -> OUnit.test_results

TAP-compatible test runner, in case we want to use a test harness. It prints one line per test.

package qcheck

Module QCheck_runnerSource

Runners for Tests

State

Console message printing

Event handlers

Run a Suite of Tests and Get Results

Conversion of tests to OUnit Tests

OUnit runners

Module `QCheck_runner`Source