package qcheck-alcotest
Alcotest backend for qcheck
Install
Dune Dependency
Authors
Maintainers
Sources
v0.22.tar.gz
md5=f05bdc3a15da4c65e9e4a87f845bc1ef
sha512=2bc273d73b9732564eadb78a2ab390b85c6f9ffb7ba46d3ec8725384631f199e17937740264c550ef4f06df62ffe655f2be19259f8f4cf4c2e81010a9c1a19e3
Description
README
README.adoc
= QCheck :toc: macro :toclevels: 4 :source-highlighter: pygments QuickCheck inspired property-based testing for OCaml, and combinators to generate random values to run tests on. image::https://github.com/c-cube/qcheck/actions/workflows/main.yml/badge.svg[alt="build", link=https://github.com/c-cube/qcheck/actions/workflows/main.yml] The documentation can be found https://c-cube.github.io/qcheck/[here]. This library spent some time in https://github.com/vincent-hugot/iTeML[qtest], but is now standalone again! To construct advanced random generators, the following libraries might be of interest: - https://gitlab.inria.fr/fpottier/feat/[Feat] - @gasche's https://github.com/gasche/random-generator/[generator library] Jan Midtgaard (@jmid) has http://janmidtgaard.dk/quickcheck/index.html[a lecture] about property-based testing that relies on QCheck. toc::[] == Use See the documentation. I also wrote https://cedeela.fr/quickcheck-for-ocaml[a blog post] that explains how to use it and some design choices; however, be warned that the API changed in lots of small ways (in the right direction, I hope) so the code will not work any more. <<examples>> is an updated version of the blog post's examples. == Build $ make You can use opam: $ opam install qcheck == License The code is now released under the BSD license. [[examples]] == An Introduction to the Library First, let's see a few tests. Let's open a toplevel (e.g. utop) and type the following to load QCheck: [source,OCaml] ---- #require "qcheck";; ---- NOTE: alternatively, it is now possible to locally do: `dune utop src` to load `qcheck`. === List Reverse is Involutive We write a random test for checking that `List.rev (List.rev l) = l` for any list `l`: [source,OCaml] ---- let test = QCheck.Test.make ~count:1000 ~name:"list_rev_is_involutive" QCheck.(list small_nat) (fun l -> List.rev (List.rev l) = l);; (* we can check right now the property... *) QCheck.Test.check_exn test;; ---- In the above example, we applied the combinator `list` to the random generator `small_nat` (ints between 0 and 100), to create a new generator of lists of random integers. These builtin generators come with printers and shrinkers which are handy for outputting and minimizing a counterexample when a test fails. Consider the buggy property `List.rev l = l`: [source,OCaml] ---- let test = QCheck.Test.make ~count:1000 ~name:"my_buggy_test" QCheck.(list small_nat) (fun l -> List.rev l = l);; ---- When we run this test we are presented with a counterexample: [source,OCaml] ---- # QCheck.Test.check_exn test;; Exception: QCheck.Test.Test_fail ("my_buggy_test", ["[0; 1] (after 23 shrink steps)"]). ---- In this case QCheck found the minimal counterexample `[0;1]` to the property `List.rev l = l` and it spent 23 steps shrinking it. Now, let's run the buggy test with a decent runner that will print the results nicely (the exact output will change at each run, because of the random seed): ---- # QCheck_runner.run_tests [test];; --- Failure -------------------------------------------------------------------- Test my_buggy_test failed (10 shrink steps): [0; 1] ================================================================================ failure (1 tests failed, 0 tests errored, ran 1 tests) - : int = 1 ---- For an even nicer output `QCheck_runner.run_tests` also accepts an optional parameter `~verbose:true`. === Mirrors and Trees `QCheck` provides many useful combinators to write generators, especially for recursive types, algebraic types, and tuples. Let's see how to generate random trees: [source,OCaml] ---- type tree = Leaf of int | Node of tree * tree let leaf x = Leaf x let node x y = Node (x,y) let tree_gen = QCheck.Gen.(sized @@ fix (fun self n -> match n with | 0 -> map leaf nat | n -> frequency [1, map leaf nat; 2, map2 node (self (n/2)) (self (n/2))] ));; (* generate a few trees, just to check what they look like: *) QCheck.Gen.generate ~n:20 tree_gen;; let arbitrary_tree = let open QCheck.Iter in let rec print_tree = function | Leaf i -> "Leaf " ^ (string_of_int i) | Node (a,b) -> "Node (" ^ (print_tree a) ^ "," ^ (print_tree b) ^ ")" in let rec shrink_tree = function | Leaf i -> QCheck.Shrink.int i >|= leaf | Node (a,b) -> of_list [a;b] <+> (shrink_tree a >|= fun a' -> node a' b) <+> (shrink_tree b >|= fun b' -> node a b') in QCheck.make tree_gen ~print:print_tree ~shrink:shrink_tree;; ---- Here we write a generator of random trees, `tree_gen`, using the `fix` combinator. `fix` is *sized* (it is a function from `int` to a random generator; in particular for size 0 it returns only leaves). The `sized` combinator first generates a random size, and then applies its argument to this size. Other combinators include monadic abstraction, lifting functions, generation of lists, arrays, and a choice function. Then, we define `arbitrary_tree`, a `tree QCheck.arbitrary` value, which contains everything needed for testing on trees: - a random generator (mandatory), weighted with `frequency` to increase the chance of generating deep trees - a printer (optional), very useful for printing counterexamples - a *shrinker* (optional), very useful for trying to reduce big counterexamples to small counterexamples that are usually more easy to understand. The above shrinker strategy is to - reduce the integer leaves, and - substitute an internal `Node` with either of its subtrees or by splicing in a recursively shrunk subtree. A range of combinators in `QCheck.Shrink` and `QCheck.Iter` are available for building shrinking functions. We can write a failing test using this generator to see the printer and shrinker in action: [source,OCaml] ---- let rec mirror_tree (t:tree) : tree = match t with | Leaf _ -> t | Node (a,b) -> node (mirror_tree b) (mirror_tree a);; let test_buggy = QCheck.Test.make ~name:"buggy_mirror" ~count:200 arbitrary_tree (fun t -> t = mirror_tree t);; QCheck_runner.run_tests [test_buggy];; ---- This test fails with: [source,OCaml] ---- --- Failure -------------------------------------------------------------------- Test mirror_buggy failed (6 shrink steps): Node (Leaf 0,Leaf 1) ================================================================================ failure (1 tests failed, 0 tests errored, ran 1 tests) - : int = 1 ---- With the (new found) understanding that mirroring a tree changes its structure, we can formulate another property that involves sequentializing its elements in a traversal: [source,OCaml] ---- let tree_infix (t:tree): int list = let rec aux acc t = match t with | Leaf i -> i :: acc | Node (a,b) -> aux (aux acc b) a in aux [] t;; let test_mirror = QCheck.Test.make ~name:"mirror_tree" ~count:200 arbitrary_tree (fun t -> List.rev (tree_infix t) = tree_infix (mirror_tree t));; QCheck_runner.run_tests [test_mirror];; ---- === Preconditions The functions `QCheck.assume` and `QCheck.(==>)` can be used for tests with preconditions. For instance, `List.hd l :: List.tl l = l` only holds for non-empty lists. Without the precondition, the property is false and will even raise an exception in some cases. [source,OCaml] ---- let test_hd_tl = QCheck.(Test.make (list int) (fun l -> assume (l <> []); l = List.hd l :: List.tl l));; QCheck_runner.run_tests [test_hd_tl];; ---- === Long tests It is often useful to have two version of a testsuite: a short one that runs reasonably fast (so that it is effectively run each time a projet is built), and a long one that might be more exhaustive (but whose running time makes it impossible to run at each build). To that end, each test has a 'long' version. In the long version of a test, the number of tests to run is multiplied by the `~long_factor` argument of `QCheck.Test.make`. === Runners The module `QCheck_runner` defines several functions to run tests, including compatibility with `OUnit`. The easiest one is probably `run_tests`, but if you write your tests in a separate executable you can also use `run_tests_main` which parses command line arguments and exits with `0` in case of success, or an error number otherwise. === Integration within OUnit https://github.com/gildor478/ounit[OUnit] is a popular unit-testing framework for OCaml. QCheck provides a sub-library `qcheck-ounit` with some helpers, in `QCheck_ounit`, to convert its random tests into OUnit tests that can be part of a wider test-suite. [source,OCaml] ---- let passing = QCheck.Test.make ~count:1000 ~name:"list_rev_is_involutive" QCheck.(list small_nat) (fun l -> List.rev (List.rev l) = l);; let failing = QCheck.Test.make ~count:10 ~name:"fail_sort_id" QCheck.(list small_nat) (fun l -> l = List.sort compare l);; let _ = let open OUnit in run_test_tt_main ("tests" >::: List.map QCheck_ounit.to_ounit_test [passing; failing]) ---- NOTE: the package `qcheck` contains the module `QCheck_runner` which contains both custom runners and OUnit-based runners. === Integration within alcotest https://github.com/mirage/alcotest/[Alcotest] is a simple and colorful test framework for OCaml. QCheck now provides a sub-library `qcheck-alcotest` to easily integrate into an alcotest test suite: [source,OCaml] ---- let passing = QCheck.Test.make ~count:1000 ~name:"list_rev_is_involutive" QCheck.(list small_int) (fun l -> List.rev (List.rev l) = l);; let failing = QCheck.Test.make ~count:10 ~name:"fail_sort_id" QCheck.(list small_int) (fun l -> l = List.sort compare l);; let () = let suite = List.map QCheck_alcotest.to_alcotest [ passing; failing] in Alcotest.run "my test" [ "suite", suite ] ---- === Integration within Rely https://reason-native.com/docs/rely/[Rely] is a Jest-inspire native reason testing framework. @reason-native/qcheck-rely is available via NPM and provides matchers for the easy use of qCheck within Rely. [source, Reason] ---- open TestFramework; open QCheckRely; let {describe} = extendDescribe(QCheckRely.Matchers.matchers); describe("qcheck-rely", ({test}) => { test("passing test", ({expect}) => { let passing = QCheck.Test.make( ~count=1000, ~name="list_rev_is_involutive", QCheck.(list(small_int)), l => List.rev(List.rev(l)) == l ); expect.ext.qCheckTest(passing); (); }); test("failing test", ({expect}) => { let failing = QCheck.Test.make( ~count=10, ~name="fail_sort_id", QCheck.(list(small_int)), l => l == List.sort(compare, l) ); expect.ext.qCheckTest(failing); (); }); }); ---- === Deriver A ppx_deriver is provided to derive QCheck generators from a type declaration. [source,OCaml] ---- type tree = Leaf of int | Node of tree * tree [@@deriving qcheck] ---- See the according https://github.com/c-cube/qcheck/tree/master/src/ppx_deriving_qcheck/[README] for more information and examples. === Compatibility notes Starting with 0.9, the library is split into several components: - `qcheck-core` depends only on unix and bytes. It contains the module `QCheck` and a `QCheck_base_runner` module with our custom runners. - `qcheck-ounit` provides an integration layer for `OUnit` - `qcheck` provides a compatibility API with older versions of qcheck, using both `qcheck-core` and `qcheck-ounit`. It provides `QCheck_runner` which is similar to older versions and contains both custom and Ounit-based runners. - `qcheck-alcotest` provides an integration layer with `alcotest` Normally, for contributors, `opam pin https://github.com/c-cube/qcheck` will pin all these packages.
Dependencies (5)
-
ocaml
>= "4.08.0"
-
alcotest
>= "1.2.0"
-
qcheck-core
= version
- base-unix
-
dune
>= "2.8.0"
Dev Dependencies (1)
-
odoc
with-doc
Used by (84)
- alg_structs_qcheck
- base32
- bastet
-
bls12-381
>= "18.0"
- cborl
-
docfd
>= "2.2.0"
- eris
- eris-lwt
-
inferno
>= "20220603"
-
irmin
>= "3.4.0"
-
irmin-test
>= "3.9.0"
-
lockfree
>= "0.3.0"
-
logtk
>= "1.5.1"
-
lru
>= "0.3.0"
- lt-code
- obatcher
- octez-bls12-381-polynomial
- octez-l2-libs
- octez-libs
- octez-plonk
- octez-proto-libs
-
octez-protocol-017-PtNairob-libs
< "20.1"
- octez-protocol-018-Proxford-libs
- octez-protocol-019-PtParisB-libs
- octez-protocol-020-PsParisC-libs
- octez-protocol-alpha-libs
- octez-shell-libs
- ometrics
-
ppx_deriving_qcheck
>= "0.2.0"
-
ppx_pbt
>= "0.2.1"
-
pratter
>= "1.2.1"
-
prbnmcn-dagger-test
>= "0.0.2"
- preface
-
psq
>= "0.1.1"
- saturn
- saturn_lockfree
- seqes
- stramon-lib
- term-indexing
-
tezos-012-Psithaca-test-helpers
>= "14.0"
- tezos-013-PtJakart-test-helpers
- tezos-014-PtKathma-test-helpers
-
tezos-alpha-test-helpers
>= "14.0"
-
tezos-base
>= "11.0" & < "12.0" | >= "13.0"
-
tezos-base-test-helpers
>= "13.0"
- tezos-bls12-381-polynomial
-
tezos-client-011-PtHangz2
< "12.0" | >= "13.0" & < "17.1"
-
tezos-client-012-Psithaca
>= "13.0" & < "17.1"
-
tezos-client-013-PtJakart
< "17.1"
-
tezos-client-014-PtKathma
< "17.1"
-
tezos-client-015-PtLimaPt
< "17.1"
- tezos-client-016-PtMumbai
- tezos-client-017-PtNairob
-
tezos-client-alpha
>= "11.0" & < "12.0" | >= "13.0"
-
tezos-context
>= "13.0"
-
tezos-crypto
>= "11.0" & < "12.0" | >= "13.0"
- tezos-crypto-dal
- tezos-hacl
- tezos-layer2-store
-
tezos-lwt-result-stdlib
>= "11.0" & < "12.0" | >= "13.0"
-
tezos-mockup
>= "11.0" & < "12.0" | >= "13.0"
-
tezos-plonk
= "0.1.3"
-
tezos-protocol-environment
>= "11.0" & < "12.0" | >= "13.0"
- tezos-protocol-plugin-012-Psithaca-tests
- tezos-protocol-plugin-013-PtJakart-tests
- tezos-protocol-plugin-alpha-tests
-
tezos-proxy
>= "11.0" & < "12.0" | >= "13.0"
- tezos-proxy-server-config
-
tezos-requester
>= "11.0" & < "12.0" | >= "13.0"
-
tezos-rpc-http-server
>= "11.0" & < "12.0" | >= "13.0"
-
tezos-scoru-wasm-helpers
< "17.1"
-
tezos-shell
>= "11.0" & < "12.0"
-
tezos-shell-services-test-helpers
< "12.0" | >= "13.0"
-
tezos-stdlib
>= "11.0" & < "12.0" | >= "13.0"
-
tezos-stdlib-unix
>= "17.1"
- tezos-test-helpers
-
tezos-webassembly-interpreter
< "15.0"
- timedesc
- timere
- weevil
-
yocaml
>= "2.0.0"
-
yocaml_syndication
>= "2.0.0"
-
yuujinchou
= "2.0.0"
- zar
Conflicts
None
sectionYPositions = computeSectionYPositions($el), 10)"
x-init="setTimeout(() => sectionYPositions = computeSectionYPositions($el), 10)"
>
On This Page