package alcotest

  1. Overview
  2. Docs
Alcotest is a lightweight and colourful test framework

Install

Dune Dependency

github.com Readme Changelog ISC License Edit opam file Versions (40)

Authors

  1. T
    Thomas Gazagnaire

Maintainers

  1. T
    thomas@gazagnaire.org

Sources

alcotest-mirage-1.2.3.tbz
sha256=085c481aeedf80d766ff9ba4d9929688bed01ef390915dc28a9bb4ba7664b2ae
sha512=ca489811d3f13a2604a4b0a2b7463d611741bf8a96655e3ae1dfbeb60f2e81f589d389f12379543e5e2a31e973170134855f10d1ba94ffdc6123e34227a7d37a

Description

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Published: 08 Sep 2020

README

Alcotest is a lightweight and colourful test framework.

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run. See the manpage for details.

For information on contributing to Alcotest, see CONTRIBUTING.md.

Examples

A simple example (taken from examples/simple.ml):

Generated by the following test suite specification:

(* Build with `ocamlbuild -pkg alcotest simple.byte` *)

(* A module with functions to test *)
module To_test = struct
  let lowercase = String.lowercase_ascii
  let capitalize = String.capitalize_ascii
  let str_concat = String.concat ""
  let list_concat = List.append
end

(* The tests *)
let test_lowercase () =
  Alcotest.(check string) "same string" "hello!" (To_test.lowercase "hELLO!")

let test_capitalize () =
  Alcotest.(check string) "same string" "World." (To_test.capitalize "world.")

let test_str_concat () =
  Alcotest.(check string) "same string" "foobar" (To_test.str_concat ["foo"; "bar"])

let test_list_concat () =
  Alcotest.(check (list int)) "same lists" [1; 2; 3] (To_test.list_concat [1] [2; 3])

(* Run it *)
let () =
  let open Alcotest in
  run "Utils" [
      "string-case", [
          test_case "Lower case"     `Quick test_lowercase;
          test_case "Capitalization" `Quick test_capitalize;
        ];
      "string-concat", [ test_case "String mashing" `Quick test_str_concat  ];
      "list-concat",   [ test_case "List mashing"   `Slow  test_list_concat ];
    ]

The result is a self-contained binary which displays the test results. Use dune exec examples/simple.exe -- --help to see the runtime options.

Here's an example of a of failing test suite:

By default, only the first failing test log is printed to the console (and all test logs are captured on disk). Pass --show-errors to print all error messages.

Selecting tests to execute

You can filter which tests to run by supplying a regular expression matching the names of the tests to execute, or by passing a regular expression and a comma-separated list of test numbers (or ranges of test numbers, e.g. 2,4..9):

$ ./simple.native test '.*concat*'
Testing Utils.
[SKIP]     string-case            0   Lower case.
[SKIP]     string-case            1   Capitalization.
[OK]       string-concat          0   String mashing.
[OK]       list-concat            0   List mashing.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 tests run.

$ ./simple.native test 'string-case' '1..3'
Testing Utils.
[SKIP]     string-case            0   Lower case.
[OK]       string-case            1   Capitalization.
[SKIP]     string-concat          0   String mashing.
[SKIP]     list-concat            0   List mashing.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.

Note that you cannot filter by test case name (i.e. Lower case or Capitalization), you must filter by test name & number instead.

See the examples folder for more examples.

Quick and Slow tests

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

Passing custom options to the tests

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your prefered number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ test.exe test
test.exe: required option -n is missing

$ test.exe test -n 42
Testing foo.
[OK]                all          0   int.

Lwt

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type unit -> unit Lwt.t and alcotest-lwt calls Lwt_main.run for you.

However, there are a couple of extra features:

  • If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).

  • You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Lwt_main.run @@ Alcotest_lwt.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ test.exe
Testing foo.
[ERROR]             all          0   one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken

Comparison with other testing frameworks

The README is pretty clear about that:

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

  • Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;

  • Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

  • qcheck qcheck does random generation and property testing (e.g. Quick Check)

  • crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.

  • ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.

Dependencies (9)

  1. uutf >= "1.0.0"
  2. stdlib-shims
  3. re >= "1.7.2"
  4. uuidm
  5. cmdliner >= "1.0.3" & < "1.1.0"
  6. astring
  7. fmt >= "0.8.7"
  8. ocaml >= "4.03.0"
  9. dune >= "2.2"

Dev Dependencies (1)

  1. odoc with-doc

  1. ahrocksdb
  2. albatross >= "1.5.0"
  3. alcotest-async < "1.0.0" | = "1.2.3"
  4. alcotest-lwt < "1.0.0" | = "1.2.3"
  5. alcotest-mirage = "1.2.3"
  6. alg_structs_qcheck
  7. ambient-context
  8. ambient-context-eio
  9. ambient-context-lwt
  10. angstrom >= "0.7.0"
  11. ansi >= "0.6.0"
  12. anycache >= "0.7.4"
  13. anycache-async
  14. anycache-lwt
  15. archetype >= "1.4.2"
  16. archi
  17. arp
  18. arp-mirage
  19. arrakis
  20. art
  21. asak >= "0.2"
  22. asli >= "0.2.0"
  23. asn1-combinators >= "0.2.2"
  24. atd >= "2.3.3"
  25. atdgen >= "2.10.0"
  26. atdpy
  27. atdts
  28. base32
  29. base64 >= "2.1.2" & < "3.2.0" | >= "3.4.0"
  30. bastet
  31. bastet_async
  32. bastet_lwt
  33. bech32
  34. bechamel >= "0.5.0"
  35. bigarray-overlap
  36. bigstringaf
  37. bitlib
  38. blake2
  39. bloomf
  40. bls12-381 < "0.4.1" | >= "3.0.0" & < "18.0"
  41. bls12-381-hash
  42. bls12-381-js >= "0.4.2"
  43. bls12-381-js-gen >= "0.4.2"
  44. bls12-381-legacy
  45. bls12-381-signature
  46. bls12-381-unix
  47. blurhash
  48. builder-web
  49. bulletml
  50. bytebuffer
  51. ca-certs
  52. ca-certs-nss
  53. cactus
  54. caldav
  55. calendar >= "3.0.0"
  56. callipyge
  57. camlix
  58. camlkit
  59. camlkit-base
  60. capnp-rpc < "1.2.3"
  61. capnp-rpc-lwt < "0.3"
  62. capnp-rpc-mirage >= "0.9.0"
  63. capnp-rpc-unix >= "0.9.0" & < "1.2.3"
  64. carray
  65. carton
  66. carton-git
  67. carton-lwt
  68. cborl
  69. ccss >= "1.6"
  70. cf-lwt
  71. chacha
  72. channel
  73. charrua-client
  74. charrua-client-lwt
  75. charrua-client-mirage < "0.11.0"
  76. checked_oint < "0.1.1"
  77. checkseum >= "0.0.3"
  78. cid
  79. clarity-lang
  80. class_group_vdf
  81. cohttp >= "0.17.0"
  82. cohttp-curl-async
  83. cohttp-curl-lwt
  84. cohttp-eio >= "6.0.0~beta2"
  85. colombe >= "0.2.0"
  86. color
  87. conan
  88. conan-cli
  89. conan-database
  90. conan-lwt
  91. conan-unix
  92. conduit = "3.0.0"
  93. conex < "0.10.0"
  94. conex-mirage-crypto
  95. conex-nocrypto
  96. conformist
  97. cookie
  98. cow >= "2.2.0"
  99. css
  100. css-parser
  101. cstruct >= "3.3.0"
  102. cstruct-sexp
  103. ctypes-zarith
  104. cuid
  105. curly
  106. current >= "0.4"
  107. current_git >= "0.6.4"
  108. current_incr
  109. cwe_checker
  110. data-encoding
  111. datakit >= "0.12.0"
  112. datakit-bridge-github >= "0.12.0"
  113. datakit-ci
  114. datakit-client-git >= "0.12.0"
  115. decompress >= "0.8" & < "1.5.3"
  116. depyt
  117. digestif >= "0.8.1"
  118. dispatch >= "0.4.1"
  119. dkim
  120. dkim-bin
  121. dkim-mirage
  122. dns >= "4.0.0"
  123. dns-cli
  124. dns-client >= "4.6.0"
  125. dns-forward < "0.9.0"
  126. dns-forward-lwt-unix
  127. dns-resolver
  128. dns-server
  129. dns-tsig
  130. dnssd
  131. dnssec
  132. docfd >= "2.2.0"
  133. dog < "0.2.1"
  134. domain-name
  135. dream
  136. dream-pure
  137. duff
  138. dune-release >= "1.0.0"
  139. duration >= "0.1.1"
  140. emile
  141. encore
  142. eqaf >= "0.5"
  143. equinoxe
  144. equinoxe-cohttp
  145. equinoxe-hlc
  146. eris
  147. eris-lwt
  148. ezgzip
  149. ezjsonm >= "0.4.2" & < "1.3.0"
  150. ezjsonm-lwt
  151. FPauth
  152. FPauth-core
  153. FPauth-responses
  154. FPauth-strategies
  155. faraday != "0.2.0"
  156. farfadet
  157. fat-filesystem >= "0.12.0"
  158. ff
  159. ff-pbt
  160. fiat-p256
  161. flex-array
  162. fsevents-lwt
  163. functoria >= "2.2.0"
  164. functoria-runtime >= "2.2.0" & != "3.0.1" & < "4.0.0~beta1"
  165. geojson
  166. geoml >= "0.1.1"
  167. git = "1.4.10" | = "1.5.0" | >= "1.5.2" & != "1.10.0"
  168. git-cohttp
  169. git-cohttp-mirage
  170. git-cohttp-unix
  171. git-mirage
  172. git-unix >= "1.10.0" & != "2.1.0"
  173. gitlab-unix
  174. glicko2
  175. gmap >= "0.3.0"
  176. gobba
  177. gpt
  178. graphql
  179. graphql-async
  180. graphql-cohttp >= "0.13.0"
  181. graphql-lwt
  182. graphql_parser != "0.11.0"
  183. graphql_ppx >= "0.7.1"
  184. h1_parser
  185. h2
  186. hacl
  187. hacl-star >= "0.6.0"
  188. hacl_func
  189. hacl_x25519 >= "0.2.0"
  190. highlexer
  191. hkdf
  192. hockmd
  193. html_of_jsx
  194. http
  195. http-multipart-formdata < "2.0.0"
  196. httpaf >= "0.2.0"
  197. hvsock
  198. icalendar >= "0.1.4"
  199. imagelib >= "20200929"
  200. index
  201. inferno >= "20220603"
  202. influxdb-async
  203. influxdb-lwt
  204. inquire < "0.2.0"
  205. interval-map
  206. iomux
  207. irmin < "0.8.0" | >= "0.9.6" & != "0.11.1" & < "1.0.0" | >= "2.0.0" & != "2.3.0"
  208. irmin-bench >= "2.7.0"
  209. irmin-chunk < "1.3.0" | >= "2.3.0"
  210. irmin-cli
  211. irmin-containers
  212. irmin-fs < "1.3.0" | >= "2.3.0"
  213. irmin-git < "2.0.0" | >= "2.3.0"
  214. irmin-graphql >= "2.3.0"
  215. irmin-http < "2.0.0"
  216. irmin-mem < "1.3.0" | >= "2.3.0"
  217. irmin-pack >= "2.4.0" & != "2.6.1"
  218. irmin-pack-tools
  219. irmin-test >= "2.2.0" & < "3.0.0"
  220. irmin-tezos
  221. irmin-tezos-utils
  222. irmin-unix >= "1.0.0" & < "1.3.3" | >= "2.4.0" & != "2.6.1"
  223. irmin-watcher
  224. jekyll-format
  225. jerboa
  226. jitsu
  227. jose
  228. json-data-encoding >= "0.9"
  229. json_decoder
  230. jsonxt
  231. junit_alcotest
  232. jwto
  233. ke >= "0.2"
  234. kkmarkdown
  235. lambda-runtime
  236. lambda_streams
  237. lambda_streams_async
  238. lambdapi >= "2.0.0"
  239. lambdoc >= "1.0-beta4"
  240. ledgerwallet-tezos >= "0.2.1" & < "0.4.0"
  241. letters
  242. lmdb >= "1.0"
  243. logical
  244. logtk >= "1.6"
  245. lp
  246. lp-glpk
  247. lp-glpk-js
  248. lp-gurobi
  249. lru
  250. lt-code
  251. luv
  252. mbr-format >= "1.0.0"
  253. mdx >= "1.6.0"
  254. mec
  255. mechaml >= "1.0.0"
  256. merge-queues >= "0.2.0"
  257. merge-ropes >= "0.2.0"
  258. metrics
  259. middleware
  260. mimic
  261. minicaml = "0.3.1" | >= "0.4"
  262. mirage >= "4.0.0~beta1"
  263. mirage-block-partition
  264. mirage-block-ramdisk >= "0.3"
  265. mirage-channel >= "4.0.0"
  266. mirage-channel-lwt
  267. mirage-crypto-ec
  268. mirage-flow >= "1.0.2" & < "1.2.0"
  269. mirage-flow-unix
  270. mirage-fs-mem
  271. mirage-fs-unix >= "1.2.0"
  272. mirage-kv >= "2.0.0"
  273. mirage-kv-mem
  274. mirage-kv-unix
  275. mirage-logs >= "0.3.0"
  276. mirage-nat
  277. mirage-net-unix >= "2.3.0"
  278. mirage-runtime >= "4.0.0~beta1" & < "4.5.0"
  279. mirage-tc
  280. mjson
  281. mmdb
  282. mnd
  283. monocypher
  284. mrmime >= "0.2.0"
  285. mrt-format
  286. msgpck >= "1.6"
  287. mssql >= "2.0.3"
  288. multibase
  289. multihash
  290. multihash-digestif
  291. multipart-form-data
  292. multipart_form
  293. multipart_form-eio
  294. multipart_form-lwt
  295. named-pipe
  296. nanoid
  297. nbd >= "4.0.3"
  298. nbd-tool
  299. nloge
  300. nocoiner
  301. non_empty_list
  302. OCADml >= "0.6.0"
  303. ocaml-r >= "0.4.0"
  304. ocaml-version >= "3.1.0"
  305. ocamlformat >= "0.13.0" & != "0.19.0~4.13preview" & < "0.25.1"
  306. ocamlformat-rpc < "removed"
  307. ocamline
  308. ocluster < "0.3.0"
  309. odoc >= "1.4.0" & < "2.1.0"
  310. ohex
  311. oidc
  312. opam-0install
  313. opam-compiler
  314. opam-file-format >= "2.1.1"
  315. opentelemetry >= "0.6"
  316. opentelemetry-client-cohttp-lwt >= "0.6"
  317. opentelemetry-client-ocurl >= "0.6"
  318. opentelemetry-cohttp-lwt >= "0.6"
  319. opentelemetry-lwt >= "0.6"
  320. opium >= "0.15.0"
  321. opium-graphql
  322. opium-testing
  323. opium_kernel
  324. orewa
  325. ortac-core
  326. osx-acl
  327. osx-attr
  328. osx-cf
  329. osx-fsevents
  330. osx-membership
  331. osx-mount
  332. osx-xattr
  333. otoggl
  334. owl >= "0.6.0" & != "0.9.0" & != "1.0.0"
  335. owl-base < "0.5.0"
  336. owl-ode >= "0.1.0" & != "0.2.0"
  337. owl-symbolic
  338. passmaker
  339. patch
  340. pbkdf
  341. pecu >= "0.2"
  342. pf-qubes
  343. pg_query >= "0.9.6"
  344. pgx >= "1.0"
  345. pgx_unix >= "1.0"
  346. pgx_value_core
  347. pgx_value_ptime < "2.2"
  348. phylogenetics
  349. piaf
  350. polyglot
  351. polynomial
  352. ppx_blob >= "0.3.0"
  353. ppx_deriving_cmdliner
  354. ppx_deriving_rpc
  355. ppx_deriving_yaml
  356. ppx_graphql >= "0.2.0"
  357. ppx_inline_alcotest
  358. ppx_parser
  359. ppx_protocol_conv >= "5.0.0"
  360. ppx_protocol_conv_json >= "5.0.0"
  361. ppx_protocol_conv_jsonm >= "5.0.0"
  362. ppx_protocol_conv_msgpack >= "5.0.0"
  363. ppx_protocol_conv_xml_light >= "5.0.0"
  364. ppx_protocol_conv_xmlm
  365. ppx_protocol_conv_yaml >= "5.0.0"
  366. ppx_repr < "0.4.0"
  367. ppx_subliner
  368. ppx_units
  369. ppx_yojson >= "1.1.0"
  370. pratter
  371. prc
  372. preface
  373. pretty_expressive
  374. prettym
  375. proc-smaps
  376. producer < "0.2.0"
  377. progress < "0.2.0"
  378. prom
  379. prometheus < "1.2"
  380. prometheus-app
  381. protocell
  382. protocol-9p >= "0.3" & < "0.11.0" | >= "0.11.2"
  383. protocol-9p-unix
  384. psq
  385. qcheck >= "0.18"
  386. qcheck-alcotest
  387. qcheck-core >= "0.18"
  388. quickjs
  389. radis
  390. randii
  391. reason-standard
  392. reparse >= "2.0.0" & < "3.0.0"
  393. reparse-unix < "2.1.0"
  394. resp
  395. resp-unix >= "0.10.0"
  396. rfc1951 < "1.0.0"
  397. routes < "2.0.0"
  398. rpc >= "7.1.0"
  399. rpclib >= "7.1.0"
  400. rpclib-async
  401. rpclib-lwt >= "7.1.0"
  402. rubytt
  403. SZXX >= "4.0.0"
  404. salsa20
  405. salsa20-core
  406. sanddb >= "0.2"
  407. scaml >= "1.5.0"
  408. scrypt-kdf
  409. secp256k1 >= "0.4.1"
  410. secp256k1-internal
  411. semver >= "0.2.1"
  412. sendmail
  413. sendmail-lwt
  414. sendmsg
  415. server-reason-react
  416. session-cookie
  417. session-cookie-async
  418. session-cookie-lwt
  419. sherlodoc
  420. sihl < "0.2.0"
  421. sihl-type
  422. slug
  423. sodium-fmt
  424. solidity-alcotest
  425. spin >= "0.7.0"
  426. squirrel
  427. ssh-agent
  428. ssl >= "0.6.0"
  429. stramon-lib
  430. styled-ppx
  431. syslog-rfc5424
  432. tcpip >= "2.4.2" & < "4.0.0" | >= "5.0.1" & < "7.0.0"
  433. tdigest < "2.1.0"
  434. terminal_size >= "0.1.1"
  435. terminus
  436. terminus-cohttp
  437. terminus-hlc
  438. terml
  439. textrazor
  440. tezos-base-test-helpers < "13.0"
  441. tezos-bls12-381-polynomial
  442. tezos-client-base < "12.0"
  443. tezos-crypto >= "8.0" & < "9.0"
  444. tezos-lmdb
  445. tezos-plompiler = "0.1.3"
  446. tezos-plonk = "0.1.3"
  447. tezos-signer-backends >= "8.0" & < "13.0"
  448. tezos-stdlib >= "8.0" & < "12.0"
  449. tezos-test-helpers < "12.0"
  450. tftp
  451. timedesc
  452. timere
  453. tls >= "0.12.0"
  454. toc
  455. topojson
  456. topojsone
  457. transept
  458. twostep
  459. type_eq
  460. type_id
  461. typebeat
  462. typeid >= "1.0.1"
  463. tyre >= "0.4"
  464. tyxml >= "4.0.0"
  465. tyxml-jsx
  466. tyxml-ppx >= "4.3.0"
  467. tyxml-syntax
  468. uecc
  469. ulid
  470. universal-portal
  471. unix-dirent
  472. unix-errno >= "0.3.0"
  473. unix-fcntl >= "0.3.0"
  474. unix-sys-resource
  475. unix-sys-stat
  476. unix-time
  477. unstrctrd
  478. user-agent-parser
  479. uspf
  480. uspf-lwt
  481. uspf-unix
  482. utop >= "2.13.0"
  483. validate
  484. validator
  485. vercel
  486. vpnkit
  487. wayland >= "2.0"
  488. wcwidth
  489. websocketaf
  490. x509 >= "0.7.0"
  491. xapi-rrd >= "1.8.2"
  492. xapi-stdext-date
  493. xapi-stdext-encodings
  494. xapi-stdext-std >= "4.16.0"
  495. yaml < "3.2.0"
  496. yaml-sexp
  497. yocaml
  498. yocaml_yaml
  499. yojson >= "1.6.0"
  500. yuscii >= "0.3.0"
  501. yuujinchou = "1.0.0"
  502. zar
  503. zed >= "3.2.2"
  504. zlist < "0.4.0"

Conflicts

None

OCaml

Innovation. Community. Security.

GitHub Discord Twitter Peertube RSS
About
Changelog Releases Industrial Users Academic Users Why OCaml
Ecosystem
Packages Community Events OCaml Planet Jobs
Resources
Install OCaml Get Started Platform Tools Language Manual Standard Library API Books Exercises Papers OCaml Playground Logo
Policies
Carbon Footprint Governance Privacy Code of Conduct