package alcotest

  1. Overview
  2. Docs
Alcotest is a lightweight and colourful test framework

Install

Dune Dependency

github.com Readme Changelog ISC License Edit opam file Versions (40)

Authors

  1. T
    Thomas Gazagnaire

Maintainers

  1. T
    thomas@gazagnaire.org

Sources

alcotest-mirage-1.2.2.tbz
sha256=b4bbfdf8fc9597d845ec4024d8a260c6cca2eac51fe166c376a6929576ad051c
sha512=73ba44c028ac70a61bb19bc08d421bc7cd8158faaabc0101b3f4ce83d4faf691f4b6212f73cadabc859984b7509a47df6a7ed27ae2578ad2efd12d2e5f7e331e

Description

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Published: 27 Aug 2020

README

Alcotest is a lightweight and colourful test framework.

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run. See the manpage for details.

For information on contributing to Alcotest, see CONTRIBUTING.md.

Examples

A simple example (taken from examples/simple.ml):

Generated by the following test suite specification:

(* Build with `ocamlbuild -pkg alcotest simple.byte` *)

(* A module with functions to test *)
module To_test = struct
  let lowercase = String.lowercase_ascii
  let capitalize = String.capitalize_ascii
  let str_concat = String.concat ""
  let list_concat = List.append
end

(* The tests *)
let test_lowercase () =
  Alcotest.(check string) "same string" "hello!" (To_test.lowercase "hELLO!")

let test_capitalize () =
  Alcotest.(check string) "same string" "World." (To_test.capitalize "world.")

let test_str_concat () =
  Alcotest.(check string) "same string" "foobar" (To_test.str_concat ["foo"; "bar"])

let test_list_concat () =
  Alcotest.(check (list int)) "same lists" [1; 2; 3] (To_test.list_concat [1] [2; 3])

(* Run it *)
let () =
  let open Alcotest in
  run "Utils" [
      "string-case", [
          test_case "Lower case"     `Quick test_lowercase;
          test_case "Capitalization" `Quick test_capitalize;
        ];
      "string-concat", [ test_case "String mashing" `Quick test_str_concat  ];
      "list-concat",   [ test_case "List mashing"   `Slow  test_list_concat ];
    ]

The result is a self-contained binary which displays the test results. Use dune exec examples/simple.exe -- --help to see the runtime options.

Here's an example of a of failing test suite:

By default, only the first failing test log is printed to the console (and all test logs are captured on disk). Pass --show-errors to print all error messages.

Selecting tests to execute

You can filter which tests to run by supplying a regular expression matching the names of the tests to execute, or by passing a regular expression and a comma-separated list of test numbers (or ranges of test numbers, e.g. 2,4..9):

$ ./simple.native test '.*concat*'
Testing Utils.
[SKIP]     string-case            0   Lower case.
[SKIP]     string-case            1   Capitalization.
[OK]       string-concat          0   String mashing.
[OK]       list-concat            0   List mashing.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 tests run.

$ ./simple.native test 'string-case' '1..3'
Testing Utils.
[SKIP]     string-case            0   Lower case.
[OK]       string-case            1   Capitalization.
[SKIP]     string-concat          0   String mashing.
[SKIP]     list-concat            0   List mashing.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.

Note that you cannot filter by test case name (i.e. Lower case or Capitalization), you must filter by test name & number instead.

See the examples folder for more examples.

Quick and Slow tests

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

Passing custom options to the tests

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your prefered number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ test.exe test
test.exe: required option -n is missing

$ test.exe test -n 42
Testing foo.
[OK]                all          0   int.

Lwt

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type unit -> unit Lwt.t and alcotest-lwt calls Lwt_main.run for you.

However, there are a couple of extra features:

  • If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).

  • You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Lwt_main.run @@ Alcotest_lwt.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ test.exe
Testing foo.
[ERROR]             all          0   one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken

Comparison with other testing frameworks

The README is pretty clear about that:

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

  • Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;

  • Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

  • qcheck qcheck does random generation and property testing (e.g. Quick Check)

  • crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.

  • ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.

Dependencies (9)

  1. uutf >= "1.0.0"
  2. stdlib-shims
  3. re >= "1.7.2"
  4. uuidm
  5. cmdliner >= "1.0.3" & < "1.1.0"
  6. astring
  7. fmt >= "0.8.7"
  8. ocaml >= "4.03.0"
  9. dune >= "2.0"

Dev Dependencies

None

  1. ahrocksdb
  2. albatross >= "1.5.0"
  3. alcotest-async < "1.0.0" | = "1.2.2"
  4. alcotest-lwt < "1.0.0" | = "1.2.2"
  5. alcotest-mirage = "1.2.2"
  6. alg_structs_qcheck
  7. ambient-context
  8. ambient-context-eio
  9. ambient-context-lwt
  10. angstrom >= "0.7.0"
  11. ansi >= "0.6.0"
  12. anycache >= "0.7.4"
  13. anycache-async
  14. anycache-lwt
  15. archetype >= "1.4.2"
  16. archi
  17. arp
  18. arp-mirage
  19. arrakis
  20. art
  21. asak >= "0.2"
  22. asli >= "0.2.0"
  23. asn1-combinators >= "0.2.2"
  24. atd >= "2.3.3"
  25. atdgen >= "2.10.0"
  26. atdpy
  27. atdts
  28. base32
  29. base64 >= "2.1.2" & < "3.2.0" | >= "3.4.0"
  30. bastet
  31. bastet_async
  32. bastet_lwt
  33. bech32
  34. bechamel >= "0.5.0"
  35. bigarray-overlap
  36. bigstringaf
  37. bitlib
  38. blake2
  39. bloomf
  40. bls12-381 < "0.4.1" | >= "3.0.0" & < "18.0"
  41. bls12-381-hash
  42. bls12-381-js >= "0.4.2"
  43. bls12-381-js-gen >= "0.4.2"
  44. bls12-381-legacy
  45. bls12-381-signature
  46. bls12-381-unix
  47. blurhash
  48. builder-web
  49. bulletml
  50. bytebuffer
  51. ca-certs
  52. ca-certs-nss
  53. cactus
  54. caldav
  55. calendar >= "3.0.0"
  56. callipyge
  57. camlix
  58. camlkit
  59. camlkit-base
  60. capnp-rpc < "1.2.3"
  61. capnp-rpc-lwt < "0.3"
  62. capnp-rpc-mirage >= "0.9.0"
  63. capnp-rpc-unix >= "0.9.0" & < "1.2.3"
  64. carray
  65. carton
  66. cborl
  67. ccss >= "1.6"
  68. cf-lwt
  69. chacha
  70. channel
  71. charrua-client
  72. charrua-client-lwt
  73. charrua-client-mirage < "0.11.0"
  74. checked_oint < "0.1.1"
  75. checkseum >= "0.0.3"
  76. cid
  77. clarity-lang
  78. class_group_vdf
  79. cohttp >= "0.17.0"
  80. cohttp-curl-async
  81. cohttp-curl-lwt
  82. cohttp-eio >= "6.0.0~beta2"
  83. colombe >= "0.2.0"
  84. color
  85. conan
  86. conan-cli
  87. conan-database
  88. conan-lwt
  89. conan-unix
  90. conduit = "3.0.0"
  91. conex < "0.10.0"
  92. conex-mirage-crypto
  93. conex-nocrypto
  94. cookie
  95. cow >= "2.2.0"
  96. css
  97. css-parser
  98. cstruct >= "3.3.0"
  99. cstruct-sexp
  100. ctypes-zarith
  101. cuid
  102. curly
  103. current >= "0.4"
  104. current_git >= "0.6.4"
  105. current_incr
  106. cwe_checker
  107. data-encoding
  108. datakit >= "0.12.0"
  109. datakit-bridge-github >= "0.12.0"
  110. datakit-ci
  111. datakit-client-git >= "0.12.0"
  112. decompress >= "0.8" & < "1.5.3"
  113. depyt
  114. digestif >= "0.8.1"
  115. dispatch >= "0.4.1"
  116. dkim
  117. dkim-bin
  118. dkim-mirage
  119. dns >= "4.0.0"
  120. dns-cli
  121. dns-client >= "4.6.0"
  122. dns-forward < "0.9.0"
  123. dns-forward-lwt-unix
  124. dns-resolver
  125. dns-server
  126. dns-tsig
  127. dnssd
  128. dnssec
  129. docfd >= "2.2.0"
  130. dog < "0.2.1"
  131. domain-name
  132. dream
  133. dream-pure
  134. duff
  135. dune-release >= "1.0.0"
  136. duration >= "0.1.1"
  137. emile
  138. encore
  139. eqaf >= "0.5"
  140. equinoxe
  141. equinoxe-cohttp
  142. equinoxe-hlc
  143. eris
  144. eris-lwt
  145. ezgzip
  146. ezjsonm >= "0.4.2" & < "1.3.0"
  147. ezjsonm-lwt
  148. FPauth
  149. FPauth-core
  150. FPauth-responses
  151. FPauth-strategies
  152. faraday != "0.2.0"
  153. farfadet
  154. fat-filesystem >= "0.12.0"
  155. ff
  156. ff-pbt
  157. fiat-p256
  158. flex-array
  159. fsevents-lwt
  160. functoria >= "2.2.0"
  161. functoria-runtime >= "2.2.0" & != "3.0.1" & < "4.0.0~beta1"
  162. geojson
  163. geoml >= "0.1.1"
  164. git = "1.4.10" | = "1.5.0" | >= "1.5.2" & != "1.10.0"
  165. git-mirage < "3.0.0"
  166. git-unix >= "1.10.0" & != "2.1.0"
  167. gitlab-unix
  168. glicko2
  169. gmap >= "0.3.0"
  170. gobba
  171. gpt
  172. graphql
  173. graphql-async
  174. graphql-cohttp >= "0.13.0"
  175. graphql-lwt
  176. graphql_parser != "0.11.0"
  177. graphql_ppx >= "0.7.1"
  178. h1_parser
  179. h2
  180. hacl
  181. hacl-star >= "0.6.0"
  182. hacl_func
  183. hacl_x25519 >= "0.2.0"
  184. highlexer
  185. hkdf
  186. hockmd
  187. html_of_jsx
  188. http
  189. http-multipart-formdata < "2.0.0"
  190. httpaf >= "0.2.0"
  191. hvsock
  192. icalendar >= "0.1.4"
  193. imagelib >= "20200929"
  194. index
  195. inferno >= "20220603"
  196. influxdb-async
  197. influxdb-lwt
  198. inquire < "0.2.0"
  199. interval-map
  200. iomux
  201. irmin < "0.8.0" | >= "0.9.6" & != "0.11.1" & < "1.0.0" | >= "2.0.0" & != "2.3.0"
  202. irmin-bench >= "2.7.0"
  203. irmin-chunk < "1.3.0" | >= "2.3.0"
  204. irmin-cli
  205. irmin-containers
  206. irmin-fs < "1.3.0" | >= "2.3.0"
  207. irmin-git < "2.0.0" | >= "2.3.0"
  208. irmin-http < "2.0.0"
  209. irmin-mem < "1.3.0"
  210. irmin-pack >= "2.4.0" & != "2.6.1"
  211. irmin-pack-tools
  212. irmin-test >= "2.2.0" & < "3.0.0"
  213. irmin-tezos
  214. irmin-tezos-utils
  215. irmin-unix >= "1.0.0" & < "1.3.3" | >= "2.4.0" & != "2.6.1"
  216. irmin-watcher
  217. jekyll-format
  218. jerboa
  219. jitsu
  220. jose
  221. json-data-encoding >= "0.9"
  222. json_decoder
  223. jsonxt
  224. junit_alcotest
  225. jwto
  226. ke >= "0.2"
  227. kkmarkdown
  228. lambda-runtime
  229. lambda_streams
  230. lambda_streams_async
  231. lambdapi >= "2.0.0"
  232. lambdoc >= "1.0-beta4"
  233. ledgerwallet-tezos >= "0.2.1" & < "0.4.0"
  234. letters
  235. lmdb >= "1.0"
  236. logical
  237. logtk >= "1.6"
  238. lp
  239. lp-glpk
  240. lp-glpk-js
  241. lp-gurobi
  242. lru
  243. lt-code
  244. luv
  245. mbr-format >= "1.0.0"
  246. mdx >= "1.6.0"
  247. mec
  248. mechaml >= "1.0.0"
  249. merge-queues >= "0.2.0"
  250. merge-ropes >= "0.2.0"
  251. metrics
  252. middleware
  253. minicaml = "0.3.1" | >= "0.4"
  254. mirage >= "4.0.0~beta1"
  255. mirage-block-partition
  256. mirage-block-ramdisk >= "0.3"
  257. mirage-channel >= "4.0.0"
  258. mirage-channel-lwt
  259. mirage-crypto-ec
  260. mirage-flow >= "1.0.2" & < "1.2.0"
  261. mirage-flow-unix
  262. mirage-fs-mem
  263. mirage-fs-unix >= "1.2.0"
  264. mirage-kv >= "2.0.0"
  265. mirage-kv-mem
  266. mirage-kv-unix
  267. mirage-logs >= "0.3.0"
  268. mirage-nat
  269. mirage-net-unix >= "2.3.0"
  270. mirage-runtime >= "4.0.0~beta1" & < "4.5.0"
  271. mirage-tc
  272. mjson
  273. mmdb
  274. mnd
  275. monocypher
  276. mrmime >= "0.2.0"
  277. mrt-format
  278. msgpck >= "1.6"
  279. mssql >= "2.0.3"
  280. multibase
  281. multihash
  282. multihash-digestif
  283. multipart-form-data
  284. multipart_form
  285. multipart_form-eio
  286. multipart_form-lwt
  287. named-pipe
  288. nanoid
  289. nbd >= "4.0.3"
  290. nbd-tool
  291. nloge
  292. nocoiner
  293. non_empty_list
  294. OCADml >= "0.6.0"
  295. ocaml-r >= "0.4.0"
  296. ocaml-version >= "3.1.0"
  297. ocamlformat >= "0.13.0" & != "0.19.0~4.13preview" & < "0.25.1"
  298. ocamlformat-rpc < "removed"
  299. ocamline
  300. ocluster < "0.3.0"
  301. odoc >= "1.4.0" & < "2.1.0"
  302. ohex
  303. oidc
  304. opam-0install
  305. opam-compiler
  306. opam-file-format >= "2.1.1"
  307. opentelemetry >= "0.6"
  308. opentelemetry-client-cohttp-lwt >= "0.6"
  309. opentelemetry-client-ocurl >= "0.6"
  310. opentelemetry-cohttp-lwt >= "0.6"
  311. opentelemetry-lwt >= "0.6"
  312. opium >= "0.15.0"
  313. opium-graphql
  314. opium-testing
  315. opium_kernel
  316. orewa
  317. ortac-core
  318. osx-acl
  319. osx-attr
  320. osx-cf
  321. osx-fsevents
  322. osx-membership
  323. osx-mount
  324. osx-xattr
  325. otoggl
  326. owl >= "0.6.0" & != "0.9.0" & != "1.0.0"
  327. owl-base < "0.5.0"
  328. owl-ode >= "0.1.0" & != "0.2.0"
  329. owl-symbolic
  330. passmaker
  331. patch
  332. pbkdf
  333. pecu >= "0.2"
  334. pf-qubes
  335. pg_query >= "0.9.6"
  336. pgx >= "1.0"
  337. pgx_unix >= "1.0"
  338. pgx_value_core
  339. pgx_value_ptime < "2.2"
  340. phylogenetics
  341. piaf
  342. polyglot
  343. polynomial
  344. ppx_blob >= "0.3.0"
  345. ppx_deriving_cmdliner
  346. ppx_deriving_rpc
  347. ppx_deriving_yaml
  348. ppx_graphql >= "0.2.0"
  349. ppx_inline_alcotest
  350. ppx_parser
  351. ppx_protocol_conv >= "5.0.0"
  352. ppx_protocol_conv_json >= "5.0.0"
  353. ppx_protocol_conv_jsonm >= "5.0.0"
  354. ppx_protocol_conv_msgpack >= "5.0.0"
  355. ppx_protocol_conv_xml_light >= "5.0.0"
  356. ppx_protocol_conv_xmlm
  357. ppx_protocol_conv_yaml >= "5.0.0"
  358. ppx_repr < "0.4.0"
  359. ppx_subliner
  360. ppx_units
  361. ppx_yojson >= "1.1.0"
  362. pratter
  363. prc
  364. preface
  365. pretty_expressive
  366. prettym
  367. proc-smaps
  368. producer < "0.2.0"
  369. progress < "0.2.0"
  370. prom
  371. prometheus < "1.2"
  372. prometheus-app
  373. protocell
  374. protocol-9p >= "0.3" & < "0.11.0" | >= "0.11.2"
  375. protocol-9p-unix
  376. psq
  377. qcheck >= "0.18"
  378. qcheck-alcotest
  379. qcheck-core >= "0.18"
  380. quickjs
  381. radis
  382. randii
  383. reason-standard
  384. reparse >= "2.0.0" & < "3.0.0"
  385. reparse-unix < "2.1.0"
  386. resp
  387. resp-unix >= "0.10.0"
  388. rfc1951 < "1.0.0"
  389. routes < "2.0.0"
  390. rpc >= "7.1.0"
  391. rpclib >= "7.1.0"
  392. rpclib-async
  393. rpclib-lwt >= "7.1.0"
  394. rubytt
  395. SZXX >= "4.0.0"
  396. salsa20
  397. salsa20-core
  398. sanddb >= "0.2"
  399. scaml >= "1.5.0"
  400. scrypt-kdf
  401. secp256k1 >= "0.4.1"
  402. secp256k1-internal
  403. semver >= "0.2.1"
  404. sendmail
  405. sendmail-lwt
  406. sendmsg
  407. server-reason-react
  408. session-cookie
  409. session-cookie-async
  410. session-cookie-lwt
  411. sherlodoc
  412. sihl < "0.2.0"
  413. sihl-type
  414. slug
  415. sodium-fmt
  416. solidity-alcotest
  417. spin >= "0.7.0"
  418. squirrel
  419. ssh-agent
  420. ssl >= "0.6.0"
  421. stramon-lib
  422. styled-ppx
  423. syslog-rfc5424
  424. tcpip >= "2.4.2" & < "4.0.0" | >= "5.0.1" & < "7.0.0"
  425. tdigest < "2.1.0"
  426. terminal_size >= "0.1.1"
  427. terminus
  428. terminus-cohttp
  429. terminus-hlc
  430. terml
  431. textrazor
  432. tezos-base-test-helpers < "13.0"
  433. tezos-bls12-381-polynomial
  434. tezos-client-base < "12.0"
  435. tezos-crypto >= "8.0" & < "9.0"
  436. tezos-lmdb
  437. tezos-plompiler = "0.1.3"
  438. tezos-plonk = "0.1.3"
  439. tezos-signer-backends >= "8.0" & < "13.0"
  440. tezos-stdlib >= "8.0" & < "12.0"
  441. tezos-test-helpers < "12.0"
  442. tftp
  443. timedesc
  444. timere
  445. tls >= "0.12.0"
  446. toc
  447. topojson
  448. topojsone
  449. transept
  450. twostep
  451. type_eq
  452. type_id
  453. typebeat
  454. typeid >= "1.0.1"
  455. tyre >= "0.4"
  456. tyxml >= "4.0.0"
  457. tyxml-jsx
  458. tyxml-ppx >= "4.3.0"
  459. tyxml-syntax
  460. uecc
  461. ulid
  462. universal-portal
  463. unix-dirent
  464. unix-errno >= "0.3.0"
  465. unix-fcntl >= "0.3.0"
  466. unix-sys-resource
  467. unix-sys-stat
  468. unix-time
  469. unstrctrd
  470. user-agent-parser
  471. uspf
  472. uspf-lwt
  473. uspf-unix
  474. utop >= "2.13.0"
  475. validate
  476. validator
  477. vercel
  478. vpnkit
  479. wcwidth
  480. websocketaf
  481. x509 >= "0.7.0"
  482. xapi-rrd >= "1.8.2"
  483. xapi-stdext-date
  484. xapi-stdext-encodings
  485. xapi-stdext-std >= "4.16.0"
  486. yaml < "3.2.0"
  487. yaml-sexp
  488. yocaml
  489. yocaml_yaml
  490. yojson >= "1.6.0"
  491. yuscii >= "0.3.0"
  492. yuujinchou = "1.0.0"
  493. zar
  494. zed >= "3.2.2"
  495. zlist < "0.4.0"

Conflicts

None

OCaml

Innovation. Community. Security.

GitHub Discord Twitter Peertube RSS
About
Changelog Releases Industrial Users Academic Users Why OCaml
Ecosystem
Packages Community Events OCaml Planet Jobs
Resources
Install OCaml Get Started Platform Tools Language Manual Standard Library API Books Exercises Papers OCaml Playground Logo
Policies
Carbon Footprint Governance Privacy Code of Conduct