package alcotest

  1. Overview
  2. Docs
Alcotest is a lightweight and colourful test framework

Install

Dune Dependency

Authors

Maintainers

Sources

alcotest-js-1.5.0.tbz
sha256=54281907e02d78995df246dc2e10ed182828294ad2059347a1e3a13354848f6c
sha512=1aea91de40795ec4f6603d510107e4b663c1a94bd223f162ad231316d8595e9e098cabbe28a46bdcb588942f3d103d8377373d533bcc7413ba3868a577469b45

Description

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Published: 12 Oct 2021

README

README.md

A lightweight and colourful test framework.


Alcotest exposes a simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run. See the manpage for details.

The API documentation can be found [here][docs]. For information on contributing to Alcotest, see CONTRIBUTING.md.


Examples

A simple example (taken from examples/simple.ml):

Generated by the following test suite specification:

(* Build with `ocamlbuild -pkg alcotest simple.byte` *)

(* A module with functions to test *)
module To_test = struct
  let lowercase = String.lowercase_ascii
  let capitalize = String.capitalize_ascii
  let str_concat = String.concat ""
  let list_concat = List.append
end

(* The tests *)
let test_lowercase () =
  Alcotest.(check string) "same string" "hello!" (To_test.lowercase "hELLO!")

let test_capitalize () =
  Alcotest.(check string) "same string" "World." (To_test.capitalize "world.")

let test_str_concat () =
  Alcotest.(check string) "same string" "foobar" (To_test.str_concat ["foo"; "bar"])

let test_list_concat () =
  Alcotest.(check (list int)) "same lists" [1; 2; 3] (To_test.list_concat [1] [2; 3])

(* Run it *)
let () =
  let open Alcotest in
  run "Utils" [
      "string-case", [
          test_case "Lower case"     `Quick test_lowercase;
          test_case "Capitalization" `Quick test_capitalize;
        ];
      "string-concat", [ test_case "String mashing" `Quick test_str_concat  ];
      "list-concat",   [ test_case "List mashing"   `Slow  test_list_concat ];
    ]

The result is a self-contained binary which displays the test results. Use dune exec examples/simple.exe -- --help to see the runtime options.

Here's an example of a of failing test suite:

By default, only the first failing test log is printed to the console (and all test logs are captured on disk). Pass --show-errors to print all error messages.

Selecting tests to execute

You can filter which tests to run by supplying a regular expression matching the names of the tests to execute, or by passing a regular expression and a comma-separated list of test numbers (or ranges of test numbers, e.g. 2,4..9):

$ ./simple.native test '.*concat*'
Testing Utils.
[SKIP]     string-case            0   Lower case.
[SKIP]     string-case            1   Capitalization.
[OK]       string-concat          0   String mashing.
[OK]       list-concat            0   List mashing.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 tests run.

$ ./simple.native test 'string-case' '1..3'
Testing Utils.
[SKIP]     string-case            0   Lower case.
[OK]       string-case            1   Capitalization.
[SKIP]     string-concat          0   String mashing.
[SKIP]     list-concat            0   List mashing.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.

Note that you cannot filter by test case name (i.e. Lower case or Capitalization), you must filter by test name & number instead.

See the examples folder for more examples.

Quick and Slow tests

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

Passing custom options to the tests

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your prefered number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ test.exe test
test.exe: required option -n is missing

$ test.exe test -n 42
Testing foo.
[OK]                all          0   int.

Lwt

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type unit -> unit Lwt.t and alcotest-lwt calls Lwt_main.run for you.

However, there are a couple of extra features:

  • If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).

  • You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Lwt_main.run @@ Alcotest_lwt.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ test.exe
Testing foo.
[ERROR]             all          0   one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken

Comparison with other testing frameworks

The README is pretty clear about that:

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

  • Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;

  • Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

  • qcheck qcheck does random generation and property testing (e.g. Quick Check)

  • crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.

  • ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.

Dependencies (9)

  1. ocaml-syntax-shims
  2. uutf >= "1.0.1"
  3. stdlib-shims
  4. re >= "1.7.2"
  5. cmdliner >= "1.0.0"
  6. astring
  7. fmt >= "0.8.7"
  8. ocaml >= "4.03.0"
  9. dune >= "2.8"

Dev Dependencies (2)

  1. odoc with-doc
  2. cmdliner with-test & < "1.1.0"

  1. ahrocksdb
  2. albatross >= "1.5.4"
  3. alcotest-async < "1.7.0"
  4. alg_structs_qcheck
  5. algaeff
  6. ambient-context
  7. ambient-context-eio
  8. ambient-context-lwt
  9. angstrom >= "0.7.0"
  10. ansi >= "0.6.0"
  11. anycache >= "0.7.4"
  12. anycache-async
  13. anycache-lwt
  14. archetype >= "1.4.2"
  15. archi
  16. arp
  17. arrakis < "1.1.0"
  18. art
  19. asai
  20. asak >= "0.2"
  21. asli >= "0.2.0"
  22. asn1-combinators >= "0.2.5"
  23. atd >= "2.3.3"
  24. atdgen >= "2.10.0"
  25. atdpy
  26. atdts
  27. base32
  28. base64 >= "2.1.2" & < "3.2.0" | >= "3.4.0"
  29. bastet
  30. bastet_async
  31. bastet_lwt
  32. bech32
  33. bechamel >= "0.5.0"
  34. bigarray-overlap
  35. bigstringaf
  36. bitlib
  37. blake2
  38. bloomf
  39. bls12-381 < "0.4.1" | >= "3.0.0" & < "18.0"
  40. bls12-381-hash
  41. bls12-381-js >= "0.4.2"
  42. bls12-381-js-gen >= "0.4.2"
  43. bls12-381-legacy
  44. bls12-381-signature
  45. bls12-381-unix
  46. blurhash
  47. brisk-reconciler
  48. builder-web
  49. bytebuffer
  50. ca-certs
  51. ca-certs-nss
  52. cactus
  53. caldav
  54. calendar >= "3.0.0"
  55. callipyge
  56. camlix
  57. camlkit
  58. camlkit-base
  59. capnp-rpc < "1.2.3"
  60. capnp-rpc-unix < "1.2.3"
  61. caqti >= "1.7.0"
  62. caqti-async >= "1.7.0"
  63. caqti-driver-mariadb >= "1.7.0"
  64. caqti-driver-postgresql >= "1.7.0"
  65. caqti-driver-sqlite3 >= "1.7.0"
  66. caqti-dynload >= "2.0.1"
  67. caqti-eio
  68. caqti-lwt >= "1.7.0"
  69. caqti-miou
  70. carray
  71. carton < "1.0.0"
  72. carton-git
  73. carton-lwt >= "0.4.3" & < "1.0.0"
  74. catala >= "0.6.0"
  75. cborl
  76. cf-lwt
  77. chacha
  78. chamelon
  79. chamelon-unix
  80. charrua-client
  81. charrua-server
  82. checkseum >= "0.0.3"
  83. cid
  84. clarity-lang
  85. class_group_vdf
  86. cohttp < "6.0.0"
  87. cohttp-curl-async < "6.1.0"
  88. cohttp-eio = "6.0.0~beta2"
  89. colombe >= "0.2.0"
  90. color
  91. commons
  92. conan
  93. conan-cli
  94. conan-database
  95. conan-lwt
  96. conan-unix
  97. conex < "0.10.0"
  98. conex-mirage-crypto
  99. conformist
  100. cookie
  101. cow >= "2.2.0"
  102. css
  103. css-parser
  104. cstruct
  105. cstruct-sexp
  106. ctypes-zarith
  107. cuid
  108. curly
  109. current
  110. current-albatross-deployer
  111. current_git >= "0.7.1"
  112. current_incr
  113. data-encoding
  114. dates_calc
  115. dbase4
  116. decimal >= "0.3.0"
  117. decompress < "1.5.3"
  118. depyt
  119. digestif >= "0.9.0"
  120. dirsp-exchange-kbb2017
  121. dirsp-proscript-mirage
  122. dirsp-ps2ocaml
  123. dispatch >= "0.4.1"
  124. dkim
  125. dkim-bin
  126. dkim-mirage
  127. dkml-dune-dsl-show
  128. dkml-install
  129. dkml-install-installer
  130. dkml-install-runner
  131. dkml-package-console
  132. dns >= "4.4.1"
  133. dns-cli
  134. dns-client >= "4.6.3"
  135. dns-forward-lwt-unix
  136. dns-resolver
  137. dns-server
  138. dns-tsig
  139. dnssd
  140. dnssec
  141. docfd >= "2.2.0"
  142. domain-name
  143. dream
  144. dream-pure
  145. duff
  146. dune-deps >= "1.4.0"
  147. dune-release >= "1.0.0"
  148. duration
  149. echo
  150. eio < "0.12"
  151. eio_linux < "0.12"
  152. eio_windows < "0.12"
  153. emile
  154. encore
  155. eqaf >= "0.5"
  156. equinoxe
  157. equinoxe-cohttp
  158. equinoxe-hlc
  159. ezgzip
  160. ezjsonm
  161. ezjsonm-lwt
  162. FPauth
  163. FPauth-core
  164. FPauth-responses
  165. FPauth-strategies
  166. faraday != "0.2.0"
  167. farfadet
  168. fat-filesystem
  169. ff
  170. ff-pbt
  171. flex-array
  172. fsevents-lwt
  173. functoria
  174. fungi
  175. geojson
  176. geoml >= "0.1.1"
  177. git
  178. git-cohttp
  179. git-cohttp-unix
  180. git-kv >= "0.2.0"
  181. git-mirage
  182. git-net
  183. git-split
  184. git-unix
  185. gitlab-unix
  186. glicko2
  187. gmap
  188. gobba
  189. gpt
  190. graphql
  191. graphql-async
  192. graphql-cohttp >= "0.13.0"
  193. graphql-lwt
  194. graphql_parser != "0.11.0"
  195. graphql_ppx
  196. h1
  197. h1_parser
  198. h2
  199. hacl
  200. hacl-star >= "0.6.0" & < "0.7.2"
  201. hacl_func
  202. hacl_x25519
  203. highlexer
  204. hkdf
  205. hockmd
  206. html_of_jsx
  207. http < "6.0.0"
  208. http-multipart-formdata < "2.0.0"
  209. httpaf >= "0.2.0"
  210. httpun
  211. httpun-ws
  212. hvsock
  213. icalendar
  214. imagelib
  215. index
  216. inferno >= "20220603"
  217. influxdb-async
  218. influxdb-lwt
  219. inquire < "0.2.0"
  220. interval-map
  221. iomux
  222. irmin
  223. irmin-bench
  224. irmin-chunk
  225. irmin-cli
  226. irmin-containers
  227. irmin-fs
  228. irmin-git
  229. irmin-graphql
  230. irmin-pack
  231. irmin-pack-tools
  232. irmin-test < "3.6.1"
  233. irmin-tezos
  234. irmin-unix
  235. irmin-watcher
  236. jekyll-format
  237. jose
  238. json-data-encoding >= "0.9"
  239. json_decoder
  240. jsonxt
  241. junit_alcotest < "2.1.0"
  242. jwto
  243. kdf
  244. ke >= "0.2"
  245. kkmarkdown
  246. kmt
  247. lambda-runtime
  248. lambda_streams
  249. lambda_streams_async
  250. lambdapi
  251. ledgerwallet-tezos >= "0.2.1" & < "0.4.0"
  252. letters
  253. lmdb >= "1.0"
  254. logical
  255. logtk >= "1.6"
  256. lp
  257. lp-glpk
  258. lp-glpk-js < "0.5.0"
  259. lp-gurobi < "0.5.0"
  260. lru
  261. lt-code
  262. luv
  263. mbr-format
  264. mdx >= "1.6.0"
  265. mec
  266. mechaml >= "1.2.1"
  267. merlin = "4.17.1-501"
  268. merlin-lib >= "4.17.1-501"
  269. metrics
  270. middleware
  271. mimic
  272. minicaml = "0.3.1" | >= "0.4"
  273. mirage >= "4.0.0"
  274. mirage-block-partition
  275. mirage-block-ramdisk
  276. mirage-channel >= "4.0.1"
  277. mirage-crypto-ec
  278. mirage-flow-unix
  279. mirage-kv >= "2.0.0"
  280. mirage-kv-mem
  281. mirage-kv-unix >= "3.0.0"
  282. mirage-logs
  283. mirage-nat
  284. mirage-net-unix
  285. mirage-runtime < "4.7.0"
  286. mirage-tc
  287. mjson
  288. mmdb < "0.3.0"
  289. mnd
  290. mqtt
  291. mrmime >= "0.2.0"
  292. msgpck >= "1.6"
  293. mssql >= "2.0.3"
  294. multibase
  295. multihash
  296. multihash-digestif
  297. multipart-form-data
  298. multipart_form
  299. multipart_form-eio
  300. multipart_form-lwt
  301. named-pipe
  302. nanoid
  303. nbd >= "4.0.3"
  304. nbd-tool
  305. nloge
  306. nocoiner
  307. non_empty_list
  308. OCADml >= "0.6.0"
  309. obatcher
  310. ocaml-index < "5.4.1-503"
  311. ocaml-r >= "0.4.0"
  312. ocaml-version >= "3.5.0"
  313. ocamlformat >= "0.13.0" & < "0.25.1"
  314. ocamlformat-lib
  315. ocamlformat-mlx-lib
  316. ocamlformat-rpc < "removed"
  317. ocamline
  318. ocluster < "0.3.0"
  319. octez-bls12-381-hash
  320. octez-bls12-381-signature
  321. octez-libs
  322. octez-mec
  323. odoc < "2.1.1"
  324. ohex
  325. oidc
  326. opam-0install
  327. opam-0install-cudf >= "0.5.0"
  328. opam-compiler
  329. opam-file-format >= "2.1.1"
  330. opentelemetry >= "0.6"
  331. opentelemetry-client-cohttp-lwt >= "0.6"
  332. opentelemetry-client-ocurl >= "0.6"
  333. opentelemetry-cohttp-lwt >= "0.6"
  334. opentelemetry-lwt >= "0.6"
  335. opium
  336. opium-graphql
  337. opium-testing
  338. opium_kernel
  339. orewa
  340. orgeat
  341. ortac-core
  342. osnap < "0.3.0"
  343. osx-acl
  344. osx-attr
  345. osx-cf
  346. osx-fsevents
  347. osx-membership
  348. osx-mount
  349. osx-xattr
  350. otoggl
  351. owl >= "0.7.0" & != "0.9.0" & != "1.0.0"
  352. owl-base < "0.5.0"
  353. owl-ode >= "0.1.0" & != "0.2.0"
  354. owl-symbolic
  355. passmaker
  356. patch < "3.0.0~alpha2"
  357. pbkdf
  358. pecu >= "0.2"
  359. pf-qubes
  360. pg_query >= "0.9.6"
  361. pgx >= "1.0"
  362. pgx_unix >= "1.0"
  363. pgx_value_core
  364. pgx_value_ptime
  365. phylogenetics
  366. piaf
  367. plebeia >= "2.0.0"
  368. polyglot
  369. polynomial
  370. ppx_blob >= "0.3.0"
  371. ppx_deriving_cmdliner
  372. ppx_deriving_ezjsonm
  373. ppx_deriving_qcheck
  374. ppx_deriving_rpc
  375. ppx_deriving_yaml
  376. ppx_inline_alcotest
  377. ppx_marshal
  378. ppx_parser
  379. ppx_protocol_conv >= "5.0.0"
  380. ppx_protocol_conv_json >= "5.0.0"
  381. ppx_protocol_conv_jsonm >= "5.0.0"
  382. ppx_protocol_conv_msgpack >= "5.0.0"
  383. ppx_protocol_conv_xml_light >= "5.0.0"
  384. ppx_protocol_conv_xmlm
  385. ppx_protocol_conv_yaml >= "5.0.0"
  386. ppx_repr
  387. ppx_subliner
  388. ppx_units
  389. ppx_yojson >= "1.1.0"
  390. pratter
  391. prbnmcn-ucb1 >= "0.0.2"
  392. prc
  393. preface
  394. pretty_expressive
  395. prettym
  396. proc-smaps
  397. producer < "0.2.0"
  398. progress
  399. prom
  400. prometheus < "1.2"
  401. prometheus-app
  402. protocell
  403. protocol-9p < "0.11.0" | >= "0.11.2"
  404. protocol-9p-unix
  405. psq
  406. pyast
  407. qcheck >= "0.25"
  408. qcheck-alcotest
  409. qcheck-core >= "0.25"
  410. quickjs
  411. randii
  412. reason-standard
  413. red-black-tree
  414. reparse >= "2.0.0" & < "3.0.0"
  415. reparse-unix < "2.1.0"
  416. resp
  417. resp-unix >= "0.10.0"
  418. resto >= "0.8"
  419. rfc1951 < "1.0.0"
  420. routes < "2.0.0"
  421. rpc
  422. rpclib
  423. rpclib-async
  424. rpclib-lwt
  425. rpmfile < "0.3.0"
  426. rpmfile-eio
  427. rpmfile-unix
  428. SZXX >= "4.0.0"
  429. salsa20
  430. salsa20-core
  431. sanddb >= "0.2"
  432. scrypt-kdf
  433. secp256k1 >= "0.4.1"
  434. secp256k1-internal
  435. semver >= "0.2.1"
  436. sendmail
  437. sendmail-lwt
  438. sendmail-miou-unix
  439. sendmail-mirage
  440. sendmsg
  441. seqes
  442. server-reason-react
  443. session-cookie
  444. session-cookie-async
  445. session-cookie-lwt
  446. sherlodoc
  447. sihl < "0.2.0"
  448. sihl-type
  449. slug
  450. smaws-clients
  451. smaws-lib
  452. smol
  453. smol-helpers
  454. sodium-fmt
  455. solidity-alcotest
  456. spdx_licenses
  457. spectrum >= "0.2.0"
  458. spin >= "0.7.0"
  459. spurs
  460. squirrel
  461. ssh-agent
  462. ssl >= "0.6.0"
  463. stramon-lib
  464. stringx
  465. styled-ppx
  466. swapfs
  467. syslog-rfc5424
  468. tcpip
  469. tdigest < "2.1.0"
  470. term-indexing
  471. term-tools
  472. terminal
  473. terminal_size >= "0.1.1"
  474. terminus
  475. terminus-cohttp
  476. terminus-hlc
  477. terml
  478. testo
  479. testo-lwt
  480. textmate-language >= "0.3.0"
  481. textrazor
  482. tezos-base-test-helpers < "17.3"
  483. tezos-bls12-381-polynomial
  484. tezos-client-base < "17.3"
  485. tezos-client-base-unix < "17.3"
  486. tezos-crypto >= "16.0" & < "17.3"
  487. tezos-crypto-dal < "17.3"
  488. tezos-error-monad >= "12.3" & < "17.3"
  489. tezos-event-logging-test-helpers < "17.3"
  490. tezos-plompiler = "0.1.3"
  491. tezos-plonk = "0.1.3"
  492. tezos-shell-services >= "16.0" & < "17.3"
  493. tezos-stdlib != "12.3" & < "17.3"
  494. tezos-test-helpers < "17.3"
  495. tezos-version >= "16.0" & < "17.3"
  496. tezos-webassembly-interpreter < "17.3"
  497. timedesc
  498. timere
  499. timmy
  500. timmy-jsoo
  501. timmy-lwt
  502. timmy-unix
  503. tls >= "0.12.8"
  504. toc
  505. topojson
  506. topojsone
  507. traits
  508. transept
  509. tsort >= "2.2.0"
  510. twostep
  511. type_eq
  512. type_id
  513. typeid >= "1.0.1"
  514. tyre >= "0.4"
  515. tyxml >= "4.2.0"
  516. tyxml-jsx
  517. tyxml-ppx >= "4.3.0"
  518. tyxml-syntax
  519. uecc
  520. ulid
  521. universal-portal
  522. unix-dirent
  523. unix-errno
  524. unix-sys-resource
  525. unix-sys-stat
  526. unix-time
  527. unstrctrd
  528. uring < "0.4"
  529. user-agent-parser
  530. uspf
  531. uspf-lwt
  532. uspf-mirage
  533. uspf-unix
  534. utop >= "2.13.0"
  535. validate
  536. validator
  537. vercel
  538. vhd-format-lwt >= "0.13.0"
  539. vpnkit
  540. wayland >= "2.0"
  541. wcwidth
  542. websocketaf
  543. x509 >= "0.7.0"
  544. xapi-rrd
  545. xapi-stdext-date
  546. xapi-stdext-encodings
  547. xapi-stdext-std >= "4.16.0"
  548. yaml
  549. yaml-sexp
  550. yocaml
  551. yocaml_syndication >= "2.0.0"
  552. yocaml_yaml < "2.0.0"
  553. yojson >= "1.6.0"
  554. yojson-five
  555. yuscii >= "0.3.0"
  556. yuujinchou >= "1.0.0"
  557. zar
  558. zed >= "3.2.2"
  559. zlist < "0.4.0"

Conflicts (1)

  1. result < "1.5"
OCaml

Innovation. Community. Security.