package alcotest

  1. Overview
  2. Docs
Alcotest is a lightweight and colourful test framework

Install

Dune Dependency

github.com Readme Changelog ISC License Edit opam file Versions (40)

Authors

  1. T
    Thomas Gazagnaire

Maintainers

  1. T
    thomas@gazagnaire.org

Sources

alcotest-0.8.5.tbz
md5=2db36741c413ab93391ecc1f983aa804

Description

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Published: 17 Jan 2019

README

Alcotest is a lightweight and colourful test framework.

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Examples

A simple example:

(* Build with `ocamlbuild -pkg alcotest simple.byte` *)

(* A module with functions to test *)
module To_test = struct
  let capit letter = Char.uppercase letter
  let plus int_list = List.fold_left (fun a b -> a + b) 0 int_list
end

(* The tests *)
let capit () =
  Alcotest.(check char) "same chars"  'A' (To_test.capit 'a')

let plus () =
  Alcotest.(check int) "same ints" 7 (To_test.plus [1;1;2;3])

let test_set = [
  "Capitalize" , `Quick, capit;
  "Add entries", `Slow , plus ;
]

(* Run it *)
let () =
  Alcotest.run "My first test" [
    "test_set", test_set;
  ]

The result is a self-contained binary which displays the test results. Use ./simple.byte --help to see the runtime options.

$ ./simple.native
[OK]        test_set  0   Capitalize.
[OK]        test_set  1   Add entries.
Test Successful in 0.001s. 2 tests run.

You can filter which tests to run by supplying either the exact test name (which would run all testcases with that name), or the exact test name and test case number (which would run just that single test):

$ ./simple.native test test_set
Testing My first test.
[OK]              test_set          0   Capitalize.
[OK]              test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 test run.
$ ./simple.native test test_set 1
Testing My first test.
[SKIP]              test_set          0   Capitalize.
[OK]                test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.

Note that you cannot filter by test case name (i.e. Capitalize or Add entries), you have to use the test case number instead.

See the examples folder for more examples.

Quick and Slow tests

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

Passing custom options to the tests

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your prefered number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ test.exe test
test.exe: required option -n is missing

$ test.exe test -n 42
Testing foo.
[OK]                all          0   int.

Lwt

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type unit -> unit Lwt.t and alcotest-lwt calls Lwt_main.run for you.

However, there are a couple of extra features:

  • If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).

  • You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Alcotest.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ test.exe
Testing foo.
[ERROR]             all          0   one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken

Screenshots

The following screenshots demonstrate the HTML testing output from the odoc project.

All tests passed Some tests failed Failed test with custom diffing
ok err diff

Comparison with other testing frameworks

The README is pretty clear about that:

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

  • Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;

  • Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

  • qcheck qcheck does random generation and property testing (e.g. Quick Check)

  • crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.

  • ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.

Dependencies (7)

  1. uuidm
  2. cmdliner
  3. result < "1.5"
  4. astring
  5. fmt >= "0.8.0"
  6. ocaml >= "4.02.3"
  7. dune >= "1.1.0"

Dev Dependencies

None

  1. ahrocksdb
  2. albatross >= "1.5.0"
  3. alcotest-async < "1.0.0"
  4. alcotest-lwt < "1.0.0"
  5. alg_structs_qcheck
  6. ambient-context
  7. ambient-context-eio
  8. ambient-context-lwt
  9. angstrom >= "0.7.0"
  10. ansi >= "0.6.0"
  11. anycache >= "0.7.4"
  12. anycache-async
  13. anycache-lwt
  14. archetype >= "1.4.2"
  15. archi
  16. arp
  17. arp-mirage
  18. arrakis
  19. art
  20. asak >= "0.2"
  21. asli >= "0.2.0"
  22. asn1-combinators >= "0.2.2"
  23. atd >= "2.3.3"
  24. atdgen >= "2.10.0"
  25. atdpy
  26. atdts
  27. base32
  28. base64 >= "2.1.2"
  29. bechamel >= "0.5.0"
  30. bigarray-overlap
  31. bigstring >= "0.3"
  32. bigstring-unix >= "0.3"
  33. bigstringaf
  34. bitlib
  35. blake2
  36. bloomf
  37. bls12-381 < "0.4.1" | >= "3.0.0" & < "18.0"
  38. bls12-381-hash
  39. bls12-381-js >= "0.4.2"
  40. bls12-381-js-gen >= "0.4.2"
  41. bls12-381-legacy
  42. bls12-381-signature
  43. bls12-381-unix
  44. blurhash
  45. builder-web
  46. bulletml
  47. bytebuffer
  48. ca-certs
  49. ca-certs-nss
  50. cactus
  51. caldav
  52. calendar >= "3.0.0"
  53. callipyge
  54. camlix
  55. capnp-rpc < "0.6.0"
  56. capnp-rpc-lwt < "0.3"
  57. carray
  58. carton
  59. cborl
  60. ccss >= "1.6"
  61. cf-lwt
  62. chacha
  63. channel
  64. charrua-client
  65. charrua-client-lwt
  66. charrua-client-mirage < "0.11.0"
  67. checkseum >= "0.0.3"
  68. cid
  69. clarity-lang
  70. class_group_vdf
  71. cohttp >= "0.17.0"
  72. cohttp-curl-async
  73. cohttp-curl-lwt
  74. cohttp-eio >= "6.0.0~beta2"
  75. colombe
  76. color
  77. conan
  78. conan-cli
  79. conan-database
  80. conan-lwt
  81. conan-unix
  82. conduit = "3.0.0"
  83. conex < "0.10.0"
  84. conex-mirage-crypto
  85. conex-nocrypto
  86. cookie
  87. cow >= "2.2.0"
  88. css
  89. css-parser
  90. cstruct >= "3.3.0"
  91. cstruct-sexp
  92. ctypes-zarith
  93. cuid
  94. curly
  95. current_incr
  96. cwe_checker
  97. data-encoding < "1.0.0"
  98. datakit >= "0.12.0"
  99. datakit-bridge-github >= "0.12.0"
  100. datakit-ci
  101. datakit-client-git >= "0.12.0"
  102. decompress >= "0.8" & < "1.5.3"
  103. depyt
  104. digestif >= "0.7"
  105. dispatch >= "0.4.1"
  106. dkim
  107. dkim-bin
  108. dkim-mirage
  109. dns >= "4.0.0"
  110. dns-cli
  111. dns-client >= "4.6.0"
  112. dns-forward < "0.9.0"
  113. dns-forward-lwt-unix
  114. dns-resolver
  115. dns-server
  116. dns-tsig
  117. dnssd
  118. dnssec
  119. docfd >= "2.2.0"
  120. dog < "0.2.1"
  121. domain-name
  122. dream
  123. dream-pure
  124. duff
  125. dune-release >= "1.0.0"
  126. duration >= "0.1.1"
  127. emile
  128. encore
  129. eqaf >= "0.5"
  130. equinoxe
  131. equinoxe-cohttp
  132. equinoxe-hlc
  133. eris
  134. eris-lwt
  135. ezgzip
  136. ezjsonm >= "0.4.2" & < "1.3.0"
  137. ezjsonm-lwt < "1.3.0"
  138. FPauth
  139. FPauth-core
  140. FPauth-responses
  141. FPauth-strategies
  142. faraday != "0.2.0"
  143. farfadet
  144. fat-filesystem >= "0.12.0"
  145. ff
  146. ff-pbt
  147. fiat-p256
  148. flex-array
  149. fsevents-lwt
  150. functoria >= "2.2.0"
  151. functoria-runtime >= "2.2.0" & != "3.0.1" & < "4.0.0~beta1"
  152. geojson
  153. geoml >= "0.1.1"
  154. git = "1.4.10" | = "1.5.0" | >= "1.5.2" & != "1.10.0" & < "3.0.0"
  155. git-mirage < "3.0.0"
  156. git-unix >= "1.10.0" & != "2.1.0" & < "3.0.0"
  157. gitlab-unix
  158. glicko2
  159. gmap >= "0.3.0"
  160. gobba
  161. gpt
  162. graphql
  163. graphql-async
  164. graphql-cohttp >= "0.13.0"
  165. graphql-lwt
  166. graphql_parser
  167. graphql_ppx >= "0.7.1"
  168. h1_parser
  169. h2
  170. hacl
  171. hacl_func
  172. hacl_x25519 >= "0.2.0"
  173. highlexer
  174. hkdf
  175. hockmd
  176. html_of_jsx
  177. http
  178. http-multipart-formdata < "2.0.0"
  179. httpaf >= "0.2.0"
  180. hvsock
  181. icalendar
  182. imagelib >= "20200929"
  183. index
  184. inferno >= "20220603"
  185. influxdb-async
  186. influxdb-lwt
  187. inquire < "0.2.0"
  188. interval-map
  189. iomux
  190. irmin < "0.8.0" | >= "0.9.6" & != "0.11.1" & < "1.2.0" | >= "2.0.0" & < "2.3.0"
  191. irmin-bench >= "2.7.0"
  192. irmin-chunk < "2.0.0" | >= "2.3.0"
  193. irmin-cli
  194. irmin-containers
  195. irmin-fs < "2.0.0" | >= "2.3.0"
  196. irmin-git < "2.0.0" | >= "2.3.0"
  197. irmin-http < "2.0.0"
  198. irmin-mem < "2.0.0"
  199. irmin-pack >= "2.4.0" & != "2.6.1"
  200. irmin-pack-tools
  201. irmin-test < "2.2.0"
  202. irmin-tezos
  203. irmin-tezos-utils
  204. irmin-unix >= "1.0.0" & < "2.0.0" | >= "2.4.0" & != "2.6.1"
  205. irmin-watcher
  206. jekyll-format
  207. jerboa
  208. jitsu
  209. jose
  210. json-data-encoding >= "0.9"
  211. json_decoder
  212. jsonxt
  213. junit_alcotest
  214. jwto
  215. ke >= "0.2"
  216. kkmarkdown
  217. lambda-runtime
  218. lambdapi >= "2.0.0"
  219. lambdoc >= "1.0-beta4"
  220. ledgerwallet-tezos >= "0.2.1" & < "0.4.0"
  221. lmdb >= "1.0"
  222. logical
  223. logtk >= "1.5.1"
  224. lp
  225. lp-glpk
  226. lp-glpk-js
  227. lp-gurobi
  228. lru
  229. lt-code
  230. luv
  231. mbr-format >= "1.0.0"
  232. mdx >= "1.6.0"
  233. mec
  234. mechaml >= "1.0.0"
  235. merge-queues >= "0.2.0"
  236. merge-ropes >= "0.2.0"
  237. metrics
  238. minicaml >= "0.3.1"
  239. mirage >= "4.0.0~beta1"
  240. mirage-block-partition < "0.2.0"
  241. mirage-block-ramdisk >= "0.3"
  242. mirage-channel >= "4.0.0"
  243. mirage-channel-lwt
  244. mirage-crypto-ec
  245. mirage-flow >= "1.0.2" & < "1.2.0"
  246. mirage-flow-unix
  247. mirage-fs-mem
  248. mirage-fs-unix >= "1.2.0"
  249. mirage-kv >= "2.0.0"
  250. mirage-kv-mem
  251. mirage-kv-unix
  252. mirage-logs >= "0.3.0"
  253. mirage-nat
  254. mirage-net-unix >= "2.3.0"
  255. mirage-runtime >= "4.0.0~beta1" & < "4.5.0"
  256. mirage-tc
  257. mjson
  258. mmdb
  259. mnd
  260. monocypher
  261. mrmime
  262. mrt-format
  263. msgpck >= "1.6"
  264. multibase
  265. multihash
  266. multihash-digestif
  267. multipart-form-data
  268. multipart_form
  269. multipart_form-eio
  270. multipart_form-lwt
  271. named-pipe
  272. nanoid
  273. nbd >= "4.0.3"
  274. nbd-tool
  275. nloge
  276. nocoiner
  277. non_empty_list
  278. OCADml >= "0.6.0"
  279. ocaml-r >= "0.5.0"
  280. ocaml-version >= "3.1.0"
  281. ocamlformat >= "0.13.0" & != "0.19.0~4.13preview" & < "0.25.1"
  282. ocamlformat-rpc < "removed"
  283. ocamline
  284. odoc >= "1.4.0" & < "2.1.0"
  285. ohex
  286. oidc
  287. opam-0install
  288. opam-file-format >= "2.1.1"
  289. opentelemetry >= "0.6"
  290. opentelemetry-client-cohttp-lwt >= "0.6"
  291. opentelemetry-client-ocurl >= "0.6"
  292. opentelemetry-cohttp-lwt >= "0.6"
  293. opentelemetry-lwt >= "0.6"
  294. opium >= "0.15.0"
  295. opium-graphql
  296. opium-testing
  297. opium_kernel
  298. orewa
  299. ortac-core
  300. osx-acl
  301. osx-attr
  302. osx-cf
  303. osx-fsevents
  304. osx-membership
  305. osx-mount
  306. osx-xattr
  307. otoggl
  308. owl >= "0.6.0" & != "0.9.0" & != "1.0.0"
  309. owl-base < "0.5.0"
  310. owl-ode >= "0.1.0" & != "0.2.0"
  311. owl-symbolic
  312. passmaker
  313. patch
  314. pbkdf
  315. pecu >= "0.2"
  316. pf-qubes
  317. pg_query >= "0.9.6"
  318. phylogenetics
  319. piaf
  320. polyglot
  321. polynomial
  322. ppx_blob >= "0.3.0"
  323. ppx_deriving_cmdliner
  324. ppx_deriving_rpc
  325. ppx_deriving_yaml
  326. ppx_graphql >= "0.2.0"
  327. ppx_mysql
  328. ppx_parser
  329. ppx_protocol_conv >= "5.0.0"
  330. ppx_protocol_conv_json >= "5.0.0"
  331. ppx_protocol_conv_jsonm >= "5.0.0"
  332. ppx_protocol_conv_msgpack >= "5.0.0"
  333. ppx_protocol_conv_xml_light >= "5.0.0"
  334. ppx_protocol_conv_xmlm
  335. ppx_protocol_conv_yaml >= "5.0.0"
  336. ppx_subliner
  337. ppx_units
  338. ppx_yojson >= "1.1.0"
  339. pratter
  340. prc
  341. preface
  342. pretty_expressive
  343. prettym
  344. proc-smaps
  345. producer < "0.2.0"
  346. prom < "0.2"
  347. prometheus < "1.2"
  348. prometheus-app
  349. protocell
  350. protocol-9p >= "0.3" & < "0.11.0" | >= "0.11.2"
  351. protocol-9p-unix
  352. psq
  353. qcheck >= "0.18"
  354. qcheck-alcotest
  355. qcheck-core >= "0.18"
  356. quickjs
  357. radis
  358. randii
  359. reason-standard
  360. reparse >= "2.0.0" & < "3.0.0"
  361. reparse-unix < "2.1.0"
  362. resp < "0.10.0"
  363. resp-unix
  364. rfc1951 < "1.0.0"
  365. routes < "2.0.0"
  366. rpc >= "5.9.0"
  367. rpclib
  368. rpclib-async
  369. rpclib-lwt
  370. rubytt
  371. SZXX >= "4.0.0"
  372. salsa20
  373. salsa20-core
  374. sanddb >= "0.2"
  375. scaml >= "1.5.0"
  376. scrypt-kdf
  377. secp256k1 >= "0.4.1"
  378. secp256k1-internal
  379. semver >= "0.2.1"
  380. sendmail
  381. sendmail-lwt
  382. sendmsg
  383. server-reason-react
  384. session-cookie
  385. session-cookie-async
  386. session-cookie-lwt
  387. sherlodoc
  388. slug
  389. sodium-fmt
  390. spin >= "0.6.0"
  391. squirrel
  392. ssh-agent
  393. ssl >= "0.6.0"
  394. stramon-lib
  395. syslog-rfc5424
  396. tcpip >= "2.4.2" & < "4.0.0" | >= "5.0.1" & < "8.0.0"
  397. tdigest < "2.1.0"
  398. terminal_size >= "0.1.1"
  399. terminus
  400. terminus-cohttp
  401. terminus-hlc
  402. terml
  403. textrazor
  404. tezos-base-test-helpers < "13.0"
  405. tezos-client-base < "12.0"
  406. tezos-crypto < "8.0"
  407. tezos-lmdb
  408. tezos-signer-backends < "8.0"
  409. tezos-stdlib < "8.0"
  410. tezos-test-helpers < "11.0"
  411. tftp
  412. timedesc
  413. timere
  414. tls >= "0.12.0"
  415. toc
  416. topojson
  417. topojsone
  418. transept
  419. twostep
  420. type_eq
  421. type_id
  422. typebeat
  423. typeid >= "1.0.1"
  424. tyre >= "0.4"
  425. tyxml >= "4.0.0"
  426. tyxml-jsx
  427. tyxml-ppx >= "4.3.0"
  428. tyxml-syntax
  429. uecc
  430. ulid
  431. universal-portal
  432. unix-dirent
  433. unix-errno >= "0.3.0"
  434. unix-fcntl >= "0.3.0"
  435. unix-sys-resource
  436. unix-sys-stat
  437. unix-time
  438. unstrctrd
  439. user-agent-parser
  440. uspf
  441. uspf-lwt
  442. uspf-unix
  443. utop >= "2.13.0"
  444. validate
  445. validator
  446. vercel
  447. vpnkit
  448. websocketaf
  449. x509 >= "0.7.0"
  450. xapi-rrd >= "1.8.2"
  451. xapi-stdext-date
  452. xapi-stdext-encodings
  453. xapi-stdext-std >= "4.16.0"
  454. yaml < "3.2.0"
  455. yaml-sexp
  456. yocaml
  457. yocaml_yaml
  458. yojson >= "1.6.0"
  459. yuscii >= "0.2.0"
  460. zar
  461. zed >= "3.2.2"
  462. zlist < "0.4.0"

Conflicts

None