package alcotest

  1. Overview
  2. Docs
Alcotest is a lightweight and colourful test framework.

Install

Dune Dependency

Authors

Maintainers

Sources

alcotest-0.8.4.tbz
md5=c940f89a2bb2e23d3f1422ce61d1396b

Description

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Published: 17 Oct 2018

README

Alcotest is a lightweight and colourful test framework.

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Examples

A simple example:

(* Build with `ocamlbuild -pkg alcotest simple.byte` *)

(* A module with functions to test *)
module To_test = struct
  let capit letter = Char.uppercase letter
  let plus int_list = List.fold_left (fun a b -> a + b) 0 int_list
end

(* The tests *)
let capit () =
  Alcotest.(check char) "same chars"  'A' (To_test.capit 'a')

let plus () =
  Alcotest.(check int) "same ints" 7 (To_test.plus [1;1;2;3])

let test_set = [
  "Capitalize" , `Quick, capit;
  "Add entries", `Slow , plus ;
]

(* Run it *)
let () =
  Alcotest.run "My first test" [
    "test_set", test_set;
  ]

The result is a self-contained binary which displays the test results. Use ./simple.byte --help to see the runtime options.

$ ./simple.native
[OK]        test_set  0   Capitalize.
[OK]        test_set  1   Add entries.
Test Successful in 0.001s. 2 tests run.

You can filter which tests to run by supplying either the exact test name (which would run all testcases with that name), or the exact test name and test case number (which would run just that single test):

$ ./simple.native test test_set
Testing My first test.
[OK]              test_set          0   Capitalize.
[OK]              test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 test run.
$ ./simple.native test test_set 1
Testing My first test.
[SKIP]              test_set          0   Capitalize.
[OK]                test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.

Note that you cannot filter by test case name (i.e. Capitalize or Add entries), you have to use the test case number instead.

See the examples folder for more examples.

Quick and Slow tests

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

Passing custom options to the tests

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your prefered number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ test.exe test
test.exe: required option -n is missing

$ test.exe test -n 42
Testing foo.
[OK]                all          0   int.

Lwt

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type unit -> unit Lwt.t and alcotest-lwt calls Lwt_main.run for you.

However, there are a couple of extra features:

  • If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).

  • You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Alcotest.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ test.exe
Testing foo.
[ERROR]             all          0   one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken

Comparison with other testing frameworks

The README is pretty clear about that:

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

  • Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;

  • Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

  • qcheck qcheck does random generation and property testing (e.g. Quick Check)

  • crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.

  • ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.

Dependencies (7)

  1. uuidm
  2. cmdliner
  3. result < "1.5"
  4. astring
  5. fmt >= "0.8.0"
  6. jbuilder >= "1.0+beta10"
  7. ocaml >= "4.02.3"

Dev Dependencies

None

  1. ahrocksdb
  2. albatross >= "1.5.0"
  3. alcotest-async < "1.0.0"
  4. alcotest-lwt < "1.0.0"
  5. ambient-context
  6. ambient-context-eio
  7. ambient-context-lwt
  8. angstrom >= "0.7.0"
  9. ansi >= "0.6.0"
  10. anycache >= "0.7.4"
  11. anycache-async
  12. anycache-lwt
  13. archetype >= "1.4.2"
  14. archi
  15. arp
  16. arp-mirage
  17. arrakis
  18. art
  19. asak >= "0.2"
  20. asli >= "0.2.0"
  21. asn1-combinators >= "0.2.2"
  22. atd >= "2.3.3"
  23. atdgen >= "2.10.0"
  24. atdpy
  25. atdts
  26. base32
  27. base64 >= "2.1.2"
  28. bechamel >= "0.5.0"
  29. bigarray-overlap
  30. bigstring >= "0.3"
  31. bigstring-unix >= "0.3"
  32. bigstringaf
  33. bitlib
  34. blake2
  35. bloomf
  36. bls12-381 < "0.4.1" | >= "3.0.0" & < "18.0"
  37. bls12-381-hash
  38. bls12-381-js >= "0.4.2"
  39. bls12-381-js-gen >= "0.4.2"
  40. bls12-381-legacy
  41. bls12-381-signature
  42. bls12-381-unix
  43. blurhash
  44. builder-web
  45. bulletml
  46. bytebuffer
  47. ca-certs
  48. ca-certs-nss
  49. cactus
  50. calendar >= "3.0.0"
  51. callipyge
  52. camlix
  53. capnp-rpc < "0.6.0"
  54. capnp-rpc-lwt < "0.3"
  55. carray
  56. carton
  57. cborl
  58. ccss >= "1.6"
  59. cf-lwt
  60. chacha
  61. channel
  62. charrua-client
  63. charrua-client-lwt
  64. charrua-client-mirage < "0.11.0"
  65. checkseum >= "0.0.3"
  66. cid
  67. clarity-lang
  68. class_group_vdf
  69. cohttp >= "0.17.0"
  70. cohttp-curl-async
  71. cohttp-curl-lwt
  72. cohttp-eio >= "6.0.0~beta2"
  73. colombe
  74. color
  75. conan
  76. conan-cli
  77. conan-database
  78. conan-lwt
  79. conan-unix
  80. conduit = "3.0.0"
  81. conex < "0.10.0"
  82. conex-mirage-crypto
  83. conex-nocrypto
  84. cookie
  85. cow >= "2.2.0"
  86. css
  87. css-parser
  88. cstruct >= "3.3.0"
  89. cstruct-sexp
  90. ctypes-zarith
  91. cuid
  92. curly
  93. current_incr
  94. cwe_checker
  95. data-encoding < "1.0.0"
  96. datakit >= "0.12.0"
  97. datakit-bridge-github >= "0.12.0"
  98. datakit-ci
  99. datakit-client-git >= "0.12.0"
  100. decompress >= "0.8" & < "1.5.3"
  101. depyt
  102. digestif >= "0.7"
  103. dispatch >= "0.4.1"
  104. dkim
  105. dkim-bin
  106. dkim-mirage
  107. dns >= "4.0.0"
  108. dns-cli
  109. dns-client >= "4.6.0"
  110. dns-forward < "0.9.0"
  111. dns-forward-lwt-unix
  112. dns-resolver
  113. dns-server
  114. dns-tsig
  115. dnssd
  116. dnssec
  117. docfd >= "2.2.0"
  118. dog < "0.2.1"
  119. domain-name
  120. dream
  121. dream-pure
  122. duff
  123. dune-release >= "1.0.0"
  124. duration >= "0.1.1"
  125. emile
  126. encore
  127. eqaf >= "0.5"
  128. equinoxe
  129. equinoxe-cohttp
  130. equinoxe-hlc
  131. eris
  132. eris-lwt
  133. ezgzip
  134. ezjsonm >= "0.4.2" & < "1.3.0"
  135. ezjsonm-lwt < "1.3.0"
  136. FPauth
  137. FPauth-core
  138. FPauth-responses
  139. FPauth-strategies
  140. faraday != "0.2.0"
  141. farfadet
  142. fat-filesystem >= "0.12.0"
  143. ff
  144. ff-pbt
  145. fiat-p256
  146. flex-array
  147. fsevents-lwt
  148. functoria >= "2.2.0"
  149. functoria-runtime >= "2.2.0" & != "3.0.1" & < "4.0.0~beta1"
  150. geojson
  151. geoml >= "0.1.1"
  152. git = "1.4.10" | = "1.5.0" | >= "1.5.2" & != "1.10.0" & < "3.0.0"
  153. git-mirage < "3.0.0"
  154. git-unix >= "1.10.0" & != "2.1.0" & < "3.0.0"
  155. gitlab-unix
  156. glicko2
  157. gmap >= "0.3.0"
  158. gpt
  159. graphql
  160. graphql-async
  161. graphql-cohttp >= "0.13.0"
  162. graphql-lwt
  163. graphql_parser
  164. graphql_ppx >= "0.7.1"
  165. h1_parser
  166. h2
  167. hacl
  168. hacl_func
  169. hacl_x25519 >= "0.2.0"
  170. highlexer
  171. hkdf
  172. hockmd
  173. html_of_jsx
  174. http
  175. http-multipart-formdata < "2.0.0"
  176. httpaf >= "0.2.0"
  177. hvsock
  178. icalendar
  179. imagelib >= "20200929"
  180. index
  181. inferno >= "20220603"
  182. influxdb-async
  183. influxdb-lwt
  184. inquire < "0.2.0"
  185. interval-map
  186. iomux
  187. irmin < "0.8.0" | >= "0.9.6" & != "0.11.1" & < "1.2.0" | >= "2.0.0" & < "2.3.0"
  188. irmin-bench >= "2.7.0"
  189. irmin-chunk < "2.0.0" | >= "2.3.0"
  190. irmin-cli
  191. irmin-containers
  192. irmin-fs < "2.0.0" | >= "2.3.0"
  193. irmin-git < "2.0.0" | >= "2.3.0"
  194. irmin-http < "2.0.0"
  195. irmin-mem < "2.0.0"
  196. irmin-pack >= "2.4.0" & != "2.6.1"
  197. irmin-pack-tools
  198. irmin-test < "2.2.0"
  199. irmin-tezos
  200. irmin-tezos-utils
  201. irmin-unix >= "1.0.0" & < "2.0.0" | >= "2.4.0" & != "2.6.1"
  202. irmin-watcher
  203. jekyll-format
  204. jerboa
  205. jitsu
  206. jose
  207. json-data-encoding >= "0.9"
  208. json_decoder
  209. jsonxt
  210. junit_alcotest
  211. jwto
  212. ke >= "0.2"
  213. kkmarkdown
  214. lambda-runtime
  215. lambdapi >= "2.0.0"
  216. lambdoc >= "1.0-beta4"
  217. ledgerwallet-tezos >= "0.2.1" & < "0.4.0"
  218. lmdb >= "1.0"
  219. logical
  220. logtk >= "1.5.1"
  221. lp
  222. lp-glpk
  223. lp-glpk-js
  224. lp-gurobi
  225. lru
  226. lt-code
  227. luv
  228. mbr-format >= "1.0.0"
  229. mdx >= "1.6.0"
  230. mec
  231. mechaml >= "1.0.0"
  232. merge-queues >= "0.2.0"
  233. merge-ropes >= "0.2.0"
  234. metrics
  235. mirage >= "4.0.0~beta1"
  236. mirage-block-partition < "0.2.0"
  237. mirage-block-ramdisk >= "0.3"
  238. mirage-channel >= "4.0.0"
  239. mirage-channel-lwt
  240. mirage-crypto-ec
  241. mirage-flow >= "1.0.2" & < "1.2.0"
  242. mirage-flow-unix
  243. mirage-fs-mem
  244. mirage-fs-unix >= "1.2.0"
  245. mirage-kv >= "2.0.0"
  246. mirage-kv-mem
  247. mirage-kv-unix
  248. mirage-logs >= "0.3.0"
  249. mirage-nat
  250. mirage-net-unix >= "2.3.0"
  251. mirage-runtime >= "4.0.0~beta1"
  252. mirage-tc
  253. mjson
  254. mnd
  255. monocypher
  256. mrmime
  257. mrt-format
  258. multibase
  259. multihash
  260. multihash-digestif
  261. multipart-form-data
  262. multipart_form
  263. multipart_form-lwt
  264. named-pipe
  265. nanoid
  266. nbd >= "4.0.3"
  267. nbd-tool
  268. nloge
  269. nocoiner
  270. non_empty_list
  271. OCADml >= "0.6.0"
  272. ocaml-r >= "0.5.0"
  273. ocaml-version >= "3.1.0"
  274. ocamlformat >= "0.13.0" & != "0.19.0~4.13preview" & < "0.25.1"
  275. ocamlformat-rpc < "removed"
  276. ocamline
  277. odoc >= "1.4.0" & < "2.1.0"
  278. ohex
  279. oidc
  280. opam-0install
  281. opam-file-format >= "2.1.1"
  282. opentelemetry >= "0.6"
  283. opentelemetry-client-cohttp-lwt >= "0.6"
  284. opentelemetry-client-ocurl >= "0.6"
  285. opentelemetry-cohttp-lwt >= "0.6"
  286. opentelemetry-lwt >= "0.6"
  287. opium >= "0.15.0"
  288. opium-graphql
  289. opium-testing
  290. opium_kernel
  291. orewa
  292. ortac-core
  293. osx-acl
  294. osx-attr
  295. osx-cf
  296. osx-fsevents
  297. osx-membership
  298. osx-mount
  299. osx-xattr
  300. otoggl
  301. owl >= "0.6.0" & != "0.9.0" & != "1.0.0"
  302. owl-base < "0.5.0"
  303. owl-ode >= "0.1.0" & != "0.2.0"
  304. owl-symbolic
  305. passmaker
  306. patch
  307. pbkdf
  308. pecu >= "0.2"
  309. pf-qubes
  310. pg_query >= "0.9.6"
  311. phylogenetics
  312. piaf
  313. polyglot
  314. polynomial
  315. ppx_blob >= "0.3.0"
  316. ppx_deriving_cmdliner
  317. ppx_deriving_rpc
  318. ppx_deriving_yaml
  319. ppx_graphql >= "0.2.0"
  320. ppx_mysql
  321. ppx_parser
  322. ppx_protocol_conv >= "5.0.0"
  323. ppx_protocol_conv_json >= "5.0.0"
  324. ppx_protocol_conv_jsonm >= "5.0.0"
  325. ppx_protocol_conv_msgpack >= "5.0.0"
  326. ppx_protocol_conv_xml_light >= "5.0.0"
  327. ppx_protocol_conv_xmlm
  328. ppx_protocol_conv_yaml >= "5.0.0"
  329. ppx_subliner
  330. ppx_units
  331. ppx_yojson >= "1.1.0"
  332. pratter
  333. prc
  334. preface
  335. pretty_expressive
  336. prettym
  337. proc-smaps
  338. producer < "0.2.0"
  339. prometheus < "1.2"
  340. prometheus-app
  341. protocell
  342. protocol-9p >= "0.3" & < "0.11.0" | >= "0.11.2"
  343. protocol-9p-unix
  344. psq
  345. qcheck >= "0.18"
  346. qcheck-alcotest
  347. qcheck-core >= "0.18"
  348. quickjs
  349. radis
  350. randii
  351. reason-standard
  352. reparse >= "2.0.0" & < "3.0.0"
  353. reparse-unix < "2.1.0"
  354. resp < "0.10.0"
  355. resp-unix
  356. rfc1951 < "1.0.0"
  357. routes < "2.0.0"
  358. rpc >= "5.9.0"
  359. rpclib
  360. rpclib-async
  361. rpclib-lwt
  362. rubytt
  363. SZXX >= "4.0.0"
  364. salsa20
  365. salsa20-core
  366. sanddb >= "0.2"
  367. scaml >= "1.5.0"
  368. scrypt-kdf
  369. secp256k1 >= "0.4.1"
  370. secp256k1-internal
  371. semver >= "0.2.1"
  372. sendmail
  373. sendmail-lwt
  374. sendmsg
  375. server-reason-react
  376. session-cookie
  377. session-cookie-async
  378. session-cookie-lwt
  379. sherlodoc
  380. slug
  381. sodium-fmt
  382. spin >= "0.6.0"
  383. squirrel
  384. ssh-agent
  385. ssl >= "0.6.0"
  386. stramon-lib
  387. tcpip >= "2.4.2" & < "4.0.0" | >= "5.0.1" & < "8.0.0"
  388. tdigest < "2.1.0"
  389. terminal_size >= "0.1.1"
  390. terminus
  391. terminus-cohttp
  392. terminus-hlc
  393. textrazor
  394. tezos-base-test-helpers < "13.0"
  395. tezos-client-base < "12.0"
  396. tezos-lmdb
  397. tezos-test-helpers < "11.0"
  398. tftp
  399. timedesc
  400. timere
  401. tls >= "0.12.0"
  402. toc
  403. topojson
  404. topojsone
  405. transept
  406. twostep
  407. type_eq
  408. type_id
  409. typebeat
  410. typeid >= "1.0.1"
  411. tyre >= "0.4"
  412. tyxml >= "4.0.0"
  413. tyxml-jsx
  414. tyxml-ppx >= "4.3.0"
  415. tyxml-syntax
  416. uecc
  417. ulid
  418. unix-dirent
  419. unix-errno >= "0.3.0"
  420. unix-fcntl >= "0.3.0"
  421. unix-sys-resource
  422. unix-sys-stat
  423. unix-time
  424. unstrctrd
  425. user-agent-parser
  426. uspf
  427. uspf-lwt
  428. uspf-unix
  429. utop >= "2.13.0"
  430. validate
  431. validator
  432. vercel
  433. vpnkit
  434. websocketaf
  435. x509 >= "0.7.0"
  436. xapi-rrd >= "1.8.2"
  437. xapi-stdext-date
  438. xapi-stdext-encodings
  439. xapi-stdext-std >= "4.16.0"
  440. yaml < "3.2.0"
  441. yaml-sexp
  442. yocaml
  443. yocaml_yaml
  444. yuscii >= "0.2.0"
  445. zar
  446. zed >= "3.2.2"
  447. zlist < "0.4.0"

Conflicts

None