package alcotest

  1. Overview
  2. Docs
Alcotest is a lightweight and colourful test framework.

Install

Dune Dependency

github.com Readme Changelog ISC License Edit opam file Versions (40)

Authors

  1. T
    Thomas Gazagnaire

Maintainers

  1. T
    thomas@gazagnaire.org

Sources

alcotest-0.8.4.tbz
sha256=8643b39e74317e2ec49be9a0986e73acd4fcd1277a6d59fb1423c3a6ebdfaf61
md5=c940f89a2bb2e23d3f1422ce61d1396b

Description

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Published: 17 Oct 2018

README

Alcotest is a lightweight and colourful test framework.

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Examples

A simple example:

(* Build with `ocamlbuild -pkg alcotest simple.byte` *)

(* A module with functions to test *)
module To_test = struct
  let capit letter = Char.uppercase letter
  let plus int_list = List.fold_left (fun a b -> a + b) 0 int_list
end

(* The tests *)
let capit () =
  Alcotest.(check char) "same chars"  'A' (To_test.capit 'a')

let plus () =
  Alcotest.(check int) "same ints" 7 (To_test.plus [1;1;2;3])

let test_set = [
  "Capitalize" , `Quick, capit;
  "Add entries", `Slow , plus ;
]

(* Run it *)
let () =
  Alcotest.run "My first test" [
    "test_set", test_set;
  ]

The result is a self-contained binary which displays the test results. Use ./simple.byte --help to see the runtime options.

$ ./simple.native
[OK]        test_set  0   Capitalize.
[OK]        test_set  1   Add entries.
Test Successful in 0.001s. 2 tests run.

You can filter which tests to run by supplying either the exact test name (which would run all testcases with that name), or the exact test name and test case number (which would run just that single test):

$ ./simple.native test test_set
Testing My first test.
[OK]              test_set          0   Capitalize.
[OK]              test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 test run.
$ ./simple.native test test_set 1
Testing My first test.
[SKIP]              test_set          0   Capitalize.
[OK]                test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.

Note that you cannot filter by test case name (i.e. Capitalize or Add entries), you have to use the test case number instead.

See the examples folder for more examples.

Quick and Slow tests

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

Passing custom options to the tests

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your prefered number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ test.exe test
test.exe: required option -n is missing

$ test.exe test -n 42
Testing foo.
[OK]                all          0   int.

Lwt

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type unit -> unit Lwt.t and alcotest-lwt calls Lwt_main.run for you.

However, there are a couple of extra features:

  • If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).

  • You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Alcotest.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ test.exe
Testing foo.
[ERROR]             all          0   one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken

Comparison with other testing frameworks

The README is pretty clear about that:

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

  • Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;

  • Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

  • qcheck qcheck does random generation and property testing (e.g. Quick Check)

  • crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.

  • ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.

Dependencies (7)

  1. uuidm
  2. cmdliner
  3. result < "1.5"
  4. astring
  5. fmt >= "0.8.0"
  6. jbuilder >= "1.0+beta10"
  7. ocaml >= "4.02.3"

Dev Dependencies

None

  1. ahrocksdb
  2. albatross >= "1.5.0"
  3. alcotest-async < "1.0.0"
  4. alcotest-lwt < "1.0.0"
  5. ambient-context
  6. ambient-context-eio
  7. ambient-context-lwt
  8. angstrom >= "0.7.0"
  9. ansi >= "0.6.0"
  10. anycache >= "0.7.4"
  11. anycache-async
  12. anycache-lwt
  13. archetype >= "1.4.2"
  14. archi
  15. arp
  16. arp-mirage
  17. arrakis
  18. art
  19. asak >= "0.2"
  20. asli >= "0.2.0"
  21. asn1-combinators >= "0.2.2"
  22. atd >= "2.3.3"
  23. atdgen >= "2.10.0"
  24. atdpy
  25. atdts
  26. base32
  27. base64 >= "2.1.2"
  28. bechamel >= "0.5.0"
  29. bigarray-overlap
  30. bigstring >= "0.3"
  31. bigstring-unix >= "0.3"
  32. bigstringaf
  33. bitlib
  34. blake2
  35. bloomf
  36. bls12-381 < "0.4.1" | >= "3.0.0" & < "18.0"
  37. bls12-381-hash
  38. bls12-381-js >= "0.4.2"
  39. bls12-381-js-gen >= "0.4.2"
  40. bls12-381-legacy
  41. bls12-381-signature
  42. bls12-381-unix
  43. blurhash
  44. builder-web
  45. bulletml
  46. bytebuffer
  47. ca-certs
  48. ca-certs-nss
  49. cactus
  50. calendar >= "3.0.0"
  51. callipyge
  52. camlix
  53. camlkit
  54. camlkit-base
  55. capnp-rpc < "0.6.0"
  56. capnp-rpc-lwt < "0.3"
  57. carray
  58. carton
  59. cborl
  60. ccss >= "1.6"
  61. cf-lwt
  62. chacha
  63. channel
  64. charrua-client
  65. charrua-client-lwt
  66. charrua-client-mirage < "0.11.0"
  67. checked_oint < "0.1.1"
  68. checkseum >= "0.0.3"
  69. cid
  70. clarity-lang
  71. class_group_vdf
  72. cohttp >= "0.17.0"
  73. cohttp-curl-async
  74. cohttp-curl-lwt
  75. cohttp-eio >= "6.0.0~beta2"
  76. colombe
  77. color
  78. conan
  79. conan-cli
  80. conan-database
  81. conan-lwt
  82. conan-unix
  83. conduit = "3.0.0"
  84. conex < "0.10.0"
  85. conex-mirage-crypto
  86. conex-nocrypto
  87. cookie
  88. cow >= "2.2.0"
  89. css
  90. css-parser
  91. cstruct >= "3.3.0"
  92. cstruct-sexp
  93. ctypes-zarith
  94. cuid
  95. curly
  96. current_incr
  97. cwe_checker
  98. data-encoding < "1.0.0"
  99. datakit >= "0.12.0"
  100. datakit-bridge-github >= "0.12.0"
  101. datakit-ci
  102. datakit-client-git >= "0.12.0"
  103. decompress >= "0.8" & < "1.5.3"
  104. depyt
  105. digestif >= "0.7"
  106. dispatch >= "0.4.1"
  107. dkim
  108. dkim-bin
  109. dkim-mirage
  110. dns >= "4.0.0"
  111. dns-cli
  112. dns-client >= "4.6.0"
  113. dns-forward < "0.9.0"
  114. dns-forward-lwt-unix
  115. dns-resolver
  116. dns-server
  117. dns-tsig
  118. dnssd
  119. dnssec
  120. docfd >= "2.2.0"
  121. dog < "0.2.1"
  122. domain-name
  123. dream
  124. dream-pure
  125. duff
  126. dune-release >= "1.0.0"
  127. duration >= "0.1.1"
  128. emile
  129. encore
  130. eqaf >= "0.5"
  131. equinoxe
  132. equinoxe-cohttp
  133. equinoxe-hlc
  134. eris
  135. eris-lwt
  136. ezgzip
  137. ezjsonm >= "0.4.2" & < "1.3.0"
  138. ezjsonm-lwt < "1.3.0"
  139. FPauth
  140. FPauth-core
  141. FPauth-responses
  142. FPauth-strategies
  143. faraday != "0.2.0"
  144. farfadet
  145. fat-filesystem >= "0.12.0"
  146. ff
  147. ff-pbt
  148. fiat-p256
  149. flex-array
  150. fsevents-lwt
  151. functoria >= "2.2.0"
  152. functoria-runtime >= "2.2.0" & != "3.0.1" & < "4.0.0~beta1"
  153. geojson
  154. geoml >= "0.1.1"
  155. git = "1.4.10" | = "1.5.0" | >= "1.5.2" & != "1.10.0" & < "3.0.0"
  156. git-mirage < "3.0.0"
  157. git-unix >= "1.10.0" & != "2.1.0" & < "3.0.0"
  158. gitlab-unix
  159. glicko2
  160. gmap >= "0.3.0"
  161. gpt
  162. graphql
  163. graphql-async
  164. graphql-cohttp >= "0.13.0"
  165. graphql-lwt
  166. graphql_parser
  167. graphql_ppx >= "0.7.1"
  168. h1_parser
  169. h2
  170. hacl
  171. hacl_func
  172. hacl_x25519 >= "0.2.0"
  173. highlexer
  174. hkdf
  175. hockmd
  176. html_of_jsx
  177. http
  178. http-multipart-formdata < "2.0.0"
  179. httpaf >= "0.2.0"
  180. httpun
  181. hvsock
  182. icalendar
  183. imagelib >= "20200929"
  184. index
  185. inferno >= "20220603"
  186. influxdb-async
  187. influxdb-lwt
  188. inquire < "0.2.0"
  189. interval-map
  190. iomux
  191. irmin < "0.8.0" | >= "0.9.6" & != "0.11.1" & < "1.2.0" | >= "2.0.0" & < "2.3.0"
  192. irmin-bench >= "2.7.0"
  193. irmin-chunk < "2.0.0" | >= "2.3.0"
  194. irmin-cli
  195. irmin-containers
  196. irmin-fs < "2.0.0" | >= "2.3.0"
  197. irmin-git < "2.0.0" | >= "2.3.0"
  198. irmin-http < "2.0.0"
  199. irmin-mem < "2.0.0"
  200. irmin-pack >= "2.4.0" & != "2.6.1"
  201. irmin-pack-tools
  202. irmin-test < "2.2.0"
  203. irmin-tezos
  204. irmin-tezos-utils
  205. irmin-unix >= "1.0.0" & < "2.0.0" | >= "2.4.0" & != "2.6.1"
  206. irmin-watcher
  207. jekyll-format
  208. jerboa
  209. jitsu
  210. jose
  211. json-data-encoding >= "0.9"
  212. json_decoder
  213. jsonxt
  214. junit_alcotest
  215. jwto
  216. ke >= "0.2"
  217. kkmarkdown
  218. lambda-runtime
  219. lambdapi >= "2.0.0"
  220. lambdoc >= "1.0-beta4"
  221. ledgerwallet-tezos >= "0.2.1" & < "0.4.0"
  222. lmdb >= "1.0"
  223. logical
  224. logtk >= "1.5.1"
  225. lp
  226. lp-glpk
  227. lp-glpk-js
  228. lp-gurobi
  229. lru
  230. lt-code
  231. luv
  232. mbr-format >= "1.0.0"
  233. mdx >= "1.6.0"
  234. mec
  235. mechaml >= "1.0.0"
  236. merge-queues >= "0.2.0"
  237. merge-ropes >= "0.2.0"
  238. metrics
  239. mirage >= "4.0.0~beta1"
  240. mirage-block-partition < "0.2.0"
  241. mirage-block-ramdisk >= "0.3"
  242. mirage-channel >= "4.0.0"
  243. mirage-channel-lwt
  244. mirage-crypto-ec
  245. mirage-flow >= "1.0.2" & < "1.2.0"
  246. mirage-flow-unix
  247. mirage-fs-mem
  248. mirage-fs-unix >= "1.2.0"
  249. mirage-kv >= "2.0.0"
  250. mirage-kv-mem
  251. mirage-kv-unix
  252. mirage-logs >= "0.3.0"
  253. mirage-nat
  254. mirage-net-unix >= "2.3.0"
  255. mirage-runtime >= "4.0.0~beta1" & < "4.5.0"
  256. mirage-tc
  257. mjson
  258. mnd
  259. monocypher
  260. mrmime
  261. mrt-format
  262. multibase
  263. multihash
  264. multihash-digestif
  265. multipart-form-data
  266. multipart_form
  267. multipart_form-eio
  268. multipart_form-lwt
  269. named-pipe
  270. nanoid
  271. nbd >= "4.0.3"
  272. nbd-tool
  273. nloge
  274. nocoiner
  275. non_empty_list
  276. OCADml >= "0.6.0"
  277. ocaml-r >= "0.5.0"
  278. ocaml-version >= "3.1.0"
  279. ocamlformat >= "0.13.0" & != "0.19.0~4.13preview" & < "0.25.1"
  280. ocamlformat-rpc < "removed"
  281. ocamline
  282. odoc >= "1.4.0" & < "2.1.0"
  283. ohex
  284. oidc
  285. opam-0install
  286. opam-file-format >= "2.1.1"
  287. opentelemetry >= "0.6"
  288. opentelemetry-client-cohttp-lwt >= "0.6"
  289. opentelemetry-client-ocurl >= "0.6"
  290. opentelemetry-cohttp-lwt >= "0.6"
  291. opentelemetry-lwt >= "0.6"
  292. opium >= "0.15.0"
  293. opium-graphql
  294. opium-testing
  295. opium_kernel
  296. orewa
  297. ortac-core
  298. osx-acl
  299. osx-attr
  300. osx-cf
  301. osx-fsevents
  302. osx-membership
  303. osx-mount
  304. osx-xattr
  305. otoggl
  306. owl >= "0.6.0" & != "0.9.0" & != "1.0.0"
  307. owl-base < "0.5.0"
  308. owl-ode >= "0.1.0" & != "0.2.0"
  309. owl-symbolic
  310. passmaker
  311. patch
  312. pbkdf
  313. pecu >= "0.2"
  314. pf-qubes
  315. pg_query >= "0.9.6"
  316. phylogenetics
  317. piaf
  318. polyglot
  319. polynomial
  320. ppx_blob >= "0.3.0"
  321. ppx_deriving_cmdliner
  322. ppx_deriving_rpc
  323. ppx_deriving_yaml
  324. ppx_graphql >= "0.2.0"
  325. ppx_mysql
  326. ppx_parser
  327. ppx_protocol_conv >= "5.0.0"
  328. ppx_protocol_conv_json >= "5.0.0"
  329. ppx_protocol_conv_jsonm >= "5.0.0"
  330. ppx_protocol_conv_msgpack >= "5.0.0"
  331. ppx_protocol_conv_xml_light >= "5.0.0"
  332. ppx_protocol_conv_xmlm
  333. ppx_protocol_conv_yaml >= "5.0.0"
  334. ppx_subliner
  335. ppx_units
  336. ppx_yojson >= "1.1.0"
  337. pratter
  338. prc
  339. preface
  340. pretty_expressive
  341. prettym
  342. proc-smaps
  343. producer < "0.2.0"
  344. prometheus < "1.2"
  345. prometheus-app
  346. protocell
  347. protocol-9p >= "0.3" & < "0.11.0" | >= "0.11.2"
  348. protocol-9p-unix
  349. psq
  350. qcheck >= "0.18"
  351. qcheck-alcotest
  352. qcheck-core >= "0.18"
  353. quickjs
  354. radis
  355. randii
  356. reason-standard
  357. reparse >= "2.0.0" & < "3.0.0"
  358. reparse-unix < "2.1.0"
  359. resp < "0.10.0"
  360. resp-unix
  361. rfc1951 < "1.0.0"
  362. routes < "2.0.0"
  363. rpc >= "5.9.0"
  364. rpclib
  365. rpclib-async
  366. rpclib-lwt
  367. rubytt
  368. SZXX >= "4.0.0"
  369. salsa20
  370. salsa20-core
  371. sanddb >= "0.2"
  372. scaml >= "1.5.0"
  373. scrypt-kdf
  374. secp256k1 >= "0.4.1"
  375. secp256k1-internal
  376. semver >= "0.2.1"
  377. sendmail
  378. sendmail-lwt
  379. sendmsg
  380. server-reason-react
  381. session-cookie
  382. session-cookie-async
  383. session-cookie-lwt
  384. sherlodoc
  385. slug
  386. sodium-fmt
  387. spin >= "0.6.0"
  388. squirrel
  389. ssh-agent
  390. ssl >= "0.6.0"
  391. stramon-lib
  392. styled-ppx
  393. tcpip >= "2.4.2" & < "4.0.0" | >= "5.0.1" & < "7.0.0"
  394. tdigest < "2.1.0"
  395. terminal_size >= "0.1.1"
  396. terminus
  397. terminus-cohttp
  398. terminus-hlc
  399. terml
  400. textrazor
  401. tezos-base-test-helpers < "13.0"
  402. tezos-client-base < "12.0"
  403. tezos-lmdb
  404. tezos-test-helpers < "11.0"
  405. tftp
  406. timedesc
  407. timere
  408. tls >= "0.12.0"
  409. toc
  410. topojson
  411. topojsone
  412. transept
  413. twostep
  414. type_eq
  415. type_id
  416. typebeat
  417. typeid >= "1.0.1"
  418. tyre >= "0.4"
  419. tyxml >= "4.0.0"
  420. tyxml-jsx
  421. tyxml-ppx >= "4.3.0"
  422. tyxml-syntax
  423. uecc
  424. ulid
  425. universal-portal
  426. unix-dirent
  427. unix-errno >= "0.3.0"
  428. unix-fcntl >= "0.3.0"
  429. unix-sys-resource
  430. unix-sys-stat
  431. unix-time
  432. unstrctrd
  433. user-agent-parser
  434. uspf
  435. uspf-lwt
  436. uspf-unix
  437. utop >= "2.13.0"
  438. validate
  439. validator
  440. vercel
  441. vpnkit
  442. wcwidth
  443. websocketaf
  444. x509 >= "0.7.0"
  445. xapi-rrd >= "1.8.2"
  446. xapi-stdext-date
  447. xapi-stdext-encodings
  448. xapi-stdext-std >= "4.16.0"
  449. yaml < "3.2.0"
  450. yaml-sexp
  451. yocaml
  452. yocaml_yaml
  453. yuscii >= "0.2.0"
  454. zar
  455. zed >= "3.2.2"
  456. zlist < "0.4.0"

Conflicts

None

OCaml

Innovation. Community. Security.

GitHub Discord Twitter Peertube RSS
About
Changelog Releases Industrial Users Academic Users Why OCaml
Ecosystem
Packages Community Events OCaml Planet Jobs
Resources
Install OCaml Get Started Platform Tools Language Manual Standard Library API Books Exercises Papers OCaml Playground Logo
Policies
Carbon Footprint Governance Privacy Code of Conduct