package alcotest

  1. Overview
  2. Docs
Alcotest is a lightweight and colourful test framework.

Install

Dune Dependency

github.com Readme Changelog ISC License Edit opam file Versions (41)

Authors

  1. T
    Thomas Gazagnaire

Maintainers

  1. T
    thomas@gazagnaire.org

Sources

alcotest-0.8.4.tbz
sha256=8643b39e74317e2ec49be9a0986e73acd4fcd1277a6d59fb1423c3a6ebdfaf61
md5=c940f89a2bb2e23d3f1422ce61d1396b

Description

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Published: 17 Oct 2018

README

Alcotest is a lightweight and colourful test framework.

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Examples

A simple example:

(* Build with `ocamlbuild -pkg alcotest simple.byte` *)

(* A module with functions to test *)
module To_test = struct
  let capit letter = Char.uppercase letter
  let plus int_list = List.fold_left (fun a b -> a + b) 0 int_list
end

(* The tests *)
let capit () =
  Alcotest.(check char) "same chars"  'A' (To_test.capit 'a')

let plus () =
  Alcotest.(check int) "same ints" 7 (To_test.plus [1;1;2;3])

let test_set = [
  "Capitalize" , `Quick, capit;
  "Add entries", `Slow , plus ;
]

(* Run it *)
let () =
  Alcotest.run "My first test" [
    "test_set", test_set;
  ]

The result is a self-contained binary which displays the test results. Use ./simple.byte --help to see the runtime options.

$ ./simple.native
[OK]        test_set  0   Capitalize.
[OK]        test_set  1   Add entries.
Test Successful in 0.001s. 2 tests run.

You can filter which tests to run by supplying either the exact test name (which would run all testcases with that name), or the exact test name and test case number (which would run just that single test):

$ ./simple.native test test_set
Testing My first test.
[OK]              test_set          0   Capitalize.
[OK]              test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 test run.
$ ./simple.native test test_set 1
Testing My first test.
[SKIP]              test_set          0   Capitalize.
[OK]                test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.

Note that you cannot filter by test case name (i.e. Capitalize or Add entries), you have to use the test case number instead.

See the examples folder for more examples.

Quick and Slow tests

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

Passing custom options to the tests

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your prefered number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ test.exe test
test.exe: required option -n is missing

$ test.exe test -n 42
Testing foo.
[OK]                all          0   int.

Lwt

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type unit -> unit Lwt.t and alcotest-lwt calls Lwt_main.run for you.

However, there are a couple of extra features:

  • If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).

  • You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Alcotest.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ test.exe
Testing foo.
[ERROR]             all          0   one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken

Comparison with other testing frameworks

The README is pretty clear about that:

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

  • Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;

  • Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

  • qcheck qcheck does random generation and property testing (e.g. Quick Check)

  • crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.

  • ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.

Dependencies (7)

  1. uuidm
  2. cmdliner
  3. result < "1.5"
  4. astring
  5. fmt >= "0.8.0"
  6. jbuilder >= "1.0+beta10"
  7. ocaml >= "4.02.3"

Dev Dependencies

None

  1. ahrocksdb
  2. albatross >= "1.5.0"
  3. alcotest-async < "1.0.0"
  4. alcotest-lwt < "1.0.0"
  5. ambient-context
  6. ambient-context-eio
  7. angstrom >= "0.7.0"
  8. ansi >= "0.6.0"
  9. anycache >= "0.7.4"
  10. anycache-async
  11. anycache-lwt
  12. archetype >= "1.4.2"
  13. archi
  14. arp
  15. arp-mirage
  16. arrakis
  17. art
  18. asak >= "0.2"
  19. asli >= "0.2.0"
  20. asn1-combinators >= "0.2.2"
  21. atd >= "2.3.3"
  22. atdgen >= "2.10.0"
  23. atdpy
  24. atdts
  25. base32
  26. base64 >= "2.1.2"
  27. bechamel >= "0.5.0"
  28. bigarray-overlap
  29. bigstring >= "0.3"
  30. bigstring-unix >= "0.3"
  31. bigstringaf
  32. bitlib
  33. blake2
  34. bloomf
  35. bls12-381 < "0.4.1" | >= "3.0.0" & < "18.0"
  36. bls12-381-hash
  37. bls12-381-js >= "0.4.2"
  38. bls12-381-js-gen >= "0.4.2"
  39. bls12-381-legacy
  40. bls12-381-signature
  41. bls12-381-unix
  42. blurhash
  43. builder-web
  44. bulletml
  45. bytebuffer
  46. ca-certs
  47. ca-certs-nss
  48. cactus
  49. calendar >= "3.0.0"
  50. callipyge
  51. camlix
  52. camlkit
  53. camlkit-base
  54. capnp-rpc < "0.6.0"
  55. capnp-rpc-lwt < "0.3"
  56. carray
  57. carton
  58. cborl
  59. ccss >= "1.6"
  60. cf-lwt
  61. chacha
  62. channel
  63. charrua-client
  64. charrua-client-lwt
  65. charrua-client-mirage < "0.11.0"
  66. checked_oint < "0.1.1"
  67. checkseum >= "0.0.3"
  68. cid
  69. clarity-lang
  70. class_group_vdf
  71. cohttp >= "0.17.0"
  72. cohttp-curl-async
  73. cohttp-curl-lwt
  74. cohttp-eio >= "6.0.0~beta2"
  75. colombe
  76. color
  77. conan
  78. conan-cli
  79. conan-database
  80. conan-lwt
  81. conan-unix
  82. conduit = "3.0.0"
  83. conex < "0.10.0"
  84. conex-mirage-crypto
  85. conex-nocrypto
  86. cookie
  87. cow >= "2.2.0"
  88. css
  89. css-parser
  90. cstruct >= "3.3.0"
  91. cstruct-sexp
  92. ctypes-zarith
  93. cuid
  94. curly
  95. current_incr
  96. cwe_checker
  97. data-encoding < "1.0.0"
  98. datakit >= "0.12.0"
  99. datakit-bridge-github >= "0.12.0"
  100. datakit-ci
  101. datakit-client-git >= "0.12.0"
  102. decompress >= "0.8" & < "1.5.3"
  103. depyt
  104. digestif >= "0.7"
  105. dispatch >= "0.4.1"
  106. dkim
  107. dkim-bin
  108. dkim-mirage
  109. dns >= "4.0.0"
  110. dns-cli
  111. dns-client >= "4.6.0"
  112. dns-forward < "0.9.0"
  113. dns-forward-lwt-unix
  114. dns-resolver
  115. dns-server
  116. dns-tsig
  117. dnssd
  118. dnssec
  119. docfd >= "2.2.0"
  120. dog < "0.2.1"
  121. domain-name
  122. dream
  123. dream-pure
  124. duff
  125. dune-release >= "1.0.0"
  126. duration >= "0.1.1"
  127. emile
  128. encore
  129. eqaf >= "0.5"
  130. equinoxe
  131. equinoxe-cohttp
  132. equinoxe-hlc
  133. eris
  134. eris-lwt
  135. ezgzip
  136. ezjsonm >= "0.4.2" & < "1.3.0"
  137. ezjsonm-lwt < "1.3.0"
  138. FPauth
  139. FPauth-core
  140. FPauth-responses
  141. FPauth-strategies
  142. faraday != "0.2.0"
  143. farfadet
  144. fat-filesystem >= "0.12.0"
  145. ff
  146. ff-pbt
  147. fiat-p256
  148. flex-array
  149. fsevents-lwt
  150. functoria >= "2.2.0"
  151. functoria-runtime >= "2.2.0" & != "3.0.1" & < "4.0.0~beta1"
  152. geojson
  153. geoml >= "0.1.1"
  154. git = "1.4.10" | = "1.5.0" | >= "1.5.2" & != "1.10.0" & < "3.0.0"
  155. git-mirage < "3.0.0"
  156. git-unix >= "1.10.0" & != "2.1.0" & < "3.0.0"
  157. git_split
  158. gitlab-unix
  159. glicko2
  160. gmap >= "0.3.0"
  161. gpt
  162. graphql
  163. graphql-async
  164. graphql-cohttp >= "0.13.0"
  165. graphql-lwt
  166. graphql_parser
  167. graphql_ppx >= "0.7.1"
  168. h1_parser
  169. h2
  170. hacl
  171. hacl_func
  172. hacl_x25519 >= "0.2.0"
  173. highlexer
  174. hkdf
  175. hockmd
  176. html_of_jsx
  177. http
  178. http-multipart-formdata < "2.0.0"
  179. httpaf >= "0.2.0"
  180. httpun
  181. httpun-ws
  182. hvsock
  183. icalendar
  184. imagelib >= "20200929"
  185. index
  186. inferno >= "20220603"
  187. influxdb-async
  188. influxdb-lwt
  189. inquire < "0.2.0"
  190. interval-map
  191. iomux
  192. irmin < "0.8.0" | >= "0.9.6" & != "0.11.1" & < "1.2.0" | >= "2.0.0" & < "2.3.0"
  193. irmin-bench >= "2.7.0"
  194. irmin-chunk < "2.0.0" | >= "2.3.0"
  195. irmin-cli
  196. irmin-containers
  197. irmin-fs < "2.0.0" | >= "2.3.0"
  198. irmin-git < "2.0.0" | >= "2.3.0"
  199. irmin-http < "2.0.0"
  200. irmin-mem < "2.0.0"
  201. irmin-pack >= "2.4.0" & != "2.6.1"
  202. irmin-pack-tools
  203. irmin-test < "2.2.0"
  204. irmin-tezos
  205. irmin-tezos-utils
  206. irmin-unix >= "1.0.0" & < "2.0.0" | >= "2.4.0" & != "2.6.1"
  207. irmin-watcher
  208. jekyll-format
  209. jerboa
  210. jitsu
  211. jose
  212. json-data-encoding >= "0.9"
  213. json_decoder
  214. jsonxt
  215. junit_alcotest
  216. jwto
  217. ke >= "0.2"
  218. kkmarkdown
  219. lambda-runtime
  220. lambdapi >= "2.0.0"
  221. lambdoc >= "1.0-beta4"
  222. ledgerwallet-tezos >= "0.2.1" & < "0.4.0"
  223. lmdb >= "1.0"
  224. logical
  225. logtk >= "1.5.1"
  226. lp
  227. lp-glpk
  228. lp-glpk-js
  229. lp-gurobi
  230. lru
  231. lt-code
  232. luv
  233. mbr-format >= "1.0.0"
  234. mdx >= "1.6.0"
  235. mec
  236. mechaml >= "1.0.0"
  237. merge-queues >= "0.2.0"
  238. merge-ropes >= "0.2.0"
  239. metrics
  240. mirage >= "4.0.0~beta1"
  241. mirage-block-partition < "0.2.0"
  242. mirage-block-ramdisk >= "0.3"
  243. mirage-channel >= "4.0.0"
  244. mirage-channel-lwt
  245. mirage-crypto-ec
  246. mirage-flow >= "1.0.2" & < "1.2.0"
  247. mirage-flow-unix
  248. mirage-fs-mem
  249. mirage-fs-unix >= "1.2.0"
  250. mirage-kv >= "2.0.0"
  251. mirage-kv-mem
  252. mirage-kv-unix
  253. mirage-logs >= "0.3.0"
  254. mirage-nat
  255. mirage-net-unix >= "2.3.0"
  256. mirage-runtime >= "4.0.0~beta1" & < "4.5.0"
  257. mirage-tc
  258. mjson
  259. mnd
  260. monocypher
  261. mrmime
  262. mrt-format
  263. multibase
  264. multihash
  265. multihash-digestif
  266. multipart-form-data
  267. multipart_form
  268. multipart_form-eio
  269. multipart_form-lwt
  270. named-pipe
  271. nanoid
  272. nbd >= "4.0.3"
  273. nbd-tool
  274. nloge
  275. nocoiner
  276. non_empty_list
  277. OCADml >= "0.6.0"
  278. ocaml-r >= "0.5.0"
  279. ocaml-version >= "3.1.0"
  280. ocamlformat >= "0.13.0" & != "0.19.0~4.13preview" & < "0.25.1"
  281. ocamlformat-rpc < "removed"
  282. ocamline
  283. odoc >= "1.4.0" & < "2.1.0"
  284. ohex
  285. oidc
  286. opam-0install
  287. opam-file-format >= "2.1.1"
  288. opentelemetry >= "0.6"
  289. opentelemetry-client-cohttp-lwt >= "0.6"
  290. opentelemetry-client-ocurl >= "0.6"
  291. opentelemetry-cohttp-lwt >= "0.6"
  292. opentelemetry-lwt >= "0.6"
  293. opium >= "0.15.0"
  294. opium-graphql
  295. opium-testing
  296. opium_kernel
  297. orewa
  298. ortac-core
  299. osx-acl
  300. osx-attr
  301. osx-cf
  302. osx-fsevents
  303. osx-membership
  304. osx-mount
  305. osx-xattr
  306. otoggl
  307. owl >= "0.6.0" & != "0.9.0" & != "1.0.0"
  308. owl-base < "0.5.0"
  309. owl-ode >= "0.1.0" & != "0.2.0"
  310. owl-symbolic
  311. passmaker
  312. patch
  313. pbkdf
  314. pecu >= "0.2"
  315. pf-qubes
  316. pg_query >= "0.9.6"
  317. phylogenetics
  318. piaf
  319. polyglot
  320. polynomial
  321. ppx_blob >= "0.3.0"
  322. ppx_deriving_cmdliner
  323. ppx_deriving_rpc
  324. ppx_deriving_yaml
  325. ppx_graphql >= "0.2.0"
  326. ppx_mysql
  327. ppx_protocol_conv >= "5.0.0"
  328. ppx_protocol_conv_json >= "5.0.0"
  329. ppx_protocol_conv_jsonm >= "5.0.0"
  330. ppx_protocol_conv_msgpack >= "5.0.0"
  331. ppx_protocol_conv_xml_light >= "5.0.0"
  332. ppx_protocol_conv_xmlm
  333. ppx_protocol_conv_yaml >= "5.0.0"
  334. ppx_subliner
  335. ppx_units
  336. ppx_yojson >= "1.1.0"
  337. pratter
  338. prc
  339. preface
  340. pretty_expressive
  341. prettym
  342. proc-smaps
  343. producer < "0.2.0"
  344. prometheus < "1.2"
  345. prometheus-app
  346. protocell
  347. protocol-9p >= "0.3" & < "0.11.0" | >= "0.11.2"
  348. protocol-9p-unix
  349. psq
  350. qcheck >= "0.18" & < "0.22"
  351. qcheck-alcotest < "0.22"
  352. quickjs
  353. radis
  354. randii
  355. reason-standard
  356. reparse >= "2.0.0" & < "3.0.0"
  357. reparse-unix < "2.1.0"
  358. resp < "0.10.0"
  359. resp-unix
  360. rfc1951 < "1.0.0"
  361. routes < "2.0.0"
  362. rpc >= "5.9.0"
  363. rpclib
  364. rpclib-async
  365. rpclib-lwt
  366. rpmfile
  367. rubytt
  368. SZXX >= "4.0.0"
  369. salsa20
  370. salsa20-core
  371. sanddb >= "0.2"
  372. scaml >= "1.5.0"
  373. scrypt-kdf
  374. secp256k1 >= "0.4.1"
  375. secp256k1-internal
  376. semver >= "0.2.1"
  377. sendmail
  378. sendmail-lwt
  379. sendmsg
  380. server-reason-react
  381. session-cookie
  382. session-cookie-async
  383. session-cookie-lwt
  384. sherlodoc
  385. slug
  386. sodium-fmt
  387. spin >= "0.6.0"
  388. squirrel
  389. ssh-agent
  390. ssl >= "0.6.0"
  391. stramon-lib
  392. styled-ppx
  393. tcpip >= "2.4.2" & < "4.0.0" | >= "5.0.1" & < "7.0.0"
  394. tdigest < "2.1.0"
  395. term-indexing
  396. terminal_size >= "0.1.1"
  397. terminus
  398. terminus-cohttp
  399. terminus-hlc
  400. terml
  401. textrazor
  402. tezos-base-test-helpers < "13.0"
  403. tezos-client-base < "12.0"
  404. tezos-lmdb
  405. tezos-test-helpers < "11.0"
  406. tftp
  407. timedesc
  408. timere
  409. tls >= "0.12.0"
  410. toc
  411. topojson
  412. topojsone
  413. transept
  414. twostep
  415. type_eq
  416. type_id
  417. typebeat
  418. typeid >= "1.0.1"
  419. tyre >= "0.4"
  420. tyxml >= "4.0.0"
  421. tyxml-jsx
  422. tyxml-ppx >= "4.3.0"
  423. tyxml-syntax
  424. uecc
  425. ulid
  426. universal-portal
  427. unix-dirent
  428. unix-errno >= "0.3.0"
  429. unix-fcntl >= "0.3.0"
  430. unix-sys-resource
  431. unix-sys-stat
  432. unix-time
  433. unstrctrd
  434. user-agent-parser
  435. uspf
  436. uspf-lwt
  437. uspf-unix
  438. utop >= "2.13.0"
  439. validate
  440. validator
  441. vercel
  442. vpnkit
  443. wcwidth
  444. websocketaf
  445. x509 >= "0.7.0"
  446. xapi-rrd >= "1.8.2"
  447. xapi-stdext-date
  448. xapi-stdext-encodings
  449. xapi-stdext-std >= "4.16.0"
  450. yaml < "3.2.0"
  451. yaml-sexp
  452. yocaml
  453. yocaml_yaml
  454. yuscii >= "0.2.0"
  455. zar
  456. zed >= "3.2.2"
  457. zlist < "0.4.0"

Conflicts

None

OCaml

Innovation. Community. Security.

GitHub Discord Twitter Peertube RSS
About
Changelog Releases Industrial Users Academic Users Why OCaml
Ecosystem
Packages Community Events OCaml Planet Jobs
Resources
Install OCaml Get Started Platform Tools Language Manual Standard Library API Books Exercises Papers OCaml Playground Logo
Policies
Carbon Footprint Governance Privacy Code of Conduct