package alcotest

  1. Overview
  2. Docs
Alcotest is a lightweight and colourful test framework

Install

Dune Dependency

github.com Readme Changelog ISC License Edit opam file Versions (40)

Authors

  1. T
    Thomas Gazagnaire

Maintainers

  1. T
    thomas@gazagnaire.org

Sources

alcotest-0.8.5.tbz
sha256=6b3b638fc7c6f4c52617b0261dc9f726ce21bb0c485e05bab6fe41a57697fc6b
md5=2db36741c413ab93391ecc1f983aa804

Description

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Published: 17 Jan 2019

README

Alcotest is a lightweight and colourful test framework.

Alcotest exposes simple interface to perform unit tests. It exposes a simple TESTABLE module type, a check function to assert test predicates and a run function to perform a list of unit -> unit test callbacks.

Alcotest provides a quiet and colorful output where only faulty runs are fully displayed at the end of the run (with the full logs ready to inspect), with a simple (yet expressive) query language to select the tests to run.

Examples

A simple example:

(* Build with `ocamlbuild -pkg alcotest simple.byte` *)

(* A module with functions to test *)
module To_test = struct
  let capit letter = Char.uppercase letter
  let plus int_list = List.fold_left (fun a b -> a + b) 0 int_list
end

(* The tests *)
let capit () =
  Alcotest.(check char) "same chars"  'A' (To_test.capit 'a')

let plus () =
  Alcotest.(check int) "same ints" 7 (To_test.plus [1;1;2;3])

let test_set = [
  "Capitalize" , `Quick, capit;
  "Add entries", `Slow , plus ;
]

(* Run it *)
let () =
  Alcotest.run "My first test" [
    "test_set", test_set;
  ]

The result is a self-contained binary which displays the test results. Use ./simple.byte --help to see the runtime options.

$ ./simple.native
[OK]        test_set  0   Capitalize.
[OK]        test_set  1   Add entries.
Test Successful in 0.001s. 2 tests run.

You can filter which tests to run by supplying either the exact test name (which would run all testcases with that name), or the exact test name and test case number (which would run just that single test):

$ ./simple.native test test_set
Testing My first test.
[OK]              test_set          0   Capitalize.
[OK]              test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 2 test run.
$ ./simple.native test test_set 1
Testing My first test.
[SKIP]              test_set          0   Capitalize.
[OK]                test_set          1   Add entries.
The full test results are available in `_build/_tests`.
Test Successful in 0.000s. 1 test run.

Note that you cannot filter by test case name (i.e. Capitalize or Add entries), you have to use the test case number instead.

See the examples folder for more examples.

Quick and Slow tests

In general you should use `Quick tests: tests that are ran on any invocations of the test suite. You should only use `Slow tests for stress tests that are ran only on occasion (typically before a release or after a major change). These slow tests can be suppressed by passing the -q flag on the command line, e.g.:

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

Passing custom options to the tests

In most cases, the base tests are unit -> unit functions. However, it is also possible to pass an extra option to all the test functions by using 'a -> unit, where 'a is the type of the extra parameter.

In order to do this, you need to specify how this extra parameter is read on the command-line, by providing a Cmdliner term for command-line arguments which explains how to parse and serialize values of type 'a (note: do not use positional arguments, only optional arguments are supported).

For instance:

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your prefered number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Will generate test.exe such that:

$ test.exe test
test.exe: required option -n is missing

$ test.exe test -n 42
Testing foo.
[OK]                all          0   int.

Lwt

Alcotest provides an Alcotest_lwt module that you could use to wrap Lwt test cases. The basic idea is that instead of providing a test function in the form unit -> unit, you provide one with the type unit -> unit Lwt.t and alcotest-lwt calls Lwt_main.run for you.

However, there are a couple of extra features:

  • If an async exception occurs, it will cancel your test case for you and fail it (rather than exiting the process).

  • You get given a switch, which will be turned off when the test case finishes (or fails). You can use that to free up any resources.

For instance:

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Alcotest.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

Will generate:

$ test.exe
Testing foo.
[ERROR]             all          0   one.
-- all.000 [one.] Failed --
in _build/_tests/all.000.output:
freeing all resources
[failure] All is broken

Screenshots

The following screenshots demonstrate the HTML testing output from the odoc project.

All tests passed Some tests failed Failed test with custom diffing
ok err diff

Comparison with other testing frameworks

The README is pretty clear about that:

Alcotest is the only testing framework using colors!

More seriously, Alcotest is similar to ounit but it fixes a few of the problems found in that library:

  • Alcotest has a nicer output, it is easier to see what failed and what succeeded and to read the log outputs of the failed tests;

  • Alcotest uses combinators to define pretty-printers and comparators between the things to test.

Other nice tools doing different kind of testing also exist:

  • qcheck qcheck does random generation and property testing (e.g. Quick Check)

  • crowbar and bun are similar to qcheck, but use compiler-directed randomness, e.g. it takes advantage of the AFL support the OCaml compiler.

  • ppx_inline_tests allows to write tests in the same file as your source-code; they will be run only in a special mode of compilation.

Dependencies (7)

  1. uuidm
  2. cmdliner
  3. result < "1.5"
  4. astring
  5. fmt >= "0.8.0"
  6. ocaml >= "4.02.3"
  7. dune >= "1.1.0"

Dev Dependencies

None

  1. ahrocksdb
  2. albatross >= "1.5.0"
  3. alcotest-async < "1.0.0"
  4. alcotest-lwt < "1.0.0"
  5. alg_structs_qcheck
  6. ambient-context
  7. ambient-context-eio
  8. ambient-context-lwt
  9. angstrom >= "0.7.0"
  10. ansi >= "0.6.0"
  11. anycache >= "0.7.4"
  12. anycache-async
  13. anycache-lwt
  14. archetype >= "1.4.2"
  15. archi
  16. arp
  17. arp-mirage
  18. arrakis
  19. art
  20. asak >= "0.2"
  21. asli >= "0.2.0"
  22. asn1-combinators >= "0.2.2"
  23. atd >= "2.3.3"
  24. atdgen >= "2.10.0"
  25. atdpy
  26. atdts
  27. base32
  28. base64 >= "2.1.2"
  29. bechamel >= "0.5.0"
  30. bigarray-overlap
  31. bigstring >= "0.3"
  32. bigstring-unix >= "0.3"
  33. bigstringaf
  34. bitlib
  35. blake2
  36. bloomf
  37. bls12-381 < "0.4.1" | >= "3.0.0" & < "18.0"
  38. bls12-381-hash
  39. bls12-381-js >= "0.4.2"
  40. bls12-381-js-gen >= "0.4.2"
  41. bls12-381-legacy
  42. bls12-381-signature
  43. bls12-381-unix
  44. blurhash
  45. builder-web
  46. bulletml
  47. bytebuffer
  48. ca-certs
  49. ca-certs-nss
  50. cactus
  51. caldav
  52. calendar >= "3.0.0"
  53. callipyge
  54. camlix
  55. camlkit
  56. camlkit-base
  57. capnp-rpc < "0.6.0"
  58. capnp-rpc-lwt < "0.3"
  59. carray
  60. carton
  61. cborl
  62. ccss >= "1.6"
  63. cf-lwt
  64. chacha
  65. channel
  66. charrua-client
  67. charrua-client-lwt
  68. charrua-client-mirage < "0.11.0"
  69. checked_oint < "0.1.1"
  70. checkseum >= "0.0.3"
  71. cid
  72. clarity-lang
  73. class_group_vdf
  74. cohttp >= "0.17.0"
  75. cohttp-curl-async
  76. cohttp-curl-lwt
  77. cohttp-eio >= "6.0.0~beta2"
  78. colombe
  79. color
  80. conan
  81. conan-cli
  82. conan-database
  83. conan-lwt
  84. conan-unix
  85. conduit = "3.0.0"
  86. conex < "0.10.0"
  87. conex-mirage-crypto
  88. conex-nocrypto
  89. cookie
  90. cow >= "2.2.0"
  91. css
  92. css-parser
  93. cstruct >= "3.3.0"
  94. cstruct-sexp
  95. ctypes-zarith
  96. cuid
  97. curly
  98. current_incr
  99. cwe_checker
  100. data-encoding < "1.0.0"
  101. datakit >= "0.12.0"
  102. datakit-bridge-github >= "0.12.0"
  103. datakit-ci
  104. datakit-client-git >= "0.12.0"
  105. decompress >= "0.8" & < "1.5.3"
  106. depyt
  107. digestif >= "0.7"
  108. dispatch >= "0.4.1"
  109. dkim
  110. dkim-bin
  111. dkim-mirage
  112. dns >= "4.0.0"
  113. dns-cli
  114. dns-client >= "4.6.0"
  115. dns-forward < "0.9.0"
  116. dns-forward-lwt-unix
  117. dns-resolver
  118. dns-server
  119. dns-tsig
  120. dnssd
  121. dnssec
  122. docfd >= "2.2.0"
  123. dog < "0.2.1"
  124. domain-name
  125. dream
  126. dream-pure
  127. duff
  128. dune-release >= "1.0.0"
  129. duration >= "0.1.1"
  130. emile
  131. encore
  132. eqaf >= "0.5"
  133. equinoxe
  134. equinoxe-cohttp
  135. equinoxe-hlc
  136. eris
  137. eris-lwt
  138. ezgzip
  139. ezjsonm >= "0.4.2" & < "1.3.0"
  140. ezjsonm-lwt < "1.3.0"
  141. FPauth
  142. FPauth-core
  143. FPauth-responses
  144. FPauth-strategies
  145. faraday != "0.2.0"
  146. farfadet
  147. fat-filesystem >= "0.12.0"
  148. ff
  149. ff-pbt
  150. fiat-p256
  151. flex-array
  152. fsevents-lwt
  153. functoria >= "2.2.0"
  154. functoria-runtime >= "2.2.0" & != "3.0.1" & < "4.0.0~beta1"
  155. geojson
  156. geoml >= "0.1.1"
  157. git = "1.4.10" | = "1.5.0" | >= "1.5.2" & != "1.10.0" & < "3.0.0"
  158. git-mirage < "3.0.0"
  159. git-unix >= "1.10.0" & != "2.1.0" & < "3.0.0"
  160. gitlab-unix
  161. glicko2
  162. gmap >= "0.3.0"
  163. gobba
  164. gpt
  165. graphql
  166. graphql-async
  167. graphql-cohttp >= "0.13.0"
  168. graphql-lwt
  169. graphql_parser
  170. graphql_ppx >= "0.7.1"
  171. h1_parser
  172. h2
  173. hacl
  174. hacl_func
  175. hacl_x25519 >= "0.2.0"
  176. highlexer
  177. hkdf
  178. hockmd
  179. html_of_jsx
  180. http
  181. http-multipart-formdata < "2.0.0"
  182. httpaf >= "0.2.0"
  183. hvsock
  184. icalendar
  185. imagelib >= "20200929"
  186. index
  187. inferno >= "20220603"
  188. influxdb-async
  189. influxdb-lwt
  190. inquire < "0.2.0"
  191. interval-map
  192. iomux
  193. irmin < "0.8.0" | >= "0.9.6" & != "0.11.1" & < "1.2.0" | >= "2.0.0" & < "2.3.0"
  194. irmin-bench >= "2.7.0"
  195. irmin-chunk < "2.0.0" | >= "2.3.0"
  196. irmin-cli
  197. irmin-containers
  198. irmin-fs < "2.0.0" | >= "2.3.0"
  199. irmin-git < "2.0.0" | >= "2.3.0"
  200. irmin-http < "2.0.0"
  201. irmin-mem < "2.0.0"
  202. irmin-pack >= "2.4.0" & != "2.6.1"
  203. irmin-pack-tools
  204. irmin-test < "2.2.0"
  205. irmin-tezos
  206. irmin-tezos-utils
  207. irmin-unix >= "1.0.0" & < "2.0.0" | >= "2.4.0" & != "2.6.1"
  208. irmin-watcher
  209. jekyll-format
  210. jerboa
  211. jitsu
  212. jose
  213. json-data-encoding >= "0.9"
  214. json_decoder
  215. jsonxt
  216. junit_alcotest
  217. jwto
  218. ke >= "0.2"
  219. kkmarkdown
  220. lambda-runtime
  221. lambdapi >= "2.0.0"
  222. lambdoc >= "1.0-beta4"
  223. ledgerwallet-tezos >= "0.2.1" & < "0.4.0"
  224. lmdb >= "1.0"
  225. logical
  226. logtk >= "1.5.1"
  227. lp
  228. lp-glpk
  229. lp-glpk-js
  230. lp-gurobi
  231. lru
  232. lt-code
  233. luv
  234. mbr-format >= "1.0.0"
  235. mdx >= "1.6.0"
  236. mec
  237. mechaml >= "1.0.0"
  238. merge-queues >= "0.2.0"
  239. merge-ropes >= "0.2.0"
  240. metrics
  241. minicaml >= "0.3.1"
  242. mirage >= "4.0.0~beta1"
  243. mirage-block-partition < "0.2.0"
  244. mirage-block-ramdisk >= "0.3"
  245. mirage-channel >= "4.0.0"
  246. mirage-channel-lwt
  247. mirage-crypto-ec
  248. mirage-flow >= "1.0.2" & < "1.2.0"
  249. mirage-flow-unix
  250. mirage-fs-mem
  251. mirage-fs-unix >= "1.2.0"
  252. mirage-kv >= "2.0.0"
  253. mirage-kv-mem
  254. mirage-kv-unix
  255. mirage-logs >= "0.3.0"
  256. mirage-nat
  257. mirage-net-unix >= "2.3.0"
  258. mirage-runtime >= "4.0.0~beta1" & < "4.5.0"
  259. mirage-tc
  260. mjson
  261. mmdb
  262. mnd
  263. monocypher
  264. mrmime
  265. mrt-format
  266. msgpck >= "1.6"
  267. multibase
  268. multihash
  269. multihash-digestif
  270. multipart-form-data
  271. multipart_form
  272. multipart_form-eio
  273. multipart_form-lwt
  274. named-pipe
  275. nanoid
  276. nbd >= "4.0.3"
  277. nbd-tool
  278. nloge
  279. nocoiner
  280. non_empty_list
  281. OCADml >= "0.6.0"
  282. ocaml-r >= "0.5.0"
  283. ocaml-version >= "3.1.0"
  284. ocamlformat >= "0.13.0" & != "0.19.0~4.13preview" & < "0.25.1"
  285. ocamlformat-rpc < "removed"
  286. ocamline
  287. odoc >= "1.4.0" & < "2.1.0"
  288. ohex
  289. oidc
  290. opam-0install
  291. opam-file-format >= "2.1.1"
  292. opentelemetry >= "0.6"
  293. opentelemetry-client-cohttp-lwt >= "0.6"
  294. opentelemetry-client-ocurl >= "0.6"
  295. opentelemetry-cohttp-lwt >= "0.6"
  296. opentelemetry-lwt >= "0.6"
  297. opium >= "0.15.0"
  298. opium-graphql
  299. opium-testing
  300. opium_kernel
  301. orewa
  302. ortac-core
  303. osx-acl
  304. osx-attr
  305. osx-cf
  306. osx-fsevents
  307. osx-membership
  308. osx-mount
  309. osx-xattr
  310. otoggl
  311. owl >= "0.6.0" & != "0.9.0" & != "1.0.0"
  312. owl-base < "0.5.0"
  313. owl-ode >= "0.1.0" & != "0.2.0"
  314. owl-symbolic
  315. passmaker
  316. patch
  317. pbkdf
  318. pecu >= "0.2"
  319. pf-qubes
  320. pg_query >= "0.9.6"
  321. phylogenetics
  322. piaf
  323. polyglot
  324. polynomial
  325. ppx_blob >= "0.3.0"
  326. ppx_deriving_cmdliner
  327. ppx_deriving_rpc
  328. ppx_deriving_yaml
  329. ppx_graphql >= "0.2.0"
  330. ppx_mysql
  331. ppx_parser
  332. ppx_protocol_conv >= "5.0.0"
  333. ppx_protocol_conv_json >= "5.0.0"
  334. ppx_protocol_conv_jsonm >= "5.0.0"
  335. ppx_protocol_conv_msgpack >= "5.0.0"
  336. ppx_protocol_conv_xml_light >= "5.0.0"
  337. ppx_protocol_conv_xmlm
  338. ppx_protocol_conv_yaml >= "5.0.0"
  339. ppx_subliner
  340. ppx_units
  341. ppx_yojson >= "1.1.0"
  342. pratter
  343. prc
  344. preface
  345. pretty_expressive
  346. prettym
  347. proc-smaps
  348. producer < "0.2.0"
  349. prom < "0.2"
  350. prometheus < "1.2"
  351. prometheus-app
  352. protocell
  353. protocol-9p >= "0.3" & < "0.11.0" | >= "0.11.2"
  354. protocol-9p-unix
  355. psq
  356. qcheck >= "0.18"
  357. qcheck-alcotest
  358. qcheck-core >= "0.18"
  359. quickjs
  360. radis
  361. randii
  362. reason-standard
  363. reparse >= "2.0.0" & < "3.0.0"
  364. reparse-unix < "2.1.0"
  365. resp < "0.10.0"
  366. resp-unix
  367. rfc1951 < "1.0.0"
  368. routes < "2.0.0"
  369. rpc >= "5.9.0"
  370. rpclib
  371. rpclib-async
  372. rpclib-lwt
  373. rubytt
  374. SZXX >= "4.0.0"
  375. salsa20
  376. salsa20-core
  377. sanddb >= "0.2"
  378. scaml >= "1.5.0"
  379. scrypt-kdf
  380. secp256k1 >= "0.4.1"
  381. secp256k1-internal
  382. semver >= "0.2.1"
  383. sendmail
  384. sendmail-lwt
  385. sendmsg
  386. server-reason-react
  387. session-cookie
  388. session-cookie-async
  389. session-cookie-lwt
  390. sherlodoc
  391. slug
  392. sodium-fmt
  393. spin >= "0.6.0"
  394. squirrel
  395. ssh-agent
  396. ssl >= "0.6.0"
  397. stramon-lib
  398. styled-ppx
  399. syslog-rfc5424
  400. tcpip >= "2.4.2" & < "4.0.0" | >= "5.0.1" & < "7.0.0"
  401. tdigest < "2.1.0"
  402. terminal_size >= "0.1.1"
  403. terminus
  404. terminus-cohttp
  405. terminus-hlc
  406. terml
  407. textrazor
  408. tezos-base-test-helpers < "13.0"
  409. tezos-client-base < "12.0"
  410. tezos-crypto < "8.0"
  411. tezos-lmdb
  412. tezos-signer-backends < "8.0"
  413. tezos-stdlib < "8.0"
  414. tezos-test-helpers < "11.0"
  415. tftp
  416. timedesc
  417. timere
  418. tls >= "0.12.0"
  419. toc
  420. topojson
  421. topojsone
  422. transept
  423. twostep
  424. type_eq
  425. type_id
  426. typebeat
  427. typeid >= "1.0.1"
  428. tyre >= "0.4"
  429. tyxml >= "4.0.0"
  430. tyxml-jsx
  431. tyxml-ppx >= "4.3.0"
  432. tyxml-syntax
  433. uecc
  434. ulid
  435. universal-portal
  436. unix-dirent
  437. unix-errno >= "0.3.0"
  438. unix-fcntl >= "0.3.0"
  439. unix-sys-resource
  440. unix-sys-stat
  441. unix-time
  442. unstrctrd
  443. user-agent-parser
  444. uspf
  445. uspf-lwt
  446. uspf-unix
  447. utop >= "2.13.0"
  448. validate
  449. validator
  450. vercel
  451. vpnkit
  452. wcwidth
  453. websocketaf
  454. x509 >= "0.7.0"
  455. xapi-rrd >= "1.8.2"
  456. xapi-stdext-date
  457. xapi-stdext-encodings
  458. xapi-stdext-std >= "4.16.0"
  459. yaml < "3.2.0"
  460. yaml-sexp
  461. yocaml
  462. yocaml_yaml
  463. yojson >= "1.6.0"
  464. yuscii >= "0.2.0"
  465. zar
  466. zed >= "3.2.2"
  467. zlist < "0.4.0"

Conflicts

None

OCaml

Innovation. Community. Security.

GitHub Discord Twitter Peertube RSS
About
Changelog Releases Industrial Users Academic Users Why OCaml
Ecosystem
Packages Community Events OCaml Planet Jobs
Resources
Install OCaml Get Started Platform Tools Language Manual Standard Library API Books Exercises Papers OCaml Playground Logo
Policies
Carbon Footprint Governance Privacy Code of Conduct