package uniq

Unic, a way to infer dependencies from an OCaml project

unic is a project that enables the analysis of an OCaml project (without relying on any particular build system such as dune) in order to construct a dependency graph for that project within a given context (such as that provided by opam). In this way, it is possible to identify dependencies that require object files which may have been compiled using what is known as the "host toolchain".

The aim is therefore to identify these dependencies and determine which opam packages they originate from, so as to inform the user what might need to be recompiled if they wish to build the project using a different "toolchain".

In this case, it is a solution that works well whenever one wishes to compile a unikernel using the Solo5 toolchain. Let us take the example of immuable. This project provides a unikernel as well as an executable (which should not be compiled with the Solo5 toolchain). One of the artefacts required for the unikernel is Documents, which is generated by mcrunch. You can then infer the dependencies and determine what needs to be recompiled using:

$ opam pin add https://git.robur.coop/robur/unic -y
$ git clone https://github.com/dinosaure/immuable
$ cd immuable
$ unic infer --recurse . --toolchain solo5 --exclude bin/ --ignore Documents \
  --prefer digestif.c --prefer checkseum.c
angstrom
bigstringaf
bin
bstr
cachet
carton
checkseum
cstruct
decompress
digestif
encore
faraday
fluxt
gmp
...

All that remains is for the user to find a way to recompile these opam packages using the Solo5 toolchain. If these packages use dune, simply vendorise them (i.e. retrieve them using opam source and specify a directory containing their source code using (vendored_dirs ...)). You can also use our mfetch to download dependencies.

Note about the project

The aim of this project is to separate the issue of resolving dependencies and their versions from that of resolving the artefacts required to compile a project, and specifically to compile a unikernel. It stems from two observations:

• the first is that, in the case of unikernels, there are dependencies that do not need to be vendorised, since ocaml-solo5 does not alter their compilation (compared to the default toolchain). In other words, it is possible to use pre-compiled artefacts and simply link them to the unikernel (as is the case, for example, with fmt or cmdliner)
• the second is that resolving dependencies via their versions is a difficult problem that has already been solved by opam. So let’s let opam do it! opam-monorepo attempts this approach, but the inability to find a solution (mainly because the ecosystem is changing) results in a cryptic message that is difficult to decipher without an in-depth understanding of the OCaml ecosystem...

A final point is the lack of maintenance that opam-monorepo may have suffered (and still suffers?) from, and the lack of commitment from the organisations behind this project to improve the situation.

With this in mind, we opted to use codept, created and maintained by @Octachron (who has in-depth knowledge of dependency resolution at the OCaml level). unic therefore relies on his work to resolve dependencies according to a given environment (by default, the OPAM environment).

The project is initially focused on compiling unikernels, but it can also be used to compile static programmes (and to determine what needs to be vendored to statically link an executable), since, fundamentally, a unikernel is (also) the result of static linking.

Some other tools

unic offers other tools that can help you understand a project:

• for example, you can inspect the configuration of an OCaml toolchain if you wish, using unic cfg
• you can also extract the signature of an artefact using unic digest (useful for tracking the links between artefacts)
• it is possible to qualify project's artefacts with unic qualify (based on their signatures) and, in particular, to identify what is missing
• it is also possible to resolve (with unic resolve) missing modules using ocamlfind's packages
• An introspection and search tool (unic info) is available to display an artefact (and its details) or to search for one
• A tool for introspection and searching META files (unic meta) is also available