OCaml Changelog

Discuss this post on Discuss!

Dune can install and run developer tools in the context of a project. This feature is available in the Dune Developer Preview and in the upcoming release of Dune 3.17. As with all of Dune's package management features, consider this feature to be unstable as its UI and semantics may change without notice.

The currently supported tools are ocamllsp and ocamlformat. Dune has a new command dune tools exec <TOOL> -- [ARGS]... which downloads and installs the given tool, and then runs it with the given arguments (note the -- which separates arguments to dune from arguments to the tool). Tools are installed locally to the project, in its _build directory, which makes it easy to use different versions of a tool in different projects. An unfortunate consequence of installing tools into _build is that for the time being all tools are uninstalled whenever dune clean is run.

Let's see it in action:

$ dune tools exec ocamlformat -- --version
Solution for dev-tools.locks/ocamlformat:
- ocamlformat.0.26.2+binary-ocaml-5.2.0-built-2024-11-07.0-x86_64-unknown-linux-musl
    Building ocamlformat.0.26.2+binary-ocaml-5.2.0-built-2024-11-07.0-x86_64-unknown-linux-musl
     Running 'ocamlformat --version'
0.26.2

Precompiled Binaries

Note that in the example above, Dune's package solver chose to install version 0.26.2+binary-ocaml-5.2.0-built-2024-11-07.0-x86_64-unknown-linux-musl of ocamlformat. This packages comes from a new repository of binary packages containing pre-built executables for a select few Opam packages. Dune will search this repository in addition to the default repositories when solving packages for tools only (if a project has ocamlformat in its dependencies, the binary repository won't be searched while solving the project's dependencies).

The goal of the binary repository is to reduce the time it takes to get started working on a new project. Without it, Dune would need to build ocamlformat from source along with all of its dependencies, which can take several minutes.

For now only a small number of package versions are contained in the binary repository. To demonstrate, here's what happens if we run dune tools exec ocamlformat in a project with version=0.26.1 in its .ocamlformat file:

 $ dune tools exec ocamlformat -- --version
Solution for dev-tools.locks/ocamlformat:
- astring.0.8.5
- base.v0.17.1
- base-bytes.base
- base-unix.base
- camlp-streams.5.0.1
- cmdliner.1.3.0
...
- ocamlformat.0.26.1
...
    Building base-unix.base
    Building ocaml-base-compiler.5.1.1
    Building ocaml-config.3
    Building ocaml.5.1.1
    Building seq.base
    Building cmdliner.1.3.0
...
    Building ocamlformat.0.26.1
     Running 'ocamlformat --version'
0.26.1

Dune parses .ocamlformat to determine which version of ocamlformat to install, and 0.26.1 is not in the binary repo so it needed to be built from source.

If your project requires a version of a package not available in the binary repository, or you're on an operating system or architecture for which no binary version of a package exists, the package will be built from source instead. Currently the binary repository contains binaries of ocamlformat.0.26.2, ocaml-lsp-server.1.18.0 and ocaml-lsp-server.1.19.0 for x86_64-unknown-linux-musl, x86_64-apple-darwin and aarch64-apple-darwin.

Note that Linux binaries are statically linked with muslc so they should work on all distros regardless of dynamic linker.

Running ocamllsp

The program ocamllsp from the package ocaml-lsp-server analyzes OCaml code and sends information to text editors using the Language Server Protocol. The tool is crucial to OCaml's editor integration and it has a couple of quirks that are worth mentioning here.

TL;DR: Install Dune with the install script on the Developer Preview page and you'll get an ocamllsp shell script that will install and run the correct version of ocamllsp for your project.

Firstly the ocamllsp executable can only analyze code that has been compiled with the same version of the OCaml compiler as was used to compile the ocamllsp executable itself. Different versions of the ocaml-lsp-server package are incompatible with some versions of the OCaml compiler (e.g. ocaml-lsp-server.1.19.0 must be built with at least 5.2.0 of the compiler). This means that when Dune is choosing which version of ocaml-lsp-server to install it needs to know which version of the compiler your project is using. This is only known after the project has been locked (by running dune pkg lock), so Dune will refuse to install ocamllsp in a project that doesn't have a lock directory or for a project that doesn't depend on the OCaml compiler.

$ dune tools exec ocamllsp
Error: Unable to load the lockdir for the default build context.
Hint: Try running 'dune pkg lock'

The ocaml-lsp-server packages in the binary repository contain metadata to ensure that the ocamllsp executable that gets installed was built with the same version of the compiler as your project. For example the ocaml-lsp-server package built with ocaml.5.2.0 contains this line:

conflicts: "ocaml" {!= "5.2.0"}

This prevents it from being chosen if the project depends on any version of the compiler other than 5.2.0.

Another quirk is that ocamllsp will try to invoke the binaries ocamlformat and ocamlformat-rpc, both found in the ocamlformat package. The ocaml-lsp-server package doesn't depend on ocamlformat as the specific version of ocamlformat needed by a project is implied by the project's .ocamlformat file, which package managers don't consider when solving dependencies. This means that in general (whether using Dune or Opam for package management) it's up to the user to make sure that the correct version of ocamlformat is installed in order to use the formatting features of ocamllsp.

Otherwise expect this error in your editor:

Unable to find 'ocamlformat-rpc' binary. Types on hover may not be well-formatted. You need to install either 'ocamlformat' of version > 0.21.0 or, otherwise, 'ocamlformat-rpc' package.

Even if ocamllsp and ocamlformat are both installed by Dune, if you run dune tools exec ocamllsp you will find that ocamllsp still can't find the ocamlformat or ocamlformat-rpc executables. This is because unlike Opam, Dune does not install tools into your $PATH, and for the sake of simplicity, the dune tools exec <TOOL> command does not modify the environment of the tool it launches. This can be fixed by adding _build/_private/default/.dev-tool/ocamlformat/ocamlformat/target/bin (the directory containing ocamlformat and ocamlformat-rpc when ocamlformat is installed by dune) to the start of your $PATH variable before running dune tools exec ocamllsp. For example starting ocamllsp with the following shell script:

OCAMLFORMAT_TARGET="_build/_private/default/.dev-tool/ocamlformat/ocamlformat/target"

if [ ! -f $OCAMLFORMAT_TARGET/cookie ]; then
    # Make sure that the ocamlformat dev tool is installed as it's needed by
    # ocamllsp. There's currently no command that just installs ocamlformat so
    # we need to run it and ignore the result.
    dune tools exec ocamlformat -- --help > /dev/null
fi

# Add ocamlformat to the environment in which ocamllsp runs so ocamllsp can invoke ocamlformat.
export PATH="$PWD/$OCAMLFORMAT_TARGET/bin:$PATH"

# Build and run ocamllsp.
dune tools exec ocamllsp -- "$@"

Of course, it's rare to manually start ocamllsp directly from your terminal. It's normally launched by text editors. It would be impractical to configure your text editor to modify $PATH and run a custom command to start ocamllsp via Dune, and doing so would make it impossible to edit any project that doesn't use Dune for package management. Instead, the Dune Developer Preview ships with a shell script which installs ocamlformat and adds its bin directory to $PATH before launching dune tools exec ocamllsp. The script is simply named ocamllsp, and the Dune Developer Preview install script adds it to ~/.dune/bin which should already be in your $PATH if you're using the Developer Preview. The ocamllsp script also attempts to fall back to an Opam-managed installation of ocamllsp if it doesn't detect a Dune lockdir so the same script should work for non-Dune projects. Because the script is named the same as the ocamllsp executable, most editors don't require special configuration to run it. See the "Editor Configuration" section of the Dune Developer Preview page for more information about setting up your editor.

Some parts of the ocamllsp shell script may eventually make their way into Dune itself, but for the time being the shell script is the recommended way to launch ocamllsp for users of the Dune Developer Preview. The net result is that as long as your project has a lockfile, the first time you edit some OCaml code in the project Dune will download and run the appropriate version of ocamllsp.

Discuss this post on Discuss!

Support for dune shell completions for bash and zsh has just landed in the Dune Developer Preview!

Running the installer adds a snippet to your shell config (e.g. ~/.bashrc) that installs a completion handler for dune. The completion script was taken from here, and that page has some information about how the script was generated. Once it's installed the completions will work any time dune is typed at the start of a command, so you can still use the completions when running a version of Dune installed with Opam or your system package manager after installing the Dune Developer Preview.

Currently only command completions are supported. So you can run:

$ dune c<TAB>
cache  clean  coq

...or:

$ dune build -<TAB>
--action-stderr-on-success
--action-stdout-on-success
--always-show-command-line
--auto-promote
--build-dir
--build-info
--cache
...

But if you run dune build <TAB> then it will still suggest local files rather than build targets.

Try it out!

Getting started is easy:

$ curl -fsSL https://get.dune.build/install | sh
$ source ~/.bashrc  # or: source ~/.zshrc
$ dune <TAB>
build
cache
clean
coq
describe
diagnostics
exec
...

Hello folks! 👋

We'd like to welcome everyone to try and play with the Dune Developer Preview! 🎉

This experimental nightly release of dune includes a lot of improvements, including the much expected package management features, and it can be installed from that website or by using the new installation script:

$ curl -fsSL https://get.dune.build/install | bash

In a few seconds you should be ready to OCaml by calling dune:

You can also watch and share this demo on X and Mastodon.

Please try it out and let us know what you think 🙏

📅 You can book a feedback call with us here

📝 You can submit feedback using this form

🐛 You can submit issues to Github on ocaml/dune

Changes since last update

The Dune shared cache has been enabled by default. We're starting off by caching all downloads and dependencies.

We have improved support for dev tools. We're working to streamline this but in the latest binary you can:

• Configure your LSP (in Neovim, Vim, Emacs, etc) to call dune tools exec ocamllsp to get LSP support for your projects out of the box – this may take a little bit the first time it builds the LSP for a compiler version, but it's pretty much instant afterwards.

• Call dune fmt to get your project formatted – remember to add an .ocamlformat file if you don't have one yet. An empty one is enough.

• Call dune ocaml doc to get documentation built

What's next?

We're looking forward to streamlining the DX, working on better dependency locks, and looking into supporting Windows.

In particular, we're considering work on a few things:

• dune create <repo> – to let the community create templates that can be easily used from within dune
• dune pkg fetch – to prefetch packages and prepare a repository for working in offline mode
• dune build @deps - to build all dependencies, useful for staged builds in Dockerfiles
• dune pkg add <name> - to make adding packages straightforward
• a short-hand syntax for pins on github
• and more!

If you've got any ideas, we'd love to hear them, so please open a feature request on Github 🙏

Other updates

FunOCaml Presentation

At FunOCaml we had a last-minute opportunity to present the work being done on Dune and we used it to introduce the Developer Preview to the community, and even tested Package Management live with suggestions from the audience (thanks @anmonteiro and Paul-Elliot for participating!) – you can watch it on Twitch.

New design

We're working with @Claire_Vandenberghe on redesigning the Developer Preview website so that it'd feel like a seamless extension of OCaml.org – in this current iteration we've made it easier to get started and we're putting the FAQ front and center.

We'll be iterating on this design in the coming weeks until it fits perfectly within the OCaml.org design system 🎨

You can check the new website here: https://preview.dune.build

Upcoming Blog posts

In the near future we'll be publishing blog posts about the Developer Preview and Package Management, which we're working on with @professor.rose 👏

As we prepare for the public beta, we're ramping up the DX interviews and ensuring the first few users will have a fun, productive experience with the Developer Preview.

📥 If you signed up for the Dev Preview back in May, check your inbox for a link and instructions to schedule your DX interview with us.

Here's a sample video on (Mastodon or X) where you can see us building the Riot project on a machine that does not have OCaml installed. It is pretty neat!

Seriously, big shoutout to the Dune team at Tarides[0] and Jane Street[1] who have been doing a phenomenal job 👏 ✨ 🐫

So here's what getting started with OCaml looks like using the Dune Developer Preview as of today (August 19 2024):

1. get dune from our binary distribution – we'll soon make this public!
2. run dune pkg lock in your favorite project
3. run dune build

That's it. No need to install anything else! Dune will see that lock file, fetch, and build all necessary dependencies.

🗺️ These are some strong steps towards the OCaml Platform vision for 2026 that we are actively working towards. If you have any thoughts or feedback please let us know!

There are more improvements coming that will help remove friction to get started and create a delightful experience. Both of these things we strongly believe will help onboard new users to the OCaml world.

Here's a few in the works:

• Various DX improvements – from new outputs to simplified workflows, we want to make using Dune just delightful.

• Bundled support for dev tools (OCamlFormat, odoc, LSP) – the default toolset will be available without any extra steps! Just call dune fmt and it works. No need to manually install anything else.

• Automatic dependency locking – when building, and even in watch mode, Dune will lock your dependencies by default and keep the lock up to date.

• Cross-project caching – by default we'll enable a local Dune cache across the system, so you never rebuild the same dependency, even across projects.

• Signed binaries with certificates of origin – we care deeply about security and want to make sure that any binary we ship can be easily verified and tracked back to its sources.

Stay tuned! 👋

PS: here's a longer video on (Mastodon or X) showing you the setup for OCaml from zero, creating a new project, and adding a dependency, all within ~5 minutes

[0] @emillon @Leonidas @gridbugs @tmattio @maiste . Ambre Suhamy, Alpha Diallo [1] @rgrinberg