Back to News

Platform Newsletter: May 2023

Welcome to the second issue of the OCaml Platform newsletter!

We're excited to share the work we've done in May on improving OCaml developer experience with the OCaml Platform. Similar to the previous update, this issue is structured around the development workflow we're currently exploring or improving.

The highlight of this month is the publication of the work-in-progress roadmap for the OCaml Platform. We published it to start gathering community feedback on the Design Principles and Persona of the Platform. The feedback will be used to establish our plans for the next three years. We've received tons of very insightful and constructive feedback already, and in the coming weeks and months, we'll revise the roadmap based on that feedback. As a next step, we'll share the first version of the proposed developer workflows.

Another important milestone this month is the release of Dune 3.8. The release comes with support for compiling OCaml projects to JavaScript using Melange, which has seen its first stable release this month! It also contains several important features and improvements that have been in the work for some time, like the new concurrent action and the composition of Coq rules.

As a last highlight, the first alpha of opam 2.2 is getting very close. There were some unexpected issues while preparing the release this month, but the opam team is still aiming for a release in June.

There's a lot of other very exciting work to talk about, so let's delve into it!

Releases

Here are the new versions of Platform tools we released in April. Have a look at the OCaml Changelog to read announcements and feature highlights!

Building Packages

[Dune] Exploring Package Management in Dune

Contributors: @rgrinberg (Tarides), @Leonidas-from-XIV (Tarides), @gridbugs (Tarides), @kit-ty-kate (Tarides)

Explorations continue on adding package management support to Dune. This month progress has been made on several fronts:

  • The work on the solver has been started, including vendoring the opam-0install solver for solving dependencies when generating Dune lockfiles. A work-in-progress implementation of lockfile generation is available on the main branch.
  • The source tree handling has undergone a refactor to eventually allow multiple context-specific lockfiles.
  • The source fetching implementation has seen improvements, including checksum handling and a better Fetch API. This results in a cleaner interface for building opam packages.
  • Work continues on prototyping the building of opam packages, which includes the addition of new Patch and Substitute actions. This has increased the subset of opam packages that can now be built.

Activities:

[opam] Native Support for Windows in opam 2.2

Contributors: @rjbou (OCamlPro), @kit-ty-kate (Tarides), @dra27 (Tarides), @emillon (Tarides), @Leonidas-from-XIV (Tarides)

Throughout May, the opam team has focused on reviewing and fixing remaining PRs for the 2.2 alpha release. This effort debugged and resolved issues found during testing, and the team is now working through the handful PRs pending review.

A new release, opam 2.1.5 point release, has also been rolled out, backporting several fixes from the upcoming 2.2 release, and an important security fix. You can read the announcement on the OCaml Changelog.

Activities:

[Dune] Improving Dune's Documentation

Contributors: @emillon (Tarides)

Two Dune libraries now have a documentation page on OCaml.org: both dune-build-info and dune-configurator now have their API documentation directly on their package page, and for Dune itself, a link to its official documentation has been included.

A new action: directive has been added to the Dune Sphinx domain, allowing for improved cross-referencing. Finally, the old .org format Dune example docs have been converted to Markdown to standardize the documentation format further.

The improvements to the documentation have been published as part of the Dune 3.8 release. The new structure and many improvements can be viewed on Dune documentation.

Activities:

[Dune] Composing installed Coq theories

Contributors: @Alizter and @ejgallego (IRIF)

Last month, the PR that brings support for composing Coq theories with Dune was merged.

This is now available in the release of Dune 3.8.0! From this point onwards, Coq users can utilize Dune to build Coq projects even if they depend on Coq projects that use other build systems!

[Dune] Running Actions Concurrently

Contributors: @Alizter and @hhugo (Nomadic Labs)

Last month, a couple of PRs were merged into Dune to add a new concurrent action and it was utilized to run inline tests concurrently.

These patches are part of Dune 3.8.0 release, starting now, you can use the new concurrent action in your Dune rules:

(rule
 (action
  (concurrent
   (run <prog> <args>)
   (run <prog> <args>))))

[Dune] Benchmarking Dune on Large Code Bases

Contributors: @gridbugs (Tarides), @Leonidas-from-XIV (Tarides)

The quality of the dune benchmark results has been improved, averaging out the variance seen in short tests by running them multiple times. This enhancement aims to reduce the effect of the background noise inherent in the environment.

The dune benchmarks also exposed two broken packages: the hash of ppx_rapper (3.1.0) artefact has been updated as it had changed in place, and ocamlcodoc artefacts have been added to the opam-source-archives as the original URL is no longer reachable.

Moreover, a stack overflow in dune-rpc-lwt exposed by the benchmarks has been fixed.

Activities:

  • Ran short monorepo benchmarks multiple times -- ocaml/dune#7798.
  • Fixed issues in monorepo benchmarks -- ocaml/dune#7786.
  • Added ocamlcodoc to opam-source-archives and updated opam-repository.
  • Opened an issue about ppx_rapper.3.1.0 package changing in place.

Compiling to JavaScript

[Dune] Compile to JavaScript with Melange in Dune

Contributors: @anmonteiro, @jchavarri (Ahrefs), @rgrinberg (Tarides)

We're thrilled to see the joint release of Dune 3.8.0 and Melange 1.0 this month!

Melange is a compiler from OCaml to JavaScript with the vision of maintaining compatibility with OCaml and providing the best OCaml experience within the modern JavaScript ecosystem.

Have a look at the Dune documentation and Melange documentation to learn how to get started using Melange to compile your OCaml projects to JavaScript.

You can also refer to the template to get started.

Generating Documentation

[Odoc] Add Search Capabilities to odoc

Contributors: @panglesd (Tarides), @EmileTrotignon (Tarides), @trefis (Tarides)

The odoc team is making steady progress on adding a search bar to odoc's generated documentation.

In May, an interface for interaction between odoc and search engines was designed. Sherlodoc, which can now run in the browser, was updated to use the new interface exposed by odoc. Improvements were also made to sherlodoc itself to enable searching for constructors and record fields, as well as in docstrings. The larger database from all this extra indexing prompted work on profiling it and implementing optimizations.

Anticipating the June update, the odoc PR that was open a few days ago can be checked out.

In parallel, a working prototype of counting occurrences in odoc was developed. The aim is to add usage statistics in the generated index, so that search engines can use it to sort search results. It will also allow for a "jump-to-documentation" feature in the rendered source code that was merged in March.

[Odoc] Support for Tables in odoc

Contributors: @gpetiot (Tarides), @panglesd (Tarides), @Julow (Tarides), @jonludlam (Tarides), @trefis (Tarides)

The PRs adding support for a new syntax to create tables in Odoc have been merged!

As a reminder, this new feature will enable the creation of tables using a syntax similar to Markdown:

{t
  a  | b | c | d
  ---|:--|--:|:-:
  a  | b | c | d
}

Odoc will generate tables for different backends, including LaTex and HTML. This new syntax will be available in the upcoming release of Odoc 2.3.0. Stay tuned!

Activities:

Editing and Refactoring Code

[Merlin] Support for Project-Wide References in Merlin

Contributors: @voodoos (Tarides), @let-def (Tarides)

Not much progress was made on the support for project-wide occurrences in Merlin this month due to the Merlin team's focus on performance improvements, fixing user-reported bugs, compatibility with OCaml 5.1, and the release of Merlin 4.9

The remaining issues, such as module aliases traversal and index filtering, were discussed, with the implementation of identified solutions set to commence soon.

[Merlin] Improving Merlin's Performance

Contributed by: @pitag-ha (Tarides), @3Rafal (Tarides), @voodoos (Tarides), @let-def (Tarides)

The final stages of work on benchmarking Merlin are in progress, with the Merlin team focusing on integrating the developed benchmarking tooling into Merlin's CI using current-bench. The PR on merl-an has been opened to add a new current-bench compatible backend and work has started on adding memory usage information to Merlin's telemetry.

On the performance optimisations front, the PPX phase cache in Merlin was completed and merged. Issues were opened upstream on how tools that use Merlin can benefit from the new PPX cache.

Activities:

  • Implemented a sketch of current-bench backend in merl-an -- pitag-ha/merl-an#2.
  • Started work on adding memory usage information to Merlin's telemetry.
  • Completed and merged the PPX phase cache PR -- ocaml/merlin#1584.
  • Opened issues at dune and ocaml-lsp-server to keep record of the necessities to enable the PPX phase cache
  • Identified and fixed a memory "leak" related to the (F A).t syntax, causing uncontrolled memoization table growth -- ocaml/merlin#1609.

[OCaml LSP] Using Dune RPC on Windows

Contributors: @nojb (LexiFi)

In May, a couple of patches that build on the support for watch mode on Windows introduced in Dune 3.7.0 were merged in Dune and OCaml LSP to allow OCaml LSP to use Dune RPC. The aim is to enable Windows users to leverage Dune RPC and receive build statuses and more exhaustive build errors in the editor when Dune is running in watch mode.

Dune 3.8.0 was released with the above patches and a release of OCaml LSP will follow in the coming weeks.

[OCaml LSP] Upstreaming OCaml LSP's Fork of Merlin

Contributors: @voodoos (Tarides), @3Rafal (Tarides)

The effort to upstream OCaml LSP's fork of merlin continued. Necessary patches in Merlin have been reviewed and merged. Work also continued on the PR to use Merlin as a library in OCaml LSP.

Activities:

Formatting Code

[OCamlFormat] Closing the Gap Between OCamlFormat and ocp-indent

Contributors: @gpetiot (Tarides) and @EmileTrotignon (Tarides), @Julow (Tarides), @ceastlund (Jane Street)

The effort continued to tune the janestreet profile so that it aligns better with the output of ocp-indent. Despite encountering difficulties with formatting "cinaps" comments, considerable progress was made in May and work will continue throughout June.

Activities: