The b0 opam manual

B0_opam has support for opam, the OCaml package manager. It provides:

• A convention to represent opam packages in the metadata of b0 packs.
• The b0 -- .opam action to generate opam package files and publish packages on the public OCaml opam repository.

Package representation

opam packages are represented by build packs tagged with the B0_opam.tag. The name of the defined opam package is the basename of the pack or the value of the B0_opam.name meta key; see B0_opam.pkg_name_of_pack for the full details.

The final metadata of the opam package is defined by standard and opam specific meta key values. The map between metadata key and opam fields is described in B0_opam.File.pkg_of_meta. Some keys are derived by B0_opam.pkg_meta_of_pack if unspecified.

The pack should contain exactly the build units that must be built by the opam package, this allows B0_opam.pkg_meta_of_pack to automatically infer some of the fields (e.g. build: or depends:).

Here's an example:

let mypkg =
  let meta =
    B0_meta.empty
    |> ~~ B0_meta.authors ["The mylib programmers"]
    |> ~~ B0_meta.maintainers ["mylib@example.org"]
    |> ~~ B0_meta.homepage "https://example.org/software/mylib"
    |> ~~ B0_meta.online_doc "https://example.org/software/mylib/doc"
    |> ~~ B0_meta.license ["MIT"]
    |> ~~ B0_meta.repo "git+https://example.org/repos/mylib.git"
    |> ~~ B0_meta.issues "https://github.com/mylib/mylib/issues"
    |> B0_meta.tag B0_opam.tag
    |> B0_meta.add B0_opam.Meta.depends [ "ocaml", {|>= "4.14.0" |} ]
  in
  B0_pack.v "mypkg" ~doc:"opam package mypkg" ~meta ~locked:true @@
  B0_unit.list () (* You likely want to be more refined than that *)

The opam packages defined in a b0 root along with the pack in which they are defined can be listed with the .opam list command:

b0 -- .opam list           # List opam packages in the b0 root
b0 -- .opam list --help    # See more options

Package file generation

The opam file for a package is generated from its pack metadata via B0_opam.pkg_meta_of_pack and B0_opam.File.pkg_of_meta.

The .opam file command outputs the generated files in various ways.

b0 -- .opam file mypkg             # opam file for mypkg on stdout
b0 -- .opam file                   # All packages, on stdout
b0 -- .opam file -d /tmp/opam/     # Write them to /tmp/opam
b0 -- .opam file --in-scope-dir    # Write them in their scopes
b0 -- .opam file --help            # See more options

Note that the generation differs slightly when opam files are written to stdout, the name: field is added; use option --no-name to prevent that.

Package publication

The .opam publish command publishes packages on opam by making a pull request on the OCaml opam package repository on GitHub. In order to do so you must have the GitHub setup in place.

Basics

opam package publication happens after the software source archive has been released on the web. To get help for releasing your software with b0 see the b0 release manual.

The opam publication process depends on the current work tree of the VCSes that are in your b0 root as follows:

• The current metadata of the packs defining opam packages is used to generate the opam package files to publish. See Package file generation.
• The last reachable annotated tags of the VCSes in charge of the packs defining opam packages define the published versions; unless they are explictely specified on the command line. See Version determination.
• The first section of the current changelog files are used to derive release notes added to the pull request. See Release notes determination

Multiple package releases and incompatibilites with existing packages can be stated via a single pull request. For example:

b0 -- .opam publish p1 p2 p3.2.0.1 -i otherpackage

has this meaning:

• Add the opam packages p1 and p2 at a version determined by the VCSes in charge of the packs that define them.
• Add the opam package p3 at version 2.0.1
• Update constraints of all versions of otherpackage to state that they are incompatible with these new versions – for those that exist in its dependencies.

Effectively it does the following. For each opam package p given on the command line it:

1. Determines the package version.
2. Determines the URL of the package's release archive to download.
3. Downloads the archive and checksums it via the shasum tool.
4. Generates a versioned opam package file along with a checksumed url: field to the archive.
5. Determines release notes for the package; for adding information on the pull request.

If everything worked well it continues with:

1. Clones or updates the OCaml opam repository to the bare repository ~/.caches/opam-repository.git (the XDG spec is honoured).
2. Creates a branch for the publication, adds a commit with the generated files and constraints on incompatible packages.
3. Pushes the branch on your fork of the OCaml opam repository on GitHub.
4. Opens a pull request for the branch on the OCaml opam repository on GitHub.

Testing

Invoking .opam publish with --dry-run outputs on stdout the various bits that are being derived and performs various checks. For example it lints opam files and HTTP HEADs archives to check that they are not 404. It stops the process before updating the fork of the git opam repository and creating the pull request. Invoking with --no-pr the fork is updated but the PR is not created.

Handling build failures

Once the opam repository continuous integration has reported the disastrous effects of your publication on dependent packages, you may need to state constraints on other packages. There are a few different ways to do this.

On the publication PR

Package incompatibilites can be stated with the repeatable -i PKG[.version] option of .opam publish. This adds a proper upper bound to PKG as needed.

You can do this at publication time or afterwards. If you repeat the same .opam publish invocation (which can be found in the message of the PR) with more -i options, this simply updates the existing pull request.

For example the following publishes a new mypkg and states that all versions of otherpack and the version 2.1.0 of thispack are incompatible with it – provided mypkg is in their dependencies:

b0 -- .opam publish mypkg -i otherpack -i thispack.2.1.0

It is also possible to directly tweak the publication PR made by .opam publish by cloning your github opam-repository and checking out the branch of publication and push changes on it.

git clone git@github.com:$(GHUSER)/opam-repository.git
git checkout $B0-PUBLISH-BRANCH
… # Edit
git commit -m 'Handle first round of failures'
git push # This will automatically update the PR.

With a separate PR merged before

If for some reason using -i is not suitable you can manually make a separate request to state the incompatibilites and have it merged by the opam repository maintainers before the publication pull request. Once this is done you can re-run your .opam invocation (which can be found in the message of the PR), it will rebase the pull request.

For long lists of packages this can be more convenient. For example suppose you have a text file failures.txt with one PACKAGE.VERSION per line followed by an error message, opam admin is your friend:

   cd opam-repository   # Your opam-repository checkout
   git fetch upstream
   git checkout -b p.X.Y.Z-failures upstream/master
   opam admin add-constraints 'p<X.Z.Z' \
        --packages $(cat failures.txt | cut -d' ' -f 1  | tr '\n' ',')
   git commit -m 'Constrain p.X.Y.Z failures'
   git push origin  # Go make a pull request with it

Handling metadata issues

If opam repository maintainers complain about your opam files you can adjust them and invoke again the same .opam publish command (which can be found in the message of the PR). As long as the same set of opam packages with the same version end-up being published this updates the existing pull request.

Starting over

If for some reason you find it preferable to start over the whole process. Close the PR manually. The next time you invoke .opam publish a new pull request will be opened.

Package publication details

GitHub setup

To be able to make a pull request on the OCaml opam package repository in the name of your GitHub $USER you need to:

1. Create a personal access token for $USER with repo permission by following these instructions. Paste the token in the file ~/.config/b0/github/$USER.token and:

    chmod 600 ~/.config/b0/github/$USER.token

   Alternatively you can specify the user and the token in the B0_GITHUB_{USER,TOKEN} environment variables.

2. Fork the OCaml opam repository so that it becomes available from https://github.com/$USER/opam-repository.

Version determination

The version number of a package can be specified explicitely on the command line by separating it with a dot from the opam package name. For example mypkg.2.0.1 publishes version 2.0.1 of the package. If no version number is explicitely given it is defined by using B0_release.vcs_repo_version_of_pack on its pack.

Release archive URL determination

The URL to the source archive of the opam package is determined by its pack and version via B0_release.src_archive_url_of_pack.

Note that the archive name defaults to the pack name when unspecified. If you want to define multiple packages for the same source archive you will need to define the B0_release.src_archive_name appropriately.

For example suppose that in addition to mypkg you publish on opam a separate package mypkg-unix which compiles the Unix library support from the same source achive as mypkg. In the metadata of the pack defining the package mypkg-unix you can add:

let mypkg = ...
let mypkg_unix =
  let meta =
    B0_meta.empty
    ...
    |> ~~ B0_release.src_archive_name (B0_pack.basename mypkg)
  in
  B0_pack.v "mypkg-unix" ~meta @@ mypkg_unix_units

Release notes determination

The release notes of the opam package are determined by looking for a file CHANGES.md in the scope directory of its pack and picking up the first section. See B0_release.changes_file_of_pack and B0_release.changes_latest_of_file.

Packages sharing the same release notes are aggregated in the pull request.

Tips

Multiple packages from the same source archive

If you want to define multiple packages from a single archive tarball follow the instructions in Release archive URL determination.

.opam publish hangs after push

If publish hangs after push: go to cd ~/.cache/opam-repository && git gc and try again.