OCaml Changelog

Minor release.

Initial release.

Minor release.

opam-lib 1.3

The package for opam-lib version 1.3 has just been released in the official opam repository. There is no release of opam with version 1.3, but this is an intermediate version of the library that retains compatibility of the file formats with 1.2.2.

The purpose of this release is twofold:

• provide some fixes and enhancements over opam-lib 1.2.2. For example, 1.3 has an enhanced lint function
• be a step towards migration to opam-lib 2.0.

This version is compatible with the current stable release of opam (1.2.2), but dependencies have been updated so that you are not (e.g.) stuck on an old version of ocamlgraph.

Therefore, I encourage all maintainers of tools based on opam-lib to migrate to 1.3.

The respective APIs are available in HTML for 1.2 and 1.3.

A note on plugins: when you write opam-related tools, remember that by setting flags: plugin in their definition and installing a binary named opam-toolname, you will enable the users to install package toolname and run your tool with a single opam toolname command.

Architectural changes

If you need to migrate from 1.2 to 1.3, these tips may help:

• there are now 6 different ocamlfind sub-libraries instead of just 4: format contains the handlers for opam types and file formats, has been split out from the core library, while state handles the state of a given opam root and switch and has been split from the client library.

• OpamMisc is gone and moved into the better organised OpamStd, with submodules for String, List, etc.

• OpamGlobals is gone too, and its contents have been moved to:

  • OpamConsole for the printing, logging, and shell interface handling part
  • OpamXxxConfig modules for each of the libraries for handling the global configuration variables. You should call the respective init functions, with the options you want to set, for proper initialisation of the lib options (and handling the OPAMXXX environment variables)

• OpamPath.Repository is now OpamRepositoryPath, and part of the repository sub-library.

opam-lib 2.0 ?

The development version of the opam-lib (2.0~alpha5 as of writing) is already available on opam. The name has been changed to provide a finer granularity, so it can actually be installed concurrently -- but be careful not to confuse the ocamlfind package names (opam-lib.format for 1.3 vs opam-format for 2.0).

The provided packages are:

• opam-file-format: now separated from the opam source tree, this has no dependencies and can be used to parse and print the raw opam syntax.
• opam-core: the basic toolbox used by opam, which actually doesn't include the opam specific part. Includes a tiny extra stdlib, the engine for running a graph of processes in parallel, some system handling functions, etc. Depends on ocamlgraph and re only.
• opam-format: defines opam data types and their file i/o functions. Depends just on the two above.
• opam-solver: opam's interface with the dose3 library and external solvers.
• opam-repository: fetching repositories and package sources from all handled remote types.
• opam-state: handling of the opam states, at the global, repository and switch levels.
• opam-client: the client library, providing the top-level operations (installing packages...), and CLI.
• opam-devel: this packages the development version of the opam tool itself, for bootstrapping. You can install it safely as it doesn't install the new opam in the PATH.

The new API can be also be browsed ; please get in touch if you have trouble migrating.

This release mainly brings support for OCaml 4.04. Internal code was simplified and bugs were fixed in the meantime (cache invalidation, ast traversal, type error recovery, certain cases of completion, ppx working directory, locate, ...).

Bug fix release before major version.

This release also contains contributions from: Rudi Grinberg, Fourchaux, Christopher Reichert, David Allsopp, Nick Borden, Mario Rodas, @Twinside, Pierre Chambart, Philipp Haselwarter, Tomasz Kołodziejski and Syohei Yoshida.

OPAM 1.2.2 has just been released. This fixes a few issues over 1.2.1 and brings a couple of improvements, in particular better use of the solver to keep the installation as up-to-date as possible even when the latest version of a package can not be installed.

Upgrade from 1.2.1 (or earlier)

See the normal installation instructions: you should generally pick up the packages from the same origin as you did for the last version -- possibly switching from the official repository packages to the ones we provide for your distribution, in case the former are lagging behind.

There are no changes in repository format, and you can roll back to earlier versions in the 1.2 branch if needed.

Improvements

• Conflict messages now report the original version constraints without translation, and they have been made more concise in some cases
• Some new opam lint checks, opam lint now numbers its warnings and may provide script-friendly output
• Feature to automatically install plugins, e.g. opam depext will prompt to install depext if available and not already installed
• Priority to newer versions even when the latest can't be installed (with a recent solver only. Before, all non-latest versions were equivalent to the solver)
• Added opam list --resolve to list a consistent installation scenario
• Be cool by default on errors in OPAM files, these don't concern end-users and packagers and CI now have opam lint to check them.

Fixes

• OSX: state cache got broken in 1.2.1, which could induce longer startup times. This is now fixed
• opam config report has been fixed to report the external solver properly
• --dry-run --verbose properly outputs all commands that would be run again
• Providing a simple path to an aspcud executable as external solver (through options or environment) works again, for backwards-compatibility
• Fixed a fd leak on solver calls (thanks Ivan Gotovchits)
• opam list now returns 0 when no packages match but no pattern was supplied, which is more helpful in scripts relying on it to check dependencies.

OPAM 1.2.1 has just been released. This patch version brings a number of fixes and improvements over 1.2.0, without breaking compatibility.

Upgrade from 1.2.0 (or earlier)

See the normal installation instructions: you should generally pick up the packages from the same origin as you did for the last version -- possibly switching from the official repository packages to the ones we provide for your distribution, in case the former are lagging behind.

What's new

No huge new features in this point release -- which means you can roll back to 1.2.0 in case of problems -- but lots going on under the hood, and quite a few visible changes nonetheless:

• The engine that processes package builds and other commands in parallel has been rewritten. You'll notice the cool new display but it's also much more reliable and efficient. Make sure to set jobs: to a value greater than 1 in ~/.opam/config in case you updated from an older version.
• The install/upgrade/downgrade/remove/reinstall actions are also processed in a better way: the consequences of a failed actions are minimised, when it used to abort the full command.
• When using version control to pin a package to a local directory without specifying a branch, only the tracked files are used by OPAM, but their changes don't need to be checked in. This was found to be the most convenient compromise.
• Sources used for several OPAM packages may use <name>.opam files for package pinning. URLs of the form git+ssh:// or hg+https:// are now allowed.
• opam lint has been vastly improved.

... and much more

There is also a new manual documenting the file and repository formats.

Fixes

See the changelog for a summary or closed issues in the bug-tracker for an overview.

Experimental features

These are mostly improvements to the file formats. You are welcome to use them, but they won't be accepted into the official repository until the next release.

• New field features: in opam files, to help with ./configure scripts and documenting the specific features enabled in a given build. See the original proposal and the section in the new manual
• The "filter" language in opam files is now well defined, and documented in the manual. In particular, undefined variables are consistently handled, as well as conversions between string and boolean values, with new syntax for converting bools to strings.
• New package flag "verbose" in opam files, that outputs the package's build script to stdout
• New field libexec: in <name>.install files, to install into the package's lib dir with the execution bit set.
• Compilers can now be defined without source nor build instructions, and the base packages defined in the packages: field are now resolved and then locked. In practice, this means that repository maintainers can move the compiler itself to a package, giving a lot more flexibility.

Main new feature is a faster short-path, and also a lot of buxfixes.

This release also contains contributions from: Geoff Gole, Rudi Grinberg, Markus Mottl, Roman Vorobets and Arthur Wendling.

We are very proud to announce the availability of OPAM 1.2.0.

Upgrade from 1.1

Simply follow the usual instructions, using your preferred method (package from your distribution, binary, source, etc.) as documented on the homepage.

NOTE: There are small changes to the internal repository format (~/.opam). It will be transparently updated on first run, but in case you might want to go back and have anything precious there, you're advised to back it up.

Usability

Lot of work has been put into providing a cleaner interface, with helpful behaviour and messages in case of errors.

The documentation pages also have been largely rewritten for consistency and clarity.

New features

This is just the top of the list:

• A extended and versatile opam pin command. See the Simplified packaging workflow
• More expressive queries, see for example opam source
• New metadata fields, including source repositories, bug-trackers, and finer control of package behaviour
• An opam lint command to check the quality of packages

For more detail, see the announcement for the beta, the full changelog, and the bug-tracker.

Package format

The package format has been extended to the benefit of both packagers and users. The repository already accepts packages in the 1.2 format, and this won't affect 1.1 users as a rewrite is done on the server for compatibility with 1.1.

If you are hosting a repository, you may be interested in these administration scripts to quickly take advantage of the new features or retain compatibility.

After a few months of development, we are pleased to announce the stable release of Merlin 2.0. Supported OCaml versions range from 4.00.1 to 4.02.1.

Overview

Merlin is a tool focused on helping you code in OCaml by providing features such as:

• automatic completion of identifiers, using scope and type information,
• interactively typing definitions and expressions during edition,
• jumping to the definition of any identifier,
• quickly reporting errors in the editor.

We provide integration into Vim and Emacs. An external plugin is also available for Sublime Text.

What's new

This release provides great improvements in robustness and quality of analysis. Files that changed on disk are now automatically reloaded. The parsing process is finer grained to provide more accurate recovery and error messages. Integration with Jane Street Core and js_of_ocaml has also improved.

Vim & Emacs are still the main targeted editors. Thanks to Luc Rocher, preliminary support for Sublime Text is also available, see Sublime-text-merlin. Help is welcome to improve and extend supported editing environments.

Windows support also received some fixes. Merlin is now distributed in WODI. Integration in OCaml-on-windows is planned.

Installation

This new version of Merlin is already available with opam using opam install merlin, and can also be built from the sources which are available at the-lambda-church/merlin.

Minor update to installation procedure

Most package managers support some pin functionality to ensure that a given package remains at a particular version without being upgraded. The stable OPAM 1.1 already supported this by allowing any existing package to be pinned to a target, which could be a specific released version, a local filesystem path, or a remote version-controlled repository.

However, the OPAM 1.1 pinning workflow only lets you pin packages that already exist in your OPAM repositories. To declare a new package, you had to go through creating a local repository, registering it in OPAM, and adding your package definition there. That workflow, while reasonably clear, required the user to know about the repository format and the configuration of an internal repository in OPAM before actually getting to writing a package. Besides, you were on your own for writing the package definition, and the edit-test loop wasn't as friendly as it could have been.

A natural, simpler workflow emerged from allowing users to pin new package names that don't yet exist in an OPAM repository:

1. choose a name for your new package
2. opam pin add in the development source tree
3. the package is created on-the-fly and registered locally.

To make it even easier, OPAM can now interactively help you write the package definition, and you can test your updates with a single command. This blog post explains this new OPAM 1.2 functionality in more detail; you may also want to check out the new Packaging tutorial relying on this workflow.

From source to package

For illustration purposes in this post I'll use a tiny tool that I wrote some time ago and never released: ocp-reloc. It's a simple binary that fixes up the headers of OCaml bytecode files to make them relocatable, which I'd like to release into the public OPAM repository.

"opam pin add"

The command opam pin add <name> <target> pins package <name> to <target>. We're interested in pinning the ocp-reloc package name to the project's source directory.

cd ocp-reloc
opam pin add ocp-reloc .

If ocp-reloc were an existing package, the metadata would be fetched from the package description in the OPAM repositories. Since the package doesn't yet exist, OPAM 1.2 will instead prompt for on-the-fly creation:

Package ocp-reloc does not exist, create as a NEW package ? [Y/n] y
ocp-reloc is now path-pinned to ~/src/ocp-reloc

NOTE: if you are using beta4, you may get a version-control-pin instead, because we added auto-detection of version-controlled repos. This turned out to be confusing (issue #1582), because your changes wouldn't be reflected until you commit, so this has been reverted in favor of a warning. Add the --kind path option to make sure that you get a path-pin.

OPAM Package Template

Now your package still needs some kind of definition for OPAM to acknowledge it; that's where templates kick in, the above triggering an editor with a pre-filled opam file that you just have to complete. This not only saves time in looking up the documentation, it also helps getting consistent package definitions, reduces errors, and promotes filling in optional but recommended fields (homepage, etc.).

opam-version: "1.2"
name: "ocp-reloc"
version: "0.1"
maintainer: "Louis Gesbert <louis.gesbert@ocamlpro.com>"
authors: "Louis Gesbert <louis.gesbert@ocamlpro.com>"
homepage: ""
bug-reports: ""
license: ""
build: [
  ["./configure" "--prefix=%{prefix}%"]
  [make]
]
install: [make "install"]
remove: ["ocamlfind" "remove" "ocp-reloc"]
depends: "ocamlfind" {build}

After adding some details (most importantly the dependencies and build instructions), I can just save and exit. Much like other system tools such as visudo, it checks for syntax errors immediately:

[ERROR] File "/home/lg/.opam/4.01.0/overlay/ocp-reloc/opam", line 13, character 35-36: '.' is not a valid token.
Errors in /home/lg/.opam/4.01.0/overlay/ocp-reloc/opam, retry editing ? [Y/n]

Installation

You probably want to try your brand new package right away, so OPAM's default action is to try and install it (unless you specified -n):

ocp-reloc needs to be installed.
The following actions will be performed:
 - install   cmdliner.0.9.5                        [required by ocp-reloc]
 - install   ocp-reloc.0.1*
=== 1 to install ===
Do you want to continue ? [Y/n]

I usually don't get it working the first time around, but opam pin edit ocp-reloc and opam install ocp-reloc -v can be used to edit and retry until it does.

Package Updates

How do you keep working on your project as you edit the source code, now that you are installing through OPAM? This is as simple as:

opam upgrade ocp-reloc

This will pick up changes from your source repository and reinstall any packages that are dependent on ocp-reloc as well, if any.

So far, we've been dealing with the metadata locally used by your OPAM installation, but you'll probably want to share this among developers of your project even if you're not releasing anything yet. OPAM takes care of this by prompting you to save the opam file back to your source tree, where you can commit it directly into your code repository.

cd ocp-reloc
git add opam
git commit -m 'Add OPAM metadata'
git push

Publishing your New Package

The above information is sufficient to use OPAM locally to integrate new code into an OPAM installation. Let's look at how other developers can share this metadata.

Picking up your development package

If another developer wants to pick up ocp-reloc, they can directly use your existing metadata by cloning a copy of your repository and issuing their own pin.

git clone git://github.com/OCamlPro/ocp-reloc.git
opam pin add ocp-reloc/

Even specifying the package name is optional since this is documented in ocp-reloc/opam. They can start hacking, and if needed use opam pin edit to amend the opam file too. No need for a repository, no need to share anything more than a versioned opam file within your project.

Cloning already existing packages

We have been focusing on an unreleased package, but the same functionality is also of great help in handling existing packages, whether you need to quickly hack into them or are just curious. Let's consider how to modify the omd Markdown library.

opam source omd --pin
cd omd.0.9.7
...patch...
opam upgrade omd

The new opam source command will clone the source code of the library you specify, and the --pin option will also pin it locally to ensure it is used in preference to all other versions. This will also take care of recompiling any installed packages that are dependent on omd using your patched version so that you notice any issues right away.

There's a new OPAM field available in 1.2 called dev-repo. If you specify this in your metadata, you can directly pin to the upstream repository via opam source --dev-repo --pin.

If the upstream repository for the package contains an opam file, that file will be picked up in preference to the one from the OPAM repository as soon as you pin the package. The idea is to have:

• a development opam file that is versioned along with your source code (and thus accurately tracks the latest dependencies for your package).
• a release opam file that is published on the OPAM repository and can be updated independently without making a new release of the source code.

How to get from the former to the latter will be the subject of another post! In the meantime, all users of the beta are welcome to share their experience and thoughts on the new workflow on the bug tracker.