Most package managers support some pin functionality to ensure that a given
package remains at a particular version without being upgraded.
The stable OPAM 1.1 already supported this by allowing any existing package to be
pinned to a target, which could be a specific released version, a local filesystem
path, or a remote version-controlled repository.
However, the OPAM 1.1 pinning workflow only lets you pin packages that already exist in your OPAM
repositories. To declare a new package, you had to go through creating a
local repository, registering it in OPAM, and adding your package definition there.
That workflow, while reasonably clear, required the user to know about the repository
format and the configuration of an internal repository in OPAM before actually getting to
writing a package. Besides, you were on your own for writing the package
definition, and the edit-test loop wasn't as friendly as it could have been.
A natural, simpler workflow emerged from allowing users to pin new package
names that don't yet exist in an OPAM repository:
- choose a name for your new package
opam pin add in the development source tree- the package is created on-the-fly and registered locally.
To make it even easier, OPAM can now interactively help you write the
package definition, and you can test your updates with a single command.
This blog post explains this new OPAM 1.2 functionality in more detail;
you may also want to check out the new Packaging tutorial
relying on this workflow.
From source to package
For illustration purposes in this post I'll use a tiny tool that I wrote some time ago and
never released: ocp-reloc. It's a simple binary that fixes up the
headers of OCaml bytecode files to make them relocatable, which I'd like
to release into the public OPAM repository.
"opam pin add"
The command opam pin add <name> <target> pins package <name> to
<target>. We're interested in pinning the ocp-reloc package
name to the project's source directory.
cd ocp-reloc
opam pin add ocp-reloc .
If ocp-reloc were an existing package, the metadata would be fetched from
the package description in the OPAM repositories. Since the package doesn't yet exist,
OPAM 1.2 will instead prompt for on-the-fly creation:
Package ocp-reloc does not exist, create as a NEW package ? [Y/n] y
ocp-reloc is now path-pinned to ~/src/ocp-reloc
NOTE: if you are using beta4, you may get a version-control-pin instead,
because we added auto-detection of version-controlled repos. This turned out to
be confusing (issue #1582),
because your changes wouldn't be reflected until you commit, so
this has been reverted in favor of a warning. Add the --kind path option to
make sure that you get a path-pin.
OPAM Package Template
Now your package still needs some kind of definition for OPAM to acknowledge it;
that's where templates kick in, the above triggering an editor with a pre-filled
opam file that you just have to complete. This not only saves time in
looking up the documentation, it also helps getting consistent package
definitions, reduces errors, and promotes filling in optional but recommended
fields (homepage, etc.).
opam-version: "1.2"
name: "ocp-reloc"
version: "0.1"
maintainer: "Louis Gesbert <louis.gesbert@ocamlpro.com>"
authors: "Louis Gesbert <louis.gesbert@ocamlpro.com>"
homepage: ""
bug-reports: ""
license: ""
build: [
["./configure" "--prefix=%{prefix}%"]
[make]
]
install: [make "install"]
remove: ["ocamlfind" "remove" "ocp-reloc"]
depends: "ocamlfind" {build}
After adding some details (most importantly the dependencies and
build instructions), I can just save and exit. Much like other system tools
such as visudo, it checks for syntax errors immediately:
[ERROR] File "/home/lg/.opam/4.01.0/overlay/ocp-reloc/opam", line 13, character 35-36: '.' is not a valid token.
Errors in /home/lg/.opam/4.01.0/overlay/ocp-reloc/opam, retry editing ? [Y/n]
Installation
You probably want to try your brand new package right away, so
OPAM's default action is to try and install it (unless you specified -n):
ocp-reloc needs to be installed.
The following actions will be performed:
- install cmdliner.0.9.5 [required by ocp-reloc]
- install ocp-reloc.0.1*
=== 1 to install ===
Do you want to continue ? [Y/n]
I usually don't get it working the first time around, but opam pin edit ocp-reloc and opam install ocp-reloc -v can be used to edit and retry until
it does.
Package Updates
How do you keep working on your project as you edit the source code, now that
you are installing through OPAM? This is as simple as:
opam upgrade ocp-reloc
This will pick up changes from your source repository and reinstall any packages
that are dependent on ocp-reloc as well, if any.
So far, we've been dealing with the metadata locally used by your OPAM
installation, but you'll probably want to share this among developers of your
project even if you're not releasing anything yet. OPAM takes care of this
by prompting you to save the opam file back to your source tree, where
you can commit it directly into your code repository.
cd ocp-reloc
git add opam
git commit -m 'Add OPAM metadata'
git push
Publishing your New Package
The above information is sufficient to use OPAM locally to integrate new code
into an OPAM installation. Let's look at how other developers can share this
metadata.
Picking up your development package
If another developer wants to pick up ocp-reloc, they can directly use
your existing metadata by cloning a copy of your repository and issuing their
own pin.
git clone git://github.com/OCamlPro/ocp-reloc.git
opam pin add ocp-reloc/
Even specifying the package name is optional since this is documented in
ocp-reloc/opam. They can start hacking, and if needed use opam pin edit to
amend the opam file too. No need for a repository, no need to share anything more than a
versioned opam file within your project.
Cloning already existing packages
We have been focusing on an unreleased package, but the same
functionality is also of great help in handling existing packages, whether you
need to quickly hack into them or are just curious. Let's consider how to
modify the omd Markdown library.
opam source omd --pin
cd omd.0.9.7
...patch...
opam upgrade omd
The new opam source command will clone the source code of the library you
specify, and the --pin option will also pin it locally to ensure it is used
in preference to all other versions. This will also take care of recompiling
any installed packages that are dependent on omd using your patched version
so that you notice any issues right away.
There's a new OPAM field available in 1.2 called dev-repo. If you specify
this in your metadata, you can directly pin to the upstream repository via
opam source --dev-repo --pin.
If the upstream repository for the package contains an opam file, that file will be picked up
in preference to the one from the OPAM repository as soon as you pin the package.
The idea is to have:
- a development
opam file that is versioned along with your source code
(and thus accurately tracks the latest dependencies for your package).
- a release
opam file that is published on the OPAM repository and can
be updated independently without making a new release of the source code.
How to get from the former to the latter will be the subject of another post!
In the meantime, all users of the beta are welcome to share their
experience and thoughts on the new workflow on the bug tracker.