package odoc

  1. Overview
  2. Docs
Legend:
Library
Module
Module type
Parameter
Class
Class type

How to Drive odoc

odoc is a CLI tool to create API and documentation for OCaml projects. However, it operates at a rather low level, taking individual files through several distinct phases until the HTML output is generated.

For this reason, just like for building any multifiles OCaml project, odoc needs to be driven by a higher level tool. The driver will take care of calling the odoc command with the right arguments throughout the different phases. The odoc-driver package contains a "reference driver", that is kept up-to-date with the latest development of odoc,

Several drivers for odoc exist, such as: dune and odig.

This document explains how to drive odoc, as of version 3. It is not needed to know any of this to use odoc, it is targeted at driver authors, tools that interact with odoc, or any curious passerby. This includes several subjects:

  • A big picture view of the doc generation model,
  • A unified explanation for the various command line flags,
  • An explanation of the odoc pipeline,
  • A convention for building docs for opam-installed packages.

In addition to the documentation, the reference driver is a good tool to understand how to build odoc projects. It can be useful to look at the implementation code, but it can also help to simply look at all invocations of odoc during a run of the driver.

Trees of documentation

In its third major version, odoc has been improved so that the same documentation can work on multiple scenarios, from local switches to big monorepos, or the ocaml.org hub of documentation for all packages, without anything breaking, especially references.

The idea is that we have named groups of documentation, that we'll call trees here. We have two kinds of trees: page trees, and modules trees. Inside the trees, the hierarchy is managed by odoc. The driver is free to "root" them however they like in the overall hierarchy. So odoc is responsible for the hierarchy below the trees root, and the driver is responsible for the one outside of the trees. In order to reference another tree, a documentation author can use the name of the tree in the reference.

Different situations will give different meanings to the trees. In the case of opam packages, though, there is a natural meaning to give to those trees (you'll find more details in the convention for opam-installed packages). Any opam package will have an associated "documentation tree", named with the name of the package. Any of its libraries will have an associated "module tree", named with the name of the library. Another package can thus refer to the doc using the package name, or to any of its library using the library name, no matter where the package is located in the hierarchy.

The doc generation pipeline

Just like when compiling OCaml modules, generating docs for these modules need to be run in a specific order, as some information for generating docs for a file might reside in another one. However, odoc actually allows a particular file to reference a module that depends on it, seemingly creating a circular dependency.

This circular dependency problem is one of the reasons we have several phases in odoc. Let's review them:

  • The compile phase, which is used to create the .odoc artifacts from .cm{i;t;ti} and .mld files. This is where odoc does similar work to that of the OCaml compiler, computing expansions for each module types. The dependencies between are the same as the ones for the .cm{i;t;ti} input.
  • The link phase transforms the .odoc artifacts to .odocl. The main result of this phase is to resolve odoc references, which also has an effect on canonical modules.
  • The indexing phase generates .odoc-index files from sets of .odocl files. These index files will be used both for generating a global sidebar, and for generating a search index.
  • The generation phase takes the .odocl and .odoc-index files and turns them into either HTML, man pages or Latex files.

The compile phase

The compile phase takes as input a set of .cm{i;t;ti} as well as .mld files, and builds a directory hierarchy of .odoc files.

There are distinct commands for this phase: odoc compile for interfaces and pages, odoc compile-impl for implementations, and odoc compile-asset for assets.

Compiling interfaces

Let's have a look at a generic invocation of odoc during the compile phase:

$ odoc compile --output-dir <od> --parent-id <pid> -I <dir1> -I <dir2> <input-file>.<ext>
  • <input-file>.<ext> is the input file, either a .cm{i;t;ti} file or an .mld file. Prefer .cmti files over the other formats!
  • --output-dir <od> allows to specify the directory that will contain all the .odoc files. This directory has to be fully managed by odoc and should not be modified by another tool! The output file depends on the --parent-id option.
  • --parent-id <pid> allows to place the output .odoc file in the documentation hierarchy. This consists in a / separated sequence of non empty strings (used as directory name). This "path" determines where the .odoc file will be located below the <od> output dir. The name of the output file is <input-file>.odoc for modules, and page-<input-file>.odoc for pages. Documentation artifacts that will be in the same unit of documentation need to hare a common root in their parent id.
  • -I <dir> corresponds to the search path for other .odoc files. Multiple directory can be added to the search path, so every required .odoc file is in the search path. The required .odoc files are the one generated from a .cm{i;t;ti} file listed when calling odoc compile-deps on the input file.

A concrete example for such command would be:

$ odoc compile
     ~/.opam/5.2.0/lib/ppxlib/ppxlib__Extension.cmti
     --output-dir _odoc/
     -I _odoc/ocaml-base-compiler/compiler-libs.common
     -I _odoc/ocaml-base-compiler/stdlib
     -I _odoc/ocaml-compiler-libs/ocaml-compiler-libs.common
     -I _odoc/ppxlib/ppxlib
     -I _odoc/ppxlib/ppxlib.ast
     -I _odoc/ppxlib/ppxlib.astlib
     -I _odoc/ppxlib/ppxlib.stdppx
     -I _odoc/ppxlib/ppxlib.traverse_builtins
     -I _odoc/sexplib0/sexplib0
     --parent-id ppxlib/ppxlib

Compiling implementations

A compile-impl command is pretty similar:

$ odoc compile-impl --output-dir <od> --source-id <sid> --parent-id <pid> -I <dir1> -I <dir2> <input-file>.<ext>
  • <input-file>.cmt is the input file, it has to be a .cmt file.
  • --output-dir <od> has the same meaning as for odoc compile.
  • --parent-id <pid> also has the same meaning as for odoc compile. However, the name of the output file is impl-<input-file>.odoc. Implementations need to be available through the -I search path, so it is very likely that one wants the implementation and interface .odoc files to share the same parent id.
  • -I <dir> also corresponds to the search path for other .odoc files.
  • source-id <sid> is a new argument specific to compile-impl. This corresponds to the location of the rendering of the source, which is required to generate links to it.

A concrete example for such command would be:

$ odoc compile-impl
      ~/.opam/5.2.0/lib/ppxlib/ppxlib__Spellcheck.cmt
      --output-dir _odoc/
      -I _odoc/ocaml-base-compiler/compiler-libs.common
      -I _odoc/ocaml-base-compiler/stdlib
      -I _odoc/ocaml-compiler-libs/ocaml-compiler-libs.common
      -I _odoc/ppxlib/ppxlib
      -I _odoc/ppxlib/ppxlib.ast
      -I _odoc/ppxlib/ppxlib.astlib
      -I _odoc/ppxlib/ppxlib.stdppx
      -I _odoc/sexplib0/sexplib0
      --enable-missing-root-warning
      --parent-id ppxlib/ppxlib
      --source-id ppxlib/src/ppxlib/spellcheck.ml

Compiling assets

Assets are given during the generation phase. But we still need to create an .odoc file, for odoc's resolution mechanism.

$ odoc compile-asset --output-dir <od> --parent-id <pid> --name <assetname>
  • --output-dir and --parent-id are identical to the compile and compile-impl commands,
  • --name <assetname> gives the asset name.
  • The output file name is computed from the previous values as <output-dir>/<parent-id>/asset-<assetname>.odoc.

The link phase requires the directory of the compile phase to generate its set of .odocl files. This phase resolves references and canonicals.

A generic link command is:

$ odoc link
    -I <dir1> -I <dir2>
    -P <pname1>:<pdir1> -P <pname2>:<pdir2>
    -L <lname1>:<ldir1> -L <lname2>:<ldir2>
    <path/to/file.odoc>
  • <path/to/file.odoc is the input .odoc file. The result of this command is path/to/file.odocl. This path was determined by --output-dir and --parent-id from the link phase, and it is important for the indexing phase that it stays in the same location.
  • -P <name>:<dir> are used to list the "page trees", used to resolve references such as {!/ocamlfind/index}.
  • -L <name>:<dir> are used to list the "module trees", used to resolve references such as {!/findlib.dynload/Fl_dynload}. This also adds <dir> to the search path.
  • -I <dir> adds <dir> to the search path. The search path is used to resolve references that do not use the "named tree" mechanism, such as {!Module} and {!page-pagename}.

The indexing phase

The indexing phase refers to the "crunching" of information split in several .odocl files. Currently, there are two use-cases for this phase:

  • Generating a search index. This requires all information from linked interfaces and pages, but also form linked implementations in order to sort results (by number of occurrences).
  • Generating a global sidebar.

Counting occurrences

This step counts the number of occurrences of each value/type/... in the implementation, and stores them in a table. A generic invocation is:

$ odoc count-occurrences <dir1> <dir2> -o <path/to/name.odoc-occurrences>

An example of such command:

$ odoc count-occurrences _odoc/ -o _odoc/occurrences-all.odoc-occurrences

Indexing entries

The odoc compile-index produces an .odoc-index file, from .odocl files, other .odoc-index files, and possibly some .odoc-occurrences files.

To create an index for the page and documentation units, we use the -P and -L arguments.

$ odoc compile-index
    -o path/to/<indexname>.odoc-index
    -P <pname1>:<ppath1>
    -P <pname2>:<ppath2>
    -L <lname1>:<lpath1>
    -L <lname2>:<lpath2>
    --occurrences <path/to/name.odoc-occurrences>

An example of such command:

$ odoc compile-index
    -o _odoc/ppxlib/index.odoc-index
    -P ppxlib:_odoc/ppxlib
    -L ppxlib:_odoc/ppxlib/ppxlib
    -L ppxlib.ast:_odoc/ppxlib/ppxlib.ast
    -L ppxlib.astlib:_odoc/ppxlib/ppxlib.astlib
    -L ppxlib.metaquot:_odoc/ppxlib/ppxlib.metaquot
    -L ppxlib.metaquot_lifters:_odoc/ppxlib/ppxlib.metaquot_lifters
    -L ppxlib.print_diff:_odoc/ppxlib/ppxlib.print_diff
    -L ppxlib.runner:_odoc/ppxlib/ppxlib.runner
    -L ppxlib.runner_as_ppx:_odoc/ppxlib/ppxlib.runner_as_ppx
    -L ppxlib.stdppx:_odoc/ppxlib/ppxlib.stdppx
    -L ppxlib.traverse:_odoc/ppxlib/ppxlib.traverse
    -L ppxlib.traverse_builtins:_odoc/ppxlib/ppxlib.traverse_builtins
    --occurrences _odoc/occurrences-all.odoc-occurrences

The generation phase

The generation phase is the phase that takes all information computed in previous files, and actually generates the documentation. It can take the form of HTML, Latex and manpages, although currently HTML is the odoc backend that supports the most functionalities (such as images, videos, ...).

In this manual, we describe the HTML generation usecase. Usually, generating for other backend boils down to replacing html-generate by latex-generate or man-generate, refer to the manpage to see the diverging options.

Given an .odocl file, odoc might generate a single .html file, or a complete directory of .html files. The --output-dir option specifies the root for generating those outputs.

A JavaScript file for search requests

odoc provides a way to plugin a JavaScript file, containing the code to answer user's queries. In order to never block the UI, this file will be loaded in a web worker to perform searches:

  • The search query will be sent as a plain string to the web worker, using the standard mechanism of message passing.
  • The web worker has to send back the result as a message to the main thread, containing the results. The format for the result message is a string that can be parsed as a list of JSON objects. Each object contain two keys-value pairs: a key "url" which contains a relative URL from --ouput-dir; and a key "html" which contain the html showing the search entry.
  • The JavaScript file must be manually put in the --output-dir values, the driver can decide where.

Interfaces and pages

A generic html-generate command for interfaces has the following form:

$ odoc html-generate
  --output-dir <odir>
  --index <path/to/file.odoc-index>
  --search-uri <relative/to/output-dir/file.js>
  --search-uri <relative/to/output-dir/file2.js>
  <path/to/file.odocl>
  • --output-dir <odir> is used to specify the root output for the generated .html.
  • --index <path/to/file.odoc-index> is given to odoc for sidebar generation.
  • --search-uri <relative/to/output-dir/file.js> tells odoc which file(s) to load in a web worker.

The output directory or file can be computed from this command's --output-dir, the initial --parent-id given when creating the .odoc file, as well as the unit name. In the case of a module, the output is a directory named with the name of the module. In the case of a page, the output is a file with the name of the page and the .html extension.

An example of such command is:

$ odoc html-generate
    _odoc/ppxlib/page-index.odocl
    --index _odoc/ppxlib/index.odoc-index
    --search-uri ppxlib/sherlodoc_db.js
    --search-uri sherlodoc.js
    -o _html/

Source code

$ odoc html-generate-source --output-dir <odir> --impl <path/to/impl-file.odocl> <path/to/source/file.ml>
  • --output-dir <odir> has been covered already
  • --impl <path/to/impl-file.odocl> allows to give the implementation file.
  • <path/to/source/file.ml> is the source file.

The output file can be computed from this command's --output-dir, and the initial --source-id and --name given when creating the impl-*.odoc file.

An example of such command is:

$ odoc html-generate-source
    --impl _odoc/ppxlib/ppxlib/impl-ppxlib__Reconcile.odocl
    /home/panglesd/.opam/5.2.0/lib/ppxlib/reconcile.ml
    -o _html/

Generating docs for assets

This is the phase where we pass the actual asset. We pass it as a positional argument, and give the asset unit using --asset-unit.

$ odoc html-generate-asset --output-dir <odir> --asset-unit <path/to/asset-file.odocl> <path/to/asset/file.ext>

Convention for installed packages

In order to build the documentation for installed packages, the driver needs to give a meaning to the various concepts above. In particular, it needs to define the pages and libraries trees, know where to find the pages and assets, what id to give them, when linking it needs to know to which trees the artifact may be linking...

So that the different drivers and installed packages play well together, we define here a convention for building installed packages. If both the package and the driver follow it, building the docs should go well!

The -P and -L trees, and their root ids

Each package defines a set of trees, each of them having a root id. These roots will be used in --parent-id and in -P and -L.

The driver can decide any set of mutually disjoint set of roots, without posing problem to the reference resolution. For instance, both -P pkg:<output_dir>/pkg and -P pkg:<output_dir>/pkg/version are acceptable versions. However, we define here "canonical" roots:

Each installed package <p> defines a single page root id: <p>.

For each package <p>, each library <l> defines a library root id: <p>/<l>.

For instance, a package foo with two libraries: foo and foo.bar will define three trees:

  • A documentation tree named foo, with root id foo. When referred from other trees, a -P foo:<odoc_dir>/foo argument needs to be added at the link phase.
  • A module tree named foo, with root id foo/foo. When referred from other trees, a -L foo:<odoc_dir>/foo/foo argument needs to be added at the link phase.
  • A module tree named foo.bar, with root id foo/foo.bar. When referred from other trees, a -L foo.bar:<odoc_dir>/foo/foo.bar argument needs to be added at the link phase.

Installed OPAM packages need to specify which trees they may be referencing during the link phase, so that the proper -P and -L arguments are added. (Note that these dependencies can be circular, as they happen during the link phase and only require the artifact from the compile phase.)

An installed package <p> specifies its tree dependencies in a file at <opam root>/doc/<p>/odoc-config.sexp. This file contains s-expressions.

Stanzas of the form (packages p1 p2 ...) specifies that page trees p1, p2, ..., should be added using the -P argument: with the canonical roots, it would be -P p1:<output_dir>/p1 -P p2:<output_dir>/p2 -P ....

Stanzas of the form (libraries l1 l2 ...) specifies that module trees l1, l2, ..., should be added using the -L argument: with the canonical roots, it would be -L l1:<output_dir>/p1/l1 -L l2<output_dir>/p2/l2 -L ..., where p1 is the package l1 is in, etc.

The units

The module units of a package p are all files installed by p that can be found in <opam root>/lib/p/ or a subdirectory.

The page units are those files that can be found in <opam root>/doc/odoc-pages/ or a subdirectory, and that have an .mld extension.

The asset units are those files that can be found in <opam root>/doc/odoc-pages/ or a subdirectory, but that do not have an .mld extension. Additionally, they are all files found in <opam root>/doc/odoc-assets/.

The --parent-id arguments

Interface and implementation units have as parent id the root of the library tree they belong to: with "canonical" roots, <pkgname>/<libname>.

Page units that are found in <opam root>/doc/<pkgname>/odoc-pages/<relpath>/<name>.mld have the parent id from their page tree, followed by <relpath>. So, with canonical roots, <pkgname>/<relpath>.

Asset units that are found in <opam root>/doc/<pkgname>/odoc-pages/<relpath>/<name>.<ext> have the parent id from their page tree, followed by <relpath>. With canonical roots, <pkgname>/<relpath>.

Asset units that are found in <opam root>/doc/<pkgname>/odoc-assets/<filename> have the parent id from their page tree, followed by _asset/<filename> <p>/_assets/<filename>.

The --source-id arguments

The driver could choose the source id without breaking references. However, following the canonical roots convention, implementation units must have as source id: <pkgname>/src/<libraryname>/<filename>.ml.

Ordering the generated pages

The canonical hierarchy introduces directories (one per library) that may not be ordered by the author, either by omitting it in the @children_order tag or by not specifying any @children_order tag. In this case, odoc needs to come up with a reasonable default order, which may not be easy without some help from the driver.

In auto-generated pages (either index.mld in a directory, or page.mld), odoc supports the @order_category <category_name> tag, to help sorting the pages, if it is not sorted by the parent's @children_order. The resulting order is:

  • First, pages in the order given in their parent's @children_order,
  • Then, pages ordered lexicographically by their @order_category. An undefined category comes before a defined one.
  • Inside a category, pages are ordered lexicographically by their first title,
  • Two pages with the same name will be ordered using their file name!

Note that @order_category is not suitable for author use, as it may change in the future. Use this tag only in the driver's autogenerated pages!

OCaml

Innovation. Community. Security.