Command line interface

This manual describes how your tool ends up interacting with shells when you use Cmdliner.

Tool invocation

For tools evaluating a command without subcommands the most general form of invocation is:

tool [OPTION]… [ARG]…

The tool automatically reponds to the --help option by printing the help. If a version string is provided in the command information, it also automatically responds to the --version option by printing this string on standard output.

Command line arguments are either optional or positional. Both can be freely interleaved but since Cmdliner accepts many optional forms this may result in ambiguities. The special token -- can be used to resolve them: anything that follows it is treated as a positional argument.

Tools evaluating commands with subcommands have this form of invocation

tool [COMMAND]… [OPTION]… [ARG]…

Commands automatically respond to the --help option by printing their help. The sequence of COMMAND strings must be the first strings following the tool name – as soon as an optional argument is seen the search for a subcommand stops.

Arguments

Optional arguments

An optional argument is specified on the command line by a name possibly followed by a value.

The name of an option can be short or long.

• A short name is a dash followed by a single alphanumeric character: -h, -q, -I.
• A long name is two dashes followed by alphanumeric characters and dashes: --help, --silent, --ignore-case.

More than one name may refer to the same optional argument. For example in a given program the names -q, --quiet and --silent may all stand for the same boolean argument indicating the program to be quiet.

The value of an option can be specified in three different ways.

• As the next token on the command line: -o a.out, --output a.out.
• Glued to a short name: -oa.out.
• Glued to a long name after an equal character: --output=a.out.

Glued forms are especially useful if the value itself starts with a dash as is the case for negative numbers, --min=-10.

An optional argument without a value is either a flag (see Cmdliner.Arg.flag, Cmdliner.Arg.vflag) or an optional argument with an optional value (see the ~vopt argument of Cmdliner.Arg.opt).

Short flags can be grouped together to share a single dash and the group can end with a short option. For example assuming -v and -x are flags and -f is a short option:

• -vx will be parsed as -v -x.
• -vxfopt will be parsed as -v -x -fopt.
• -vxf opt will be parsed as -v -x -fopt.
• -fvx will be parsed as -f=vx.

Positional arguments

Positional arguments are tokens on the command line that are not option names and are not the value of an optional argument. They are numbered from left to right starting with zero.

Since positional arguments may be mistaken as the optional value of an optional argument or they may need to look like option names, anything that follows the special token "--" on the command line is considered to be a positional argument:

tool --option -- but --now we -are --all positional --argu=ments

Constraints on option names

Using the cmdliner library puts the following constraints on your command line interface:

• The option names --cmdliner and --__complete are reserved by the library.
• The option name --help, (and --version if you specify a version string) is reserved by the library. Using it as a term or option name may result in undefined behaviour.
• Defining the same option or command name via two different arguments or terms is illegal and raises Invalid_argument.

Environment variables

Non-required command line arguments can be backed up by an environment variable. If the argument is absent from the command line and the environment variable is defined, its value is parsed using the argument converter and defines the value of the argument.

For Cmdliner.Arg.flag and Cmdliner.Arg.flag_all that do not have an argument converter a boolean is parsed from the lowercased variable value as follows:

• "", "false", "no", "n" or "0" is false.
• "true", "yes", "y" or "1" is true.
• Any other string is an error.

Note that environment variables are not supported for Cmdliner.Arg.vflag and Cmdliner.Arg.vflag_all.

Help and man pages

Help and man pages are are generated when you call your tool or a subcommand with --help. By default, if the TERM environment variable is not dumb or unset, the tool tries to page the manual so that you can directly search it. Otherwise it outputs the manual as plain text.

Alternative help formats can be specified with the optional argument of --help, see your own tool --help for more information.

tool --help
tool cmd --help
tool --help=groff > tool.1

Paging

The pager is selected by looking up, in order:

1. The MANPAGER variable.
2. The PAGER variable.
3. The tool less.
4. The tool more.

Regardless of the pager, it is invoked with LESS=FRX set in the environment unless, the LESS environment variable is set in your environment.

Install

The manpages of a tool and its subcommands can be installed to a root man directory $MANDIR by invoking:

cmdliner install tool-manpages thetool $MANDIR

This looks up thetool in the PATH. Use an explicit file path like ./thetool to directly specify an executable.

If you are also installing completions rather use the install tool-support command, see this cookbook tip which also has instructions on how to install if you are using opam.

Command line completion

Cmdliner programs automatically get support for shell command line completion.

The completion process happens via a protocol which is interpreted by generic shell completion scripts that are installed by the library. For now the zsh and bash shells are supported.

Tool developers can easily install completion definitions that invoke these completion scripts. Tool end-users need to make sure these definitions are looked up by their shell.

End-user configuration

If you are the user of a cmdliner based tool, the following shell-dependent steps need to be performed in order to benefit from command line completion.

For zsh

The FPATH environment variable must be setup to include the directory where the generic cmdliner completion function is installed before properly initializing the completion system.

For example, for now, if you are using opam. You should add something like this to your .zshrc:

FPATH="$(opam var share)/zsh/site-functions:${FPATH}"
autoload -Uz compinit
compinit -u

Also make sure this happens before opam's zsh init script inclusion, see this issue. Note that these instruction do not react dynamically to opam switches changes so you may see odd completion behaviours when you do so, see this this opam issue.

After this, to test everything is right, check that the _cmdliner_generic function can be looked by invoking it (this will result in an error).

> autoload _cmdliner_generic
> _cmdliner_generic
_cmdliner_generic:1: words: assignment to invalid subscript range

If the function cannnot be found make sure the cmdliner library is installed, that the generic scripts were installed and that the _cmdliner_generic file can be found in one of the directories mentioned in the FPATH variable.

With this setup, if you are using a cmdliner based tool named thetool that did not install a completion definition. You can always do it yourself by invoking:

autoload _cmdliner_generic
compdef _cmdliner_generic thetool

For bash

These instructions assume that you have bash-completion installed and setup in some way in your .bashrc.

The XDG_DATA_DIRS environment variable must be setup to include the share directory where the generic cmdliner completion function is installed.

For example, for now, if you are using opam. You should add something like this to your .bashrc:

XDG_DATA_DIRS="$(opam var share):${XDG_DATA_DIRS}"

Note that these instruction do not react dynamically to opam switches changes so you may see odd completion behaviours when you do so, see this this opam issue.

After this, to test everything is right, check that the _cmdliner_generic function can be looked up:

> _completion_loader _cmdliner_generic
> declare -F _cmdliner_generic &>/dev/null && echo "Found" || echo "Not found"
Found!

If the function cannot be found make sure the cmdliner library is installed, that the generic scripts were installed and that the _cmdliner_generic file can be looked up by _completion_loader.

With this setup, if you are using a cmdliner based tool named thetool that did not install a completion definition. You can always do it yourself by invoking:

_completion_loader _cmdliner_generic
complete -F _cmdliner_generic thetool

Note. It seems _completion_loader was deprecated in bash-completion 2.12 in favour of _comp_load but many distributions are on < 2.12 and in 2.12 _completion_loader simply calls _comp_load.

Install

Completion scripts need to be installed in subdirectories of a share directory which we denote by the $SHAREDIR variable below. In a package installation script this variable is typically defined by:

SHAREDIR="$DESTDIR/$PREFIX/share"

The final destination directory in share depends on the shell:

• For zsh it is $SHAREDIR/zsh/site-functions
• For bash it is $SHAREDIR/bash-completion/completions

If that is unsatisfying you can output the completion scripts directly where you want with the cmdliner generic-completion and cmdliner tool-completion commands.

Generic completion scripts

The generic completion scripts must be installed by the cmdliner library. They should not be part of your tool install. If they are not installed you can inspect and install them with the following invocations, invoke with --help for more information.

cmdliner generic-completion zsh   # Output generic zsh script on stdout
cmdliner install generic-completion $SHAREDIR             # All shells
cmdliner install generic-completion --shell zsh $SHAREDIR # Only zsh

Directories are created as needed. Use option --dry-run to see which paths would be written by an install invocation.

Tool completion scripts

If your tool named thetool uses Cmdliner you should install completion definitions for them. They rely on the generic scripts to be installed. These tool specific scripts can be inspected and installed via these invocations:

cmdliner tool-completion zsh thetool  # Output tool zsh script on stdout.
cmdliner install tool-completion thetool $SHAREDIR             # All shells
cmdliner install tool-completion --shell zsh thetool $SHAREDIR # Only zsh

Directories are created as needed. Use option --dry-run to see which paths would be written by an install invocation.

If you are also installing manpages rather use the install tool-support command, see this cookbook tip which also has instructions on how to install if you are using opam.

Completion protocol

There is no standard that allows tools and shells to interact to perform shell command line completion. Completion is supposed to happen through idiosyncratic, ad-hoc, obscure and brain damaging shell-specific completion scripts.

To alleviate this, Cmdliner defines one generic script per shell and interacts with it using the protocol described below. The protocol can be used to implement generic completion scripts for other shells. The protocol is versioned but can change even between minor versions of Cmdliner. Generic scripts for popular shells can be inspected via the cmdliner generic-completion command.

The protocol betwen the shell completion script and a cmdliner based tool is as follows:

1. When completion is requested the script invokes the tool with a modified command line:

   • The first argument to the tool (Sys.argv.(1)) must be the option --__complete.
   • The (possibly empty) argument ARG on which the completion is requested must be replaced by exactly --__complete=ARG. Note that this can happen after the -- token, this is the reason why we have an explicit --__complete argument in Sys.argv.(1): it indicates the command line parser must operate in a special mode.
2. The tool responds by writing on standard output a list of completion directives which match the completions rule of the grammar given below.
3. The script interprets the completion directives according to the given semantics below so that the shell can display the completions. The script is free to ignore directives or data that it is unable to present.

The following ABNF grammar is described using the notations of RFC 5234 and RFC 7405. A few constraints are not expressed by the grammar:

• Except in the completion rule, the byte stream may contain ANSI escape sequences introduced by the byte 0x1B.
• After stripping the ANSI escape sequences, the resulting byte stream must be valid UTF-8 text.

completions = version nl directives
version = "1"
directives = *(directive nl)
directive = message / group / %s"files" / %s"dirs" / %"restart"
message = %s"message" nl text nl %s"message-end"
group = %s"group" nl group_name nl *item
group_name = *pchar
item = %s"item" nl completion nl item_doc nl %s"item-end"
completion = *pchar
item_doc = text
text = *(pchar / nl)
nl = %0A
pchar = %20-%7E / %8A-%FF

The semantics of directives is as follows:

• A message directive defines a message to be reported to the user. It is multi-line ANSI styled text which cannot have a line that is exactly made of the text message-end as it is used to signal the end of the message. Messages should be reported in the order they are received.
• A group directive defines an informational group_name followed by a possibly empty list of completion items that are part of the group. An item provides a completion value, this is a string that defines what the requested ARG value can be replaced with. It is followed by an item_doc, multi-line ANSI styled text which cannot have a line that is exactly made of the text item-end as it is used to signal the end of the item.
• A file directive indicates that the script should add existing files staring with ARG to completion values.
• A dir directive indicates that the script should add existing directories starting with ARG to completion values.
• A restart directive indicates that the script should restart shell completion as if the command line was starting after the leftmost -- disambiguation token. The directive never gets emited if there is no -- on the command line.

You can easily inspect the completions of any cmdliner based tool by invoking it like the protocol suggests. For example for the cmdliner tool itself:

cmdliner --__complete --__complete=

Error message ANSI styling

Since Cmdliner 2.0 error messages printed on stderr use styled text with ANSI escapes unless one of the following conditions is met:

• The NO_COLOR environment variable is set and different from the empty string. Yes, even if you have NO_COLOR=false, that's what the particularly dumb https://no-color.org standard says.
• The TERM environment variable is dumb.
• The TERM environment variable is unset and Sys.backend_type is not Other "js_of_ocaml". Yes, browser consoles support ANSI escapes. Yes, you can run Cmdliner in your browser.

Legacy prefix specification

Before Cmdliner 2.0, command names, long option names and Cmdliner.Arg.enum values could be specified by a prefix as long as the prefix was not ambiguous.

This turned out to be a mistake. It makes the user experience of the tool unstable as it evolves: former user established shortcuts or invocations in scripts may be broken by new command, option and enumerant additions.

Therefore this behaviour was unconditionally removed in Cmdliner 2.0. If you happen to have scripts that rely on it, you can invoke them with CMDLINER_LEGACY_PREFIXES=true set in the environment to recover the old behaviour. However the scripts should be fixed: this escape hatch will be removed in the future.

The CMDLINER_LEGACY_PREFIX=true escape hatch should not be used for interactive tool interaction. In particular the behaviour of Cmdliner completion support under this setting is undefined.