Spec for opam CLI versioning

Summary

With OPAM 1.2.2 essentially retired, at present there is only one released opam binary and one CLI to worry about. opam 2.1 introduces various new options and new behaviours and we need to consider the impact this may have on cases where opam is used in scripts or inside other systems and which must either support multiple versions of opam, or which do not want to be broken by future versions of opam.

This specification proposes a new CLI option --cli=major.minor and environment variable OPAMCLI where major and minor are the major and minor components of a released opam version (so presently 2.0 or 2.1). OPAMCLI exists partly because most opam CLI options have environment variable counterparts and also to allow control of a script/system which doesn't use the --cli option to be controlled by its caller. It is intended that any script would use the --cli for all its calls to opam, for example as:

opam='opam --cli=2.1'

$opam init
$opam switch create foo --empty

This feature is very similar to, and indeed inspired by, the lang stanza of Dune.

Status

This feature will be added to opam 2.1 pre-beta. There is no value in back-porting it to 2.0. If a script must support the opam 2.0 client, then it should export OPAMCLI=2.0 (we should highlight in documentation and feature announcements that export OPAMCLI=2.1 is not recommended). The feature could be back-ported for 2.0.8, although it would still require scripts to be written:

#!/usr/bin/env bash

opam='opam --cli=2.0'
if ! $($opam) &>/dev/null ; then
  export OPAMCLI='2.0'
  opam='opam'
fi

# ...

so the back-port seems to add little.

Implementation

Checking the CLI version must be the first task opam performs. --cli must be permitted to appear at any position in the command line arguments prior to any -- indicator. In particular, it may appear before a sub-command (since it may control both the availability and behaviour of subcommands). Either --cli=2.1 may be used or --cli 2.1 form. The option may be specified more than once, with the rightmost version taking precedence. If no --cli is given, then OPAMCLI is preferred. Note that OPAMCLI is not inspected at all if --cli was given. If any --cli or, if none was given, OPAMCLI has a corrupt or missing value, then processing fails. All --cli items are filtered out of Sys.argv and command line processing continues as if they were not present. If the specified version is not supported, then the behaviour is:

• If --cli was specified and the filtered command line is empty (i.e. opam was only called with --cli arguments) then opam displays no message and exits with code 1. This allows trivial testing of the opam client in scripts:

if ! opam --cli=2.2 ; then
  echo 'Need at least opam 2.2'
  exit 1
fi

• Otherwise, opam behaves as for a bad option value. For OPAMCLI=2.2 opam (i.e. OPAMCLI at fault and no --cli given), the text below shall say OPAMCLI: invalid value...:

opam: option `--cli': invalid value `2.2', expected one of
      `2.0' or `2.1'. 
Usage: opam COMMAND ...
Try `opam --help' for more information.

Having successfully parsed the CLI version, this information is used not only to construct the cmdliner parser but also in the logic for commands. Various notes:

• New command line options still work, but they display more informative error messages. Their --help documentation should state (only available for 2.1+) for --cli=2.0. For example, OPAMCLI=2.0 opam switch create --formula=foo should display an error that --formula is only available since version 2.1 of the opam CLI, but version 2.0 has been requested via the OPAMCLI environment variable (or via the --cli option)
• CLI versioning doesn't require opam to support old features. For example, if a feature has been removed then opam --cli=2.4 switch explode might display The explode subcommand was removed in opam 3.0. You can achieve a similar effect with rm -rf $(opam switch seek-and-locate), but without the cool ANSI art.
• In particular, this means that plugins which become sub-commands are always sub-commands, but it does provide the ability for the plugin's CLI to be used (e.g. for opam depext)
• There is no need to deprecate either options or behaviours - they can be immediately changed in a version, since the previous behaviour remains. In this instance, --help would display --explode (renamed to --erase in 2.1+) for 2.0 and --explode is now --erase in 2.1. opam --cli=2.1 --explode would display an error message: Since 2.1, you must use --erase, not --explode (see --cli for information on CLI versioning). opam --cli=2.0 --explode would proceed without warning or message.
• If a feature is actually to be removed completely at some point, then in that version the item becomes unavailable with a warning: The explode sub-command will be removed in a future release of opam. You may still access it by specifying --cli=2.4. Note this makes trivial changes like renaming or enhancing an option easy, but purposefully makes the use of features canned for destruction with new features difficult.

There should be no reason ever to remove these alterations - we might at a major version bump stop displaying the information about old options in --help (but still support the clear error messages for them). One benefit of this approach is that we will never accidentally re-use an option.

@avsm: note that we can get rid of opam 1.2 legacy like opam config exec with CLI=2.1+