conf-* packages shouldn't be upgraded

[ivg]

The default policy in OPAM is to maximize versions of installed packages, that makes sense with the packages that are distributed with OPAM, but is wrong, however, when we are dealing with the conf-packages that represent (a) system configuration and (b) set of available system packages. When a user installs conf-llvm.XXX version, that is not the latest, this is a user way of saying to us: "Guys, I want this XXX version, not the latest, maybe because I don't have the latest in the system".

As an example, consider the following interaction, we will use bap in the example, that supports llvm 3.4,3.8, and 4.0. However, we are installing on a system that has only 3.8.

opam install conf-llvm.3.8 # OK
opam install bap # fails since OPAM is upgrading conf-llvm.3.8 to conf-llvm.4.0 

OK, let's constraint it

opam install conf-llvm.3.8
opam install conf-llvm.3.8 bap

This works, but suppose a user is trying to install another package, totally independent of bap, e.g., merlin:

opam install merlin
The following actions will be performed:
  ↗  upgrade   conf-llvm      3.8 to 4.0.0
  ∗  install   conf-which     1               [required by biniou]
  ∗  install   easy-format    1.3.0           [required by yojson]
  ↻  recompile bap-llvm       master          [uses conf-llvm]
  ∗  install   biniou         1.2.0           [required by yojson]
  ↻  recompile bap-x86        master          [uses bap-llvm]
  ↻  recompile bap-arm        master          [uses bap-llvm]
  ∗  install   yojson         1.4.0           [required by merlin]
  ↻  recompile bap-primus-x86 master          [uses bap-x86]
  ∗  install   merlin         3.0.2       
  ↻  recompile bap            master          [uses bap-llvm]

This will break bap since it again will try to upgrade conf-llvm and recompile the whole bap.

So, what I'm thinking of, is it possible:

1. treat conf-packages differently, by applying some non-default criteria to them
2. maybe add an option for packages to specify criteria in the opam files.

P.S. of course I'm well aware, that I can specify my own criteria using the command line and environment variables, but my concern are regular users (students), who will try to install bap in their system and to whom both OCaml and OPAM are totally new ideas, and such behavior will piss them off.

P.P.S. (this is the reason why we had conf-bap-llvm)

[ivg]

Also, even in the OPAM documentation it is said:

Default preferences for an install

When you request to install a (set of) package(s), in general you do not expect to see all your existing packages updated, and this is why in this case opam uses a different default value -removed,-changed,-notuptodate that tries to minimise changes to the system. It can be set using the environment variable OPAMCRITERIA

But it looks like that this is no longer true, as, just using opam install without criteria, would try to upgrade the whole world, while if I specify criteria as they are specified in documentation, then I will have a sane behavior:

$ opam install merlin --criteria='-removed,-changed,-notuptodate'
The following actions will be performed:
  ∗  install easy-format 1.3.0                [required by yojson]
  ∗  install biniou      1.0.5                [required by yojson]
  ∗  install yojson      1.1.3                [required by merlin]
  ∗  install merlin      3.0.2

So it looks like that this documentation page is no longer true. It looks like that the install criteria are:

-count(removed),-notuptodate(request),-sum(request,version-lag),-count(down),-notuptodate(changed),-count(changed),-notuptodate(solution),-sum(solution,version-lag)

where the upgrade is

-count(down),-count(removed),-notuptodate(solution),-sum(solution,version-lag),-count(new)

[dbuenzli]

Note that opam v2 has the ability to specify that a package is a conf package.

[ivg]

conf: this is a "conf" package, that is intended to document capabilities of the system, or the presence of software installed outside of opam. As such, the package may not include a source URL or install anything, but just do some checks, and fail to install if they don't pass. conf packages should have a name starting with conf-, and include the appropriate depexts: field.

Yep, so the next step is to apply a special criterium to them, something like "do not touch".

[dbuenzli]

Maybe this should be reported there.

[ivg]

Maybe this should be reported there.

You mean in opam itself? Yeah, this is a good point.

[ivg]

So, if we will not confuse the system package versions with the opam conf package version, like it is described here, then there is no need for special treatment of the conf-* packages. If everyone will agree with the conf-<system-package>-<version_number> convention, we can move forward and close this issue.

[kit-ty-kate]

Is "pinning" not suitable for you ? opam pin conf-llvm $version

[dbuenzli]

If everyone will agree with the conf--<version_number> convention, we can move forward and close this issue.

Doesn't that become a bit unwieldy ?

[ivg]

Is "pinning" not suitable for you ?

I'm trying to make installation of my package as smooth as possible. Basically, it should be one command. If an installation of a package requires a user to perform commands, then basically it means that we are missing some capabilities in the package manager. It is the responsibility of the package manager to resolve all the dependencies.

P.S. Of course I know, how to force opam to behave as I like, by just setting the right criteria to the ASP solver. So I'm not looking for a workaround, I'm pointing that there is some problem with the idea of conf-* packages.

[ivg]

Doesn't that become a bit unwieldy ?

That's what I'm trying to figure out.

[dbuenzli]

I'm pointing that there is some problem with the idea of conf-* packages.

I really think this should be discussed on opam's bug tracker @AltGr doesn't watch this repo and v2 is where these things could be solved correctly along with the depext overhaul ocaml/opam#2919

[ivg]

@dbuenzli, I agree, would you like me to reopen this issue on opam's bug tracker?

[dbuenzli]

Yes I think it's better to do so. This tracker is for issue about the OCaml opam repository itself. If you think your issue can be solved by changing how the OCaml opam repository is structured then keep it open otherwise, I'd suggest you to close this and move the discussion to the opam issue tracker.

[ivg]

moved to ocaml/opam#3035