Pin and dev packages allow opam files to be generated

[dbuenzli]

In #3793 it has been proposed that one opam file should be able to define multiple package so as to be able to share metadata fields.

It is my impression that losing the property one file = one package makes things more complex in practice both for opam's implementation and the end-user experience while letting a few problems be unsolved:

1. In general it is the build system that has the package dependency information. Hence the natural information flow is to let the build system define the opam file.
2. The kind of metadata fields opam defines is required in other processes and systems that surround software development (e.g. system package managers) and there's no deep reason why we should actually store these in opam files.

Both these problems can be solved if we let another tool --- most likely the build system --- be the source of metadata and let it treat opam files and other metadata files as file formats that it can build given its own representation of metadata.

In particular this tool may allow metadata sharing to streamline the release of multiple packages from the same source while allowing opam to keep its file format simple and thus avoid the complications that may stem out of #3793.

This however breaks the pin and development package worfklows which all assume that one or more up-to-date opam files are present in the pin and/or dev package. It would be possible to commit to the dev repo opam files generated by the tool mentioned above but from an end-user perspective this is both annoying and error-prone.

As such I would like to see if it would be possible for pins and dev packages to have the following two step procedure. The opam file in the pin or dev package may simply be an opam file with one or more of the following two fields (feel free to find better names):

define-depends: [ <filtered-package-formula> ... ]
define: [ [ <term> { <filter> } ... ] { <filter> } ... ]
define-dir: [<string>] # defaults to "."

• The define-depends field has a dependencies that need to be installed right away; this will typically be the package of the tool that handles the metadata (i.e. most likely the build system).
• The define field has commands that need to be invoked after the define-depends dependencies are installed and that will write opam files in the build directory according to the conventions that already exist.
• The define-dir field may specifiy where the files were actually generated (this is not strictly needed but could allow to point to a directory in the build directory of the build system which makes things a bit more clean).

Once, define-depends have been installed and define has be run. opam would proceed the usual way it always did.

Also the idea would be that these fields are forbidden/ignored in regular opam packages since it makes the system quite slower in practice (e.g. the solver is invoked twice).

Does something like this sound feasible ?

[AltGr]

I like this better too, and indeed generating the opam files seems like the way to go for me.

[ghost]

I second this, that would be really useful for dune projects!