Make install faster by not scanning the whole switch when no install field is present

[kit-ty-kate]

Implements #4422

Currently opam scans the whole switch directory recursively twice when installing a package (per package). This introduces a huge strain on the IO of the machine. This is especially visible under high load (e.g. in CI)

This PR makes so that opam only scan the files that are going to be installed (if they exist already, this is rare) if the package does not have any install field. If the package does have an install field the old behaviour is used.

[kit-ty-kate]

This PR is now ready for review. cc @AltGr

[AltGr]

This is more complicated than what I would have thought at first; I was initially pretty confused by track_files comments saying it is "similar to [track]", while the latter is a wrapper around the execution of a job and the former just an atomic function that expects the results of a previous run as argument; there is nothing wrong with that, but what were the reasons not to go for the more straight-forward extraction of the list of files from .install, followed by the run of a version of track that is limited to a given set of files ? It would be useful to document more precisely the usage of track_files in opamDirTrack.mli.

Another remark, it seems the behaviour on pre-install hooks is changed, since they are no longer tracked. Have you studied if that had any consequences in the wild ? Though the only other option would be to disable the optimised tracking as soon as any pre-install hook is defined, I guess... The tracking behaviour for post-install is already specific and detailed here.

I am otherwise happy with the code and this will be nice to have!

[rjbou]

Just to keep a trace of the test (can't be added in reftest):

$ cat dotty/dot.install
bin: [ "script" ]
$ cat dotty/dot.opam
opam-version: "2.0"
build: [ "sh" "-c" "echo hellow > script%{exe}%" ]
$ opam pin -yn ./dotty
$ opam install dot #with win executable name
<><> Synchronising pinned packages ><><><><><><><><><><><><><><><><><><><><><><>
[dot.0.1] synchronised (no changes)

The following actions will be performed:
  ∗ install dot 0.1*

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[WARNING] .install file is missing .exe extension for script
[WARNING] Automatically adding .exe to <switchprefix>/build/dot.0.1/script.exe
∗ installed dot.0.1
Done.

[AltGr]

Just a teeny bit of nitpicking, otherwise all good! ;)