Option to install latest version detected by checkver

[shoogle]

Many apps have "nightly builds" that are published multiple times a day. It doesn't make sense to update the manifest each time one of these development builds is released, so it would be good if there was an option to always download the latest version as detected by checkver.

For example, MuseScore publishes its nightly builds on this page and the latest can be detected using the following checkver syntax:

"checkver": {
  "url": "https://ftp.osuosl.org/pub/musescore-nightlies/windows/",
  "re": ">MuseScoreNightly-(\\d{4}-\\d{2}-\\d{2}-\\d{4}-master-[0-9a-f]{7})\\.7z</"
  }

It would be great if there was an option, specifically for nightly/development builds, to have scoop install [app] and scoop update [app] always fetch the most recent nightly detected by checkver regardless of the version number hard-coded into the manifest.

[Ash258]

This could be achieved same way as vscode-insiders (for example). You only need some static URL, where is the newest archive available. Then you would download always the newest one and updating will be possible with scoop update musescore-nightly -kf. No need for checkver and autoupdate.

I can't find any static latest static URL on that page after fast going through that page. Maybe develepors could provide some latest.7z symlink (or just copy).

It should be pretty easy since, they use batch scripting for this. Just one COPY funciton on this line and new scp line which, copy latest. Or even easier. Just on scp, which will copy it and automatically rename.

[shoogle]

You only need some static URL, where is the newest archive available.

That would indeed work, and I might adopt that as a solution in this case, but in the general case it might not be possible to modify upstream's nightly scripts if the app is closed source or if its build scripts are kept on a private server.

[Ash258]

In general speaking there is no problem with frequent updates. All autoupdates are process automatically by Excavator, which by default check all apps for updates every hour and push updates.

Best example is ffmpeg-nighlyt, there are changes each day (sometimes more; 20190107)

I don't see any huge advantage of autoupdating on user side and then installing it.

It doesn't make sense to update the manifest each time one of these development builds is released

Why?

[shoogle]

Well nightly builds are kind of a special case because:

1. They are updated much more frequently than stable release builds.
2. They are not retained permanently on the server.
3. Regular users don't use them.
4. They are not used for mission critical applications.

For example, MuseScore releases upwards of 5 nightly builds per day, every day. The builds are only retained for a month or so, so there is no value to having a permanent record of what the URL and hash used to be for builds that no longer exist. Since the nightlies are only used by developers, and not for mission critical purposes, there is not the same need to verify checksums as there is for stable builds.

There is also the matter of courtesy to scoop users. You don't want to pollute the Git history with tons of updates to nightly manifests, and I when I update scoop I certainly don't want to download all those manifest updates for nightly builds of apps for which I don't even have the stable version installed; it's a huge waste of time, energy and bandwidth.

And then there is the matter of courtesy to scoop developers. Due to the way git works, the repository grows in size every time a file is changed (even if that change results in fewer lines of code). As the repository grows larger, checkout and branching operations become slower for everyone.

In my opinion there shouldn't be any nightly builds in the official scoop buckets, and certainly not if the nightly manifest is going to be updated for each new nightly release. (I plan to make a separate bucket for MuseScore's nightly so that only people who want it have to put up with the constant updates, but I'd much prefer the constant updates were not necessary, hence this issue.)

[Ash258]

This could be implemented using new manifest "type", which could be called template or skeleton. It would be completely same as normal manifest, but with some hardcoded / prefilled properties (just to make autoupdate possible and safe). There will be new property (template), which indicates this type of manifest.

Implementation

1. Present new bucket, which would conaints only templates (for advanced users, which do not need these)
   1. Readme
      1. Explanation of what it is for
      2. Forced update is needed
2. New optional manifest property [bool]template, default to false
3. Actual implementation:
   1. https://github.com/lukesampson/scoop/blob/master/lib/install.ps1#L19

   if ($manifest.template) {
   	Copy-Item template into workspace folder
   	. checkver.ps1 $app $workspace_folder other variables
   	$manifest = parse_app generated file from workspace
   	$use_cache = $false # Disable cache to prevent installing from older version in some cases
   }

4. Move all manifests, which could benefits from templates into new bucket.

[shoogle]

I'm not sure we really need a whole new type of manifest. I was thinking something more like this:

scoop install --checkver [appname]
scoop update --checkver [appname]

Or alternatively (and preferably) a "magic" version string in the manifest like "version": "checkver" or something like that. I believe there are already some magic version strings like "nightly" and "github", so this shouldn't be too controversial.

[Ash258]

Implementation would be same. There is only need to find best way how to identify, this type.

[rashil2000]

Closing in favour of #4077