Up to date check proposal

[sfoslund]

CLI Up To Date Check

Goals

This document outlines a new .NET CLI command for checking that a user's toolset is up to date. This command will notify users if any updated versions of the SDK, runtime, or workloads exist and are avaliable for installation.

In future releases we may consider expanding this feature to gather updated release information in the background and notify users of updates more frequently. However, for .NET 6 we will only run the up to date check on an explicit user gesture, i.e. a new CLI command.

Implementation Overview

Gather Updated Product Information

The Microsoft.Deployment.Dotnet.Releases API can be referenced via a nuget package reference and provides methods for gathering all current released product information. Calling this API will provide up to date product data including the latest released SDK and runtime versions for each channel (or major.minor version) and the current support phase (for example, preview or end of life).

Note that this involves fetching the release.json file from the internet, meaning this command will not be supported in offline scenarios. We will need to ensure that the error experience makes this clear to the user.

As this network call is not on a critical path, we will not retry on failure. If we find that this approach causes frequent failures after the first iteration of the feature, we will work on improving back end reliability and/or integrate retries at that time.

Determine Installed SDKs and Runtimes

We will use the same mechanism for determining installed SDKs and runtimes as the dotnet --info command, as differing information on installed SDKs between these two commands would be confusing for users. As a result we will use APIs provided by host fxr to determine what SDKs and runtimes are installed.

As the current implementation of the dotnet --info command determines installed bundles based only on directories being present, it is possible for it to wrongly report that a bundle exists. We will mitigate this issue by adding some further sanity checking to ensure that a given bundle is present, for example, by ensuring that the bundle's directory is not empty.

Select Relevant Product Information

As the releases API provides all product information, we will need to filter this down to data relevant to the user. That is, we only need to calculate and display the following:

Most Recent Patch Version of Installed SDKs and Standalone Runtimes

For each installed SDK or runtime we will find the appropriate major.minor channel (for example, 5.0). Each channel has associated data on the lastest released SDK and runtime, which we can compare to the current version to determine if we need to notify the user that they are out of date.

New Major.Minor or Feature Band SDK Versions

We will select the latest channel and the latest SDK associated with that channel. We can advertise a new SDK version to the user if we find that any of the major, minor, or feature band version numbers are greater than that of the current SDK.

Warning for EOL Support Phase

We can determine if a SDK is still in support by checking the SupportPhase property within the appropriate version channel. If the SDK is end of life or in maintenance mode (meaning it will go out of support within the next 3 months) we will print a warning to the user recommending that they update.

Most Recent Workload Versions

As with SDKs and runtimes, we want to be able to notify a user of available updates for their installed workloads. The vast majority of this work will be covered by the workloads update spec and we will leverage that result to determine which workloads are installed and their most recent versions.

Display command output

Once we have calculated the relevant information we can simply format it and write it to console output.

Outstanding questions

• Workload update check details. These will be outlined in @dsplaisted's workloads spec

[rseanhall]

I think that these two paragraphs conflict with each other:

We will use the same mechanism for determining installed SDKs and runtimes as the dotnet --info command, as differing information on installed SDKs between these two commands would be confusing for users. As a result we will use APIs provided by host fxr to determine what SDKs and runtimes are installed.

As the current implementation of the dotnet --info command determines installed bundles based only on directories being present, it is possible for it to wrongly report that a bundle exists. We will mitigate this issue by adding some further sanity checking to ensure that a given bundle is present, for example, by ensuring that the bundle's directory is not empty.

I think dotnet hosts (and dotnet --info) could be smarter without losing performance, for example if the framework is Microsoft.NETCore.App then it could verify that the Microsoft.NETCore.App.deps.json file exists instead of checking that the directory exists.

[rseanhall]

@vitek-karas @sfoslund In WiX, we are working with @japarson and @MSLukeWest to detect whether a specific runtime is currently installed on the machine. Unlike dotnet/runtime#36479, we really are trying to answer the question "is the runtime installed?" and not the question "Will application requiring framework XYZ run on this machine?". I believe that the SDK here in this issue is also interested in "is the runtime installed?".

Like I mentioned in the comment above, the current spec disagrees with itself. Either the SDK in this new command "will use the same mechanism for determining installed SDKs and runtimes as the dotnet --info command" or it won't. Mitigating the issue of "wrongly" reporting that bundle is installed by verifying the directory is not empty will cause it to no longer agree with the dotnet --info command.

In dotnet/runtime#46128, it sounded like there was internal discussions about this. Like I said there, I think that matching dotnet --info command (and therefore matching what will actually be used when running an application) should be the goal. I am interested to know what the .NET team is thinking on this. I think it would be beneficial to the .NET ecosystem if dotnet.exe, the SDK, and WiX can all agree when a runtime or SDK is "installed".

[vitek-karas]

I absolutely agree that the behavior needs to be consistent. I think that's the main concern - any additional checks we perform on the installed frameworks (to make sure they're really there) will be a perf cost on almost every application launch. That's obviously a rather touchy subject...

Just curious what's your thoughts @rseanhall on the necessity to "validate" frameworks. We have seen cases of empty folders (which cause failures), so that's probably the most likely candidate for validation - simply make sure there's at least something in that folder. I don't think we've seen cases of any other corruption (like most files there, but only some missing and so on).

[rseanhall]

@vitek-karas What did you think about what I said in my initial comment?

I think dotnet hosts (and dotnet --info) could be smarter without losing performance, for example if the framework is Microsoft.NETCore.App then it could verify that the Microsoft.NETCore.App.deps.json file exists instead of checking that the directory exists.

[vitek-karas]

There will always be "some" perf hit - because currently the code doesn't do a "Exists" check, it does directory enumeration (it needs to in order to implement all of the roll-forward capabilities). So any additional check on top of what directory enumeration returns will be "more work". In theory you're right that the .deps.json check will eventually happen anyway already, but unfortunately that's much later on - not during framework resolution.

Currently the perf impact of this would be hardly noticeable - since the host does LOT of file checks in the hostpolicy phase. Hopefully we will be able to get rid of this one day - so I don't want to sign up to a behavior which would mandate more file checks.

That said - maybe we could merge this with the version resolution and only perform the check when we actually consider the version as viable - that should cut down the number of checks to low single digits at worst.

I personally also prefer something like the check for .deps.json - this would fix two cases - empty directory, and also "random files there, which are not .NET Core at all".

[rseanhall]

These appear to be directory checks during framework resolution:

https://github.com/dotnet/runtime/blob/1daa25753f5824acb1f67f306ff5abf3a3daebeb/src/installer/corehost/cli/fxr/fx_resolver.cpp#L229

https://github.com/dotnet/runtime/blob/1daa25753f5824acb1f67f306ff5abf3a3daebeb/src/installer/corehost/cli/fxr/fx_resolver.cpp#L256

[vitek-karas]

You're right - that code is just... oh well. The first one is not that interesting (that's only for basically disable roll-forward which is rarely used), the second one is the interesting one. I have no idea why we do directory check there, since the value we use come directly from directory enumeration... https://github.com/dotnet/runtime/blob/1daa25753f5824acb1f67f306ff5abf3a3daebeb/src/installer/corehost/cli/fxr/fx_resolver.cpp#L240

In that case you're absolutely right - just replacing that check with the .deps.json check would basically solve this.

[sfoslund]

Implementation has been finished with #16141