Detected package downgrade problem

[dominikjeske]

🐛 Bug Report

I have this error during update of my solutions

Updating Dawn.Guard to 1.12.0 from 1.9.0 - 1.10.0 in 6 places since 3 months ago.
NuKeeper.Abstractions.NuKeeperException: Command dotnet failed with exit code: 1

  Determining projects to restore...
  Restored C:\Source\ArcheoFork2\humbak_archeo\infrastructure\Connectors\Connectors.Policies\Connectors.Policies.csproj (in 597 ms).
  Restored C:\Source\ArcheoFork2\humbak_archeo\infrastructure\Connectors\Connectors.Core\Connectors.Core.csproj (in 596 ms).
C:\Source\ArcheoFork2\humbak_archeo\infrastructure\Connectors\Connectors.IdentityServer\Connectors.IdentityServer.csproj : error NU1605: Detected package downgrade: Dawn.Guard from 1.12.0 to 1.9.0. Reference the package directly from the project to select a different version.
C:\Source\ArcheoFork2\humbak_archeo\infrastructure\Connectors\Connectors.IdentityServer\Connectors.IdentityServer.csproj : error NU1605:  mBank.Infra.Connectors.IdentityServer -> mBank.Infra.Connectors.Core -> mBank.Infra.Connectors.Policies -> Dawn.Guard (>= 1.12.0)
C:\Source\ArcheoFork2\humbak_archeo\infrastructure\Connectors\Connectors.IdentityServer\Connectors.IdentityServer.csproj : error NU1605:  mBank.Infra.Connectors.IdentityServer -> Dawn.Guard (>= 1.9.0)
  Failed to restore C:\Source\ArcheoFork2\humbak_archeo\infrastructure\Connectors\Connectors.IdentityServer\Connectors.IdentityServer.csproj (in 1,9 sec).
  1 of 4 projects are up-to-date for restore.



   at NuKeeper.Update.ProcessRunner.ExternalProcess.Run(String workingDirectory, String command, String arguments, Boolean ensureSuccess) in d:\a\r1\a\_NuKeeper PR Build\drop\NuKeeper.Update\ProcessRunner\ExternalProcess.cs:line 63
   at NuKeeper.Update.Process.DotNetUpdatePackageCommand.Invoke(PackageInProject currentPackage, NuGetVersion newVersion, PackageSource packageSource, NuGetSources allSources) in d:\a\r1\a\_NuKeeper PR Build\drop\NuKeeper.Update\Process\DotNetUpdatePackageCommand.cs:line 45
   at NuKeeper.Update.UpdateRunner.Update(PackageUpdateSet updateSet, NuGetSources sources) in d:\a\r1\a\_NuKeeper PR Build\drop\NuKeeper.Update\UpdateRunner.cs:line 57
   at NuKeeper.Local.LocalUpdater.ApplyUpdates(IReadOnlyCollection`1 updates, IFolder workingFolder, NuGetSources sources) in d:\a\r1\a\_NuKeeper PR Build\drop\NuKeeper\Local\LocalUpdater.cs:line 66
   at NuKeeper.Local.LocalUpdater.ApplyUpdates(IReadOnlyCollection`1 updates, IFolder workingFolder, NuGetSources sources, SettingsContainer settings) in d:\a\r1\a\_NuKeeper PR Build\drop\NuKeeper\Local\LocalUpdater.cs:line 55
   at NuKeeper.Local.LocalEngine.Run(SettingsContainer settings, Boolean write) in d:\a\r1\a\_NuKeeper PR Build\drop\NuKeeper\Local\LocalEngine.cs:line 69
   at NuKeeper.Commands.UpdateCommand.Run(SettingsContainer settings) in d:\a\r1\a\_NuKeeper PR Build\drop\NuKeeper\Commands\UpdateCommand.cs:line 52
   at NuKeeper.Commands.CommandBase.OnExecute() in d:\a\r1\a\_NuKeeper PR Build\drop\NuKeeper\Commands\CommandBase.cs:line 108
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.InvokeAsync(MethodInfo method, Object instance, Object[] arguments)
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.OnExecute(ConventionContext context, CancellationToken cancellationToken)
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.<>c__DisplayClass0_0.<<Apply>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync(String[] args, CancellationToken cancellationToken)
   at NuKeeper.Program.Main(String[] args) in d:\a\r1\a\_NuKeeper PR Build\drop\NuKeeper\Program.cs:line 35

Expected behavior

Nuget is updated

Version: 0.29

Platform if applicable:

• 🛠️ NuKeeper CLI
  =

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[CrispyDrone]

Would you be able to create a minimal reproducible example so it would be possible to test this?

[dominikjeske]

I will try to run tool on our current repo and see if I could reproduce this error

[CrispyDrone]

I was looking for some issues and came across this one that seems identical:
#325

This was included in release 0.6.4 so the problem should have been fixed.

Here is some more information on the warning

[Greybird]

Hi @CrispyDrone,

I just had a debug session this week-end about this issue.
Basically, it can still happen when you try to update a solution where there is a indirect transitive dependency between two packages referenced in the same project.

Ex: A project with dependencies as follow:

• N1 (=> N2 => N3)
• N3

The topological sort might not place N3 before N1, because, as the sort is only considering projects dependencies, the N1 to N3 dependency is never materialized, not allowing the topological sort to do its job.

This did not seem an easy fix, as it would require inspecting all nuget dependencies, but at least if this could help...

Another way of fixing is to clean dependencies, but this can be a little bit tricky without R# or an upcoming development by MS to help clean the dependencies.

[CrispyDrone]

Hey @Greybird

Thanks for the additional explanation.

Is it possible that your project is using the new PackageReference format? So no packages.config? I think @dominikjeske 's project is not using packages.config since I can see a dotnet command is being used for the restoring.

When packages.config is used, all dependencies are added to it. This means that the entire dependency graph is actually visible. I think in this case the topological sort might be doing the correct thing?

For PackageReference I think it only takes into account the first level of dependencies, and so transitive references can easily cause this issue I guess.

For this to work properly, we would have to replicate nuget's own dependency graph behavior. I don't know if there's any APIs out there for nuget that allows us to easily find the entire dependency graph of a package, I couldn't find any so far; the only thing I could find is GetMetadataAsync that returns IPackageSearchMetadata which has a property DependencySets, but this is also only 1 level deep. To find the entire graph, we would presumably have to do this recursively.

[CrispyDrone]

Here's the original PR that added the topological sort for packages: #262

I had a question there about why this was actually needed since the simple example of A -> B didn't make it clear to me. It seemed to me that updating A would automatically result in B being updated to the minimal version it needs. Updating B before A on the other hand could result in a breaking change? But maybe this is not a concern of nukeeper, and a CI pipeline with an automatic build/test should take care of that.

#262 (comment)

[Greybird]

Is it possible that your project is using the new PackageReference format? So no packages.config? I think @dominikjeske 's project is not using packages.config since I can see a dotnet command is being used for the restoring.

Yes you are right, this is actually the case.

I had a question there about why this was actually needed since the simple example of A -> B didn't make it clear to me. It seemed to me that updating A would automatically result in B being updated to the minimal version it needs.

It seems this is not the case, when B is explicitly referenced using a PackageReference (it might depend on the version selector strictess though).
It think in a way it makes sense not to update automatically, as PackageReference allows you not to reference transitive dependencies. So if you took the extra step to specify a version for a transitive dependency, it might be an indication that this particular version is important.
I think one of the main issue here is that in a development process where you are incrementally adding references, you very often end up with transitive dependencies references in the project because you needed them before referencing another package that would have brought this first dependency transitively. The lack of cleanup tools makes it a very common situation in my experience, to have actually unneeded dependencies that make the automatic dependencies update fail.

For this to work properly, we would have to replicate nuget's own dependency graph behavior. I don't know if there's any APIs out there for nuget that allows us to easily find the entire dependency graph of a package

Best I am aware of is the dotnet list package --include-transitive command, which might be something to dig into ; It seems to be backed by this method in nuget code