Should MSBuildLocator handle loading of NuGet assemblies?

[rynowak]

Using this with .NET sdk 3.1.X - I get errors like the following if I run any build targets.

Build FAILED.

/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018: The "ProcessFrameworkReferences" task failed unexpectedly.
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018: System.IO.FileNotFoundException: Could not load file or assembly 'NuGet.Frameworks, Version=5.4.0.2, Culture=neutral, PublicKeyToken=31bf3856ad364e35'. The system cannot find the file specified.
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018:
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018: File name: 'NuGet.Frameworks, Version=5.4.0.2, Culture=neutral, PublicKeyToken=31bf3856ad364e35'
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018:    at System.Linq.Enumerable.WhereEnumerableIterator`1.ToList()
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018:    at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018:    at Microsoft.NET.Build.Tasks.ProcessFrameworkReferences.ExecuteCore()
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018:    at Microsoft.NET.Build.Tasks.TaskBase.Execute()
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask)
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018:
/usr/local/share/dotnet/sdk/3.1.102/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(59,5): error MSB4018:
    0 Warning(s)
    1 Error(s)

I think that's happening here is that the tasks assembly Microsoft.NET.Build.Tasks.dll has references to NuGet's assemblies. If this were running as part of a dedicated msbuild process it would be fine because these would be in the load context. However since I'm hosting msbuild in my process they are not.

I was able to work around this with my own assembly resolve handler, but it doesn't feel good to maintain a lot of assemblies that msbuild is usually responsible for providing.

[rainersigwald]

Yes, this should be built into Locator. The MSBuild-NuGet relationship is only getting closer over time; we should treat it as part of MSBuild in this context.

[bettinaheim]

@rainersigwald We are running into the same issue here. A fix for this would really be awesome!

[rainersigwald]

In dotnet/msbuild#5241, @zivkan notest that this fails only on Core, which is extra funky.

I'll sync with @microsoft/msbuild-maintainers on scheduling this in the next sprint planning (late this week? I think).

[cdmihai]

Doesn't this problem generalize to more than just nuget? Wouldn't any dll that MSBuild redistributes also need handling in locator? Or, even more generally, any dll that MSBuild loads and isn't part of some standard BCL path probing path, needs handling in the locator? dotnet dll loading and probing logic was always a big confusing mess to me. :)

[rynowak]

I think that's true, but the nuget dlls are highly likely to have breaking changes between versions 😆. Something like JSON.NET (for example) doesn't typically make binary breaking changes so it's less crucial to use the exact version that MSBuild ships.

[rynowak]

I've now also encountered the same issue with Microsoft.DotNet.PlatformAbstractions. NBGV loads LibGit2Sharp which loads Microsoft.DotNet.PlatformAbstractions

[rainersigwald]

For Core, we should think about having a universal fallback to next-to-MSBuild. That would catch a bunch of SDK-delivered stuff.

However

NBGV loads LibGit2Sharp which loads Microsoft.DotNet.PlatformAbstractions

I don't see this disassembling LibGit2Sharp 0.25.0.0 that comes with NBGV 2.2.13. How did you figure that out?

[rynowak]

The error message told me! I think I sent the exact error message to you in teams a few days ago. Are you saying that your chat window is not an issue tracker? 😆

[4/17 3:26 PM] Ryan Nowak
    
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018: The "Nerdbank.GitVersioning.Tasks.GetBuildVersion" task failed unexpectedly.
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018: System.TypeInitializationException: The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception.
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:  ---> System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.DotNet.PlatformAbstractions, Version=2.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The system cannot find the file specified.
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018: File name: 'Microsoft.DotNet.PlatformAbstractions, Version=2.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at Nerdbank.GitVersioning.GitLoaderContext.GetNativeLibraryDirectory()
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at Nerdbank.GitVersioning.GitLoaderContext.LoadUnmanagedDll(String unmanagedDllName)
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at System.Runtime.Loader.AssemblyLoadContext.ResolveUnmanagedDll(String unmanagedDllName, IntPtr gchManagedAssemblyLoadContext)
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at LibGit2Sharp.Core.NativeMethods.git_libgit2_init()
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at LibGit2Sharp.Core.NativeMethods.InitializeNativeLibrary()
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at LibGit2Sharp.Core.NativeMethods..cctor()
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    --- End of inner exception stack trace ---
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at LibGit2Sharp.Core.NativeMethods.git_libgit2_opts(Int32 option, UInt32 level, String path)
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at LibGit2Sharp.Core.Proxy.git_libgit2_opts_set_search_path(ConfigurationLevel level, String path)
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at LibGit2Sharp.GlobalSettings.SetConfigSearchPaths(ConfigurationLevel level, String[] paths)
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at Nerdbank.GitVersioning.GitExtensions.OpenGitRepo(String pathUnderGitRepo, Boolean useDefaultConfigSearchPaths)
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at Nerdbank.GitVersioning.VersionOracle.Create(String projectDirectory, String gitRepoDirectory, ICloudBuild cloudBuild, Nullable`1 overrideBuildNumberOffset, String projectPathRelativeToGitRepoRoot)
/Users/ryan/.nuget/packages/nerdbank.gitversioning/3.1.71/build/Nerdbank.GitVersioning.targets(71,5): error MSB4018:    at Nerdbank.GitVersioning.Tasks.GetBuildVersion.ExecuteInner()

[ErikSchierboom]

Just wanted to let you know that I have the same issue :)

Yes, this should be built into Locator. The MSBuild-NuGet relationship is only getting closer over time; we should treat it as part of MSBuild in this context.

Looking forward to having this built-in!

[xsacha]

As a workaround, I'm using global.json with:

{
  "sdk": {
    "version": "3.1.100"
  }
}

[zivkan]

For what it's worth, this is my workaround:

var instance = Microsoft.Build.Locator.MSBuildLocator.RegisterDefaults();
AssemblyLoadContext.Default.Resolving += (assemblyLoadContext, assemblyName) =>
{
    var path = Path.Combine(instance.MSBuildPath, assemblyName.Name + ".dll");
    if (File.Exists(path))
    {
        return assemblyLoadContext.LoadFromAssemblyPath(path);
    }

    return null;
};

[AArnott]

@rynowak @rainersigwald The Microsoft.DotNet.PlatformAbstractions.dll is supposed to ship with nerdbank.gitversioning.dll itself, but didn't because msbuild used to include it I guess and doesn't any more. Anyway, this was tracked by dotnet/Nerdbank.GitVersioning#466 and is fixed now.

[Gambero81]

For what it's worth, this is my workaround:

var instance = Microsoft.Build.Locator.MSBuildLocator.RegisterDefaults();
AssemblyLoadContext.Default.Resolving += (assemblyLoadContext, assemblyName) =>
{
    var path = Path.Combine(instance.MSBuildPath, assemblyName.Name + ".dll");
    if (File.Exists(path))
    {
        return assemblyLoadContext.LoadFromAssemblyPath(path);
    }

    return null;
};

@zivkan i tried this workaround without success..

C:\Program Files\dotnet\sdk\3.1.401\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets: (59, 5): errore imprevisto dell'attività "ProcessFrameworkReferences".
System.IO.FileNotFoundException: Could not load file or assembly 'NuGet.Frameworks, Version=5.7.0.7

In the "C:\Program Files\dotnet\sdk\3.1.401" folder i have NuGet.Frameworks.dll version 5.7.0.6 but it search for 5.7.0.7!

[ricardo-espinoza]

We encountered the same issue in a component of the Microsoft Quantum Development Kit.
I also followed @zivkan 's workaround as shown in the pull request below:
microsoft/qsharp-compiler#566

[CAlexGonzalez]

For what it's worth, this is my workaround:

var instance = Microsoft.Build.Locator.MSBuildLocator.RegisterDefaults();
AssemblyLoadContext.Default.Resolving += (assemblyLoadContext, assemblyName) =>
{
    var path = Path.Combine(instance.MSBuildPath, assemblyName.Name + ".dll");
    if (File.Exists(path))
    {
        return assemblyLoadContext.LoadFromAssemblyPath(path);
    }

    return null;
};

I got the same issue:
C:\Program Files\dotnet\sdk\5.0.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(63,5): error MSB4018: The "ProcessFrameworkReferences" task failed unexpectedly.
C:\Program Files\dotnet\sdk\5.0.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(63,5): error MSB4018: System.IO.FileNotFoundException: Could not load file or assembly 'NuGet.Frameworks, Version=5.8.0.6930, Culture=neutral, PublicKeyToken=31bf3856ad364e35'. The system cannot find the file specified.
C:\Program Files\dotnet\sdk\5.0.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(63,5): error MSB4018: File name: 'NuGet.Frameworks, Version=5.8.0.6930, Culture=neutral, PublicKeyToken=31bf3856ad364e35'
C:\Program Files\dotnet\sdk\5.0.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(63,5): error MSB4018: at Microsoft.NET.Build.Tasks.ProcessFrameworkReferences.ExecuteCore()
C:\Program Files\dotnet\sdk\5.0.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(63,5): error MSB4018: at Microsoft.NET.Build.Tasks.TaskBase.Execute()
C:\Program Files\dotnet\sdk\5.0.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(63,5): error MSB4018: at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
C:\Program Files\dotnet\sdk\5.0.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(63,5): error MSB4018: at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) 0 Warning(s)
1 Error(s)

Thanks Andy, your workaround also solved my issue.

[BjarkeCK]

A heads up to anyone maybe struggling to get the workaround working.

static void Main(string[] args)
{
    var instance = MSBuildLocator.RegisterDefaults();
    AssemblyLoadContext.Default.Resolving += (assemblyLoadContext, assemblyName) =>
    {
        var path = Path.Combine(instance.MSBuildPath, assemblyName.Name + ".dll");
        if (File.Exists(path))
        {
            return assemblyLoadContext.LoadFromAssemblyPath(path);
        }

        return null;
    };

    Console.WriteLine(BuildManager.DefaultBuildManager); // This will prevent it from compiling.
}

That fails to compile, and gives me the following error:

Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
File name: 'Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
at WatchRun.Program.Main(String[] args)

But if you simply make sure the only thing you do in main is to register the resolver, and put everything else in a nother method. Then it will compile and the workaround works.

static void Main(string[] args)
{
    var instance = MSBuildLocator.RegisterDefaults();
    AssemblyLoadContext.Default.Resolving += (assemblyLoadContext, assemblyName) =>
    {
        var path = Path.Combine(instance.MSBuildPath, assemblyName.Name + ".dll");
        if (File.Exists(path))
        {
            return assemblyLoadContext.LoadFromAssemblyPath(path);
        }

        return null;
    };

    NowItWorks();
}

private static void NowItWorks()
{
    Console.WriteLine(BuildManager.DefaultBuildManager);
}