Multi-target compiler to netstandard2.0 and netcoreapp3.1

[jaredpar]

The compiler libraries should move to mult-target netstandard2.0 and netcoreapp3.1 where today they only target netstandard2.0. Adding a target for netcoreapp3.1 will provide some benefits to our code base:

1. Increase the accuracy of our nullable annotations. Our nullable annotations are not as effective as they could be because the entire .NET SDK is effectively oblivious. Further this means we have to duplicate common methods like Debug.Assert and string.IsNullOrEmpty as the flow attributes aren't present in netstandard2.0. This is a significant burden for us as we annotate our source tree.
2. Allows us to take advantage of .NET Core APIs that could help increase the performance of the compiler. For example our object pooling could be more effective on .NET Core.

This does have a few downsides:

1. It will add extra overhead to our build in the form of three new libraries. Given we currently build several hundred libraries, particularly when you consider satellite assemblies, this overhead is fairly minimal.
2. It will add some more rigor to our investigations. Today it's essentially safe to repro a customer bug on either .NET Desktop or Core (whatever is easier for the developer). As we begin to adopt .NET Core specific APIs though it's possible we will end up with runtime specific behavior. Unlikely this will manifest in more than a handful of bugs but it's an item we will need to be aware of.

This specifically includes the following libraries:

• Microsoft.CodeAnalysis
• Microsoft.CodeAnalysis.CSharp
• Microsoft.CodeAnalysis.VisualBasic
• Microsoft.Build.Tasks.CodeAnalysis

Check list:

• The netcoreapp target framework in the repository is unified to netcoreapp3.1. Today it is a mix of versions do to timing around when we adopted runtime features like Default Interface Methods.
• The nuspec file for the libraries is updated to have both the netstandard2.0 and netcoreapp3.1 targets.
• The Microsoft.Net.Compilers.Toolset package is updated to ship the netcoreapp3.1 binaries
  • Notify @nguerrera when this happens as it will likely cause a bit of work when merging into the SDK
• Understand the impact on the Shipped / Unshipped API files. The tool wasn't designed for multi-targeting and it's possible this will cause issues.

[tmat]

@jaredpar Shouldn't we start with switching the VBCSCompiler to run on .NET Core and keeping netstandard2.0 until that's done? Otherwise, who is gonna be actually using the netcoreapp3.1 assemblies? The build task does not load them.

[jaredpar]

@tmat VBCSCompiler is already mult-targeted to net472 and netcoreapp2.1 (as is csc and vbc).

[tmat]

But we are actually not running it on Windows using .NET Core, right?

[jaredpar]

@tmat

But we are actually not running it on Windows using .NET Core, right?

The compiler executables will use the same .NET runtime that the MSBuild process is using. It doesn't consider the underlying operating system. That means if you use dotnet build it will use the netcoreapp2.1 binaries irrespective of operating system.

[tmat]

I see. But when building from VS that never uses dotnet build, right? So this is only for command line build.

[jaredpar]

@tmat

But when building from VS that never uses dotnet build, right? So this is only for command line build.

Correct. That always goes through desktop MSBuild. Hence every tool, including the compiler, is the desktop version.

[tmat]

It's a bit worrisome that we would effectively have two different compilers once we start using .NET Core specific features and these compilers might potentially produce different results between dotnet build and build in VS (I guess this is [2] above). This would be mitigated if we run VBCSCompiler always on .NET Core.

There would still be difference between IDE analysis and build, but at least build would be consistent.

[jaredpar]

It's a bit worrisome that we would effectively have two different compilers once we start using .NET Core specific features and these compilers might potentially produce different results between dotnet build and build in VS (I guess this is [2] above).

Correct that is 2 above.

To be clear though there will be no language feature difference between .NET Desktop and .NET Core (same as today). The difference will be limited to implementation details, mostly around taking advantage of .NET Core APIs that provide better performance. Any language feature difference between .NET Desktop and .NET Core is a bug.

It's possible there will be observable compiler behavior differences between .NET Desktop and .NET Core but that is already true today. Concrete example is there are subtle differences in how analyzers load into the compiler between desktop and core. But that is true today so no change here.

This would be mitigated if we run VBCSCompiler always on .NET Core.

That is something we have looked into a few times but it's more involved than just flipping a switch. There are a couple of items that have to come along with that:

1. It's a breaking change. Today analyzers are intended to target .NET Standard but there are surely analyzers that target .NET Desktop out there. Moving to always be on .NET Core will break those analyzers. Hence it seems certain that there will need to be at least a transition period where developers can force the compiler back down to Desktop to make progress.
2. It's requires a lot of OptProf and RPS work. The Visual Studio infrastructure isn't currently setup to instrument and measure the performance of .NET Core applications. VBCSCompiler is presently a hot point in our measurements. Lots of prep work to allow .NET Core into that infrastructure has to happen before we can move the compiler over.
3. It will further complicate our setup because now we have two .NET Core versions that we have to be aware of: the one in the .NET SDK that we insert into and the one in the Visual Studio that we insert into. In general they should be the same but once we start getting into servicing then it gets a bit more complicated.

[tmat]

Yes, I am aware that it won't be a simple switch. I'm just suggesting we should at least start working on it asap since it will need work from other teams as well. At some point we would also want to run IDE services on .NET Core, which will have the same problems. #39988 tracks [1]. I think we should enable this requirement by default in the IDE with a possible off switch that we'll remove later on.

[jaredpar]

This work is complete now.