Plan for addressing torn SDK builds #41790
Conversation
This is a _very rough_ draft
Co-authored-by: Fred Silberberg <fred@silberberg.xyz>
Co-authored-by: Rainer Sigwald <raines@microsoft.com>
Co-authored-by: Jan Jones <jan.jones.cz@gmail.com>
dotnet-issue-labeler
bot
added
Area-Infrastructure
untriaged
Co-authored-by: Chris Sienkiewicz <chsienki@microsoft.com>
Co-authored-by: Sam Harwell <sam@tunnelvisionlabs.com>
| 1. The Windows .NET SDK increased in size even when it's not needed. For example our main Windows containers do not install Visual Studio hence are never in a torn state yet they'll see increased size cost here. Changing the .NET SDK such that these extra compilers are only installed sometimes would add significant complexity to our installer setup. | ||
| 2. This will make the Windows and Linux .NET SDK's visibly different in layout. Comparing them will show different contents which could be confusing to customers. At the same time the current proposal is functionally the same, it just changes where the framework compilers are placed on disk and that also could be confusing to customers. | ||
|
|
||
| ### Use the .NET Core compiler |
There was a problem hiding this comment.
I was hoping that we would explore this option as it would reduce of number of compilers and make the inbox assets in the .NET SDK the only driver. IIRC, other compiler teams in the .NET ecosystem actively consider going down that route.
I understand that there are drawbacks but to me this feels like the right option, long term.
There was a problem hiding this comment.
Our source generators that ship inbox as part of the Microsoft.NETCore.App.Ref targeting pack need to target netstandard2.0 because we don't know which compiler loads them (the .NET Framework one vs the .NETCoreApp one). This is really weird as the API surface area is very limited and "Bcl" packages need to be referenced.
It would really be great if our inbox source generators could just target the TFM that the .NETCoreApp compiler targets (i.e. net8.0 / net9.0).
There was a problem hiding this comment.
It would really be great if our inbox source generators could just target the TFM that the .NETCoreApp compiler targets (i.e. net8.0 / net9.0).
This option only says that the .NET Core compiler would be used if you are in a torn state and run msbuild. But your source generators would still need to be able to run in Visual Studio and hence target netstandard2.0, right?
I understand that there are drawbacks but to me this feels like the right option, long term.
This makes sense to me. There are drawbacks to the currently selected options as well, but we say, "well if you are in a torn state and don't like the drawbacks, get out of the torn state", maybe we could say that here as well. Furthermore, couldn't we make the .NET Core compiler when run in the torn state also run in a process named VBCSCompiler somehow? I didn't really understand the servicing lifetime drawback, doesn't it apply to the currently selected option as well?
There was a problem hiding this comment.
This option only says that the .NET Core compiler would be used if you are in a torn state and run msbuild. But your source generators would still need to be able to run in Visual Studio and hence target netstandard2.0, right?
VS already carries a .NET runtime and SDK. Why not always depend on the .NETCoreApp version of the compiler, even inside VS? The .NETCoreApp variant might even be significantly faster (?).
There was a problem hiding this comment.
I understand that there are drawbacks but to me this feels like the right option, long term.
Agree long term the ideal solution is use a .NET Core based compiler everywhere but we are a very long way away from that. Consider that for the compiler the following big rocks exist:
- The significant servicing lifetime differences between VS and the .NET SDK (5x5 vs. 3x3). This is the most unmovable rock.
- The lack of apphost files in the .NET SDK. There are many, many build scripts and infra that depend on the names of processes.
- The. NET SDK servicing lifetime poses intenal challenges for teams that want very long serviced toolsets.
Our source generators that ship inbox as part of the Microsoft.NETCore.App.Ref targeting pack need to target netstandard2.0 because we don't know which compiler loads them
Source generators need to be consumable in every compiler host, not just csc, VBCSCompiler, etc ... Even if csc moved to .NET core that wouldn't change the requirements for source generators to target netstandard2.0. Consider for example that generators can still be loaded in VS which is a .NET Framework process. There are many other compiler hosts out there which are .NET Framework based (consider that source indexer is a .NET Framework compiler host).
Source generators will be the absolute last component to move off of netstandard2.0 in the .NET SDK. 😦
VS already carries a .NET runtime and SDK. Why not always depend on the .NETCoreApp version of the compiler, even inside VS?
True that VS carries a .NET Runtime but there are hurdles to using it:
- How do we find the .NET runtime to launch child processes? Consider specifically that we mus continue to support launching
csc.exeas a standalone binary from the msbuild install. That would need to discover and use the VS private runtime. App hosts don't support that kind of look up. - There are no guarantees about what version of the .NET runtime VS will / won't use. It's possible they could stay on .NET 8 for quite some time. That would mean that for however long that was we must continue supporting building roslyn targeting that TFM.
- etc ...
This is probably the most fixable item though. Had discussions with VS team a few years ago about this and we think we had solutions for all these problems (and others I didn't list). But it is a significant investment across a number of teams to make that happen.
There was a problem hiding this comment.
I'll note that AppHost don't support this kind of private runtime lookup yet. The runtime is adding a feature to 9 (PR is out now) that would allow telling an apphosts where to look up dotnet from, which could reasonably be used to make a 'VS-private-runtime-aware' AppHost.
There was a problem hiding this comment.
Where is this PR? Very interested in that because it has implications for other work we want to look at.
There was a problem hiding this comment.
#42455, which links to the original runtime issue and the dotnet/designs spec.
Co-authored-by: Jan Jones <jan.jones.cz@gmail.com>
| 1. The Windows .NET SDK increased in size even when it's not needed. For example our main Windows containers do not install Visual Studio hence are never in a torn state yet they'll see increased size cost here. Changing the .NET SDK such that these extra compilers are only installed sometimes would add significant complexity to our installer setup. | ||
| 2. This will make the Windows and Linux .NET SDK's visibly different in layout. Comparing them will show different contents which could be confusing to customers. At the same time the current proposal is functionally the same, it just changes where the framework compilers are placed on disk and that also could be confusing to customers. | ||
|
|
||
| ### Use the .NET Core compiler |
There was a problem hiding this comment.
I'll note that AppHost don't support this kind of private runtime lookup yet. The runtime is adding a feature to 9 (PR is out now) that would allow telling an apphosts where to look up dotnet from, which could reasonably be used to make a 'VS-private-runtime-aware' AppHost.
This describes the approach we're considering for addressing our torn .NET SDK build problem.