Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[VMR] Build failure with portable RID (e.g. /p:TargetRid=linux-x64) #3131

Closed
directhex opened this issue Nov 23, 2022 · 11 comments
Closed

[VMR] Build failure with portable RID (e.g. /p:TargetRid=linux-x64) #3131

directhex opened this issue Nov 23, 2022 · 11 comments
Labels
area-build Improvements in source-build's own build process

Comments

@directhex
Copy link

I'm still working on the exact root cause, but I wanted to have a written record somewhere other than internal Teams.

Building for a generic, portable RID fails currently with the VMR, in aspnetcore, citing that it can't find the crossgen2 binary. Looking at the build, and comparing to a "good" ubuntu.22.04-x64 build, it seems the crossgen2 nupkg is created as expected during the runtime build, but never restored in aspnetcore.

It appears the build explicitly rejects most of the package references from earlier in the build, in fact, citing that they do not need to be explicitly depended on because they're part of the SDK.

This also blocks building an osx-x64 RID build on Mac.
@dotnet-issue-labeler dotnet-issue-labeler bot added area-build Improvements in source-build's own build process untriaged labels Nov 23, 2022
@directhex
Copy link
Author

directhex commented Nov 23, 2022 •
edited
Loading 

    /home/directhex/Projects/dotnet/.dotnet/sdk/7.0.100-rc.2.22477.23/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.DefaultItems.Shared.targets(152,5): error NETSDK1023: A PackageReference for 'Microsoft.NETCore.App.Runtime.linux-x64' was included in your project. This package is implicitly referenced by the .NET SDK and you do not typically need to reference it from your project. For more information, see https://aka.ms/sdkimplicitrefs [/home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj]
    /home/directhex/Projects/dotnet/.dotnet/sdk/7.0.100-rc.2.22477.23/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.DefaultItems.Shared.targets(152,5): error NETSDK1023: A PackageReference for 'Microsoft.NETCore.App.Runtime.linux-x64' was included in your project. This package is implicitly referenced by the .NET SDK and you do not typically need to reference it from your project. For more information, see https://aka.ms/sdkimplicitrefs [/home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj]
    /home/directhex/Projects/dotnet/.dotnet/sdk/7.0.100-rc.2.22477.23/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.DefaultItems.Shared.targets(152,5): error NETSDK1023: A PackageReference for 'Microsoft.NETCore.App.Runtime.linux-x64' was included in your project. This package is implicitly referenced by the .NET SDK and you do not typically need to reference it from your project. For more information, see https://aka.ms/sdkimplicitrefs [/home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj]
    /home/directhex/Projects/dotnet/.dotnet/sdk/7.0.100-rc.2.22477.23/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.DefaultItems.Shared.targets(152,5): error NETSDK1023: A PackageReference for 'Microsoft.NETCore.App.Runtime.linux-x64' was included in your project. This package is implicitly referenced by the .NET SDK and you do not typically need to reference it from your project. For more information, see https://aka.ms/sdkimplicitrefs [/home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj]
    /home/directhex/Projects/dotnet/.dotnet/sdk/7.0.100-rc.2.22477.23/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.DefaultItems.Shared.targets(152,5): error NETSDK1023: A PackageReference for 'Microsoft.NETCore.App.Crossgen2.linux-x64' was included in your project. This package is implicitly referenced by the .NET SDK and you do not typically need to reference it from your project. For more information, see https://aka.ms/sdkimplicitrefs [/home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj]
    /home/directhex/Projects/dotnet/.dotnet/sdk/7.0.100-rc.2.22477.23/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.DefaultItems.Shared.targets(152,5): error NETSDK1023: A PackageReference for 'Microsoft.NETCore.App.Crossgen2.linux-x64' was included in your project. This package is implicitly referenced by the .NET SDK and you do not typically need to reference it from your project. For more information, see https://aka.ms/sdkimplicitrefs [/home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj]
    /home/directhex/Projects/dotnet/.dotnet/sdk/7.0.100-rc.2.22477.23/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.DefaultItems.Shared.targets(152,5): error NETSDK1023: A PackageReference for 'Microsoft.NETCore.App.Crossgen2.linux-x64' was included in your project. This package is implicitly referenced by the .NET SDK and you do not typically need to reference it from your project. For more information, see https://aka.ms/sdkimplicitrefs [/home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj]
    /home/directhex/Projects/dotnet/.dotnet/sdk/7.0.100-rc.2.22477.23/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.DefaultItems.Shared.targets(152,5): error NETSDK1023: A PackageReference for 'Microsoft.NETCore.App.Crossgen2.linux-x64' was included in your project. This package is implicitly referenced by the .NET SDK and you do not typically need to reference it from your project. For more information, see https://aka.ms/sdkimplicitrefs [/home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj]
    /home/directhex/Projects/dotnet/.dotnet/sdk/7.0.100-rc.2.22477.23/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.DefaultItems.Shared.targets(152,5): error NETSDK1023: A PackageReference for 'Microsoft.NETCore.App.Runtime.linux-x64' was included in your project. This package is implicitly referenced by the .NET SDK and you do not typically need to reference it from your project. For more information, see https://aka.ms/sdkimplicitrefs [/home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj]
    /home/directhex/Projects/dotnet/.dotnet/sdk/7.0.100-rc.2.22477.23/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.DefaultItems.Shared.targets(152,5): error NETSDK1023: A PackageReference for 'Microsoft.NETCore.App.Runtime.linux-x64' was included in your project. This package is implicitly referenced by the .NET SDK and you do not typically need to reference it from your project. For more information, see https://aka.ms/sdkimplicitrefs [/home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj]
    /home/directhex/Projects/dotnet/.dotnet/sdk/7.0.100-rc.2.22477.23/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.DefaultItems.Shared.targets(152,5): error NETSDK1023: A PackageReference for 'Microsoft.NETCore.App.Runtime.linux-x64' was included in your project. This package is implicitly referenced by the .NET SDK and you do not typically need to reference it from your project. For more information, see https://aka.ms/sdkimplicitrefs [/home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj]
    /home/directhex/Projects/dotnet/.dotnet/sdk/7.0.100-rc.2.22477.23/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.DefaultItems.Shared.targets(152,5): error NETSDK1023: A PackageReference for 'Microsoft.NETCore.App.Runtime.linux-x64' was included in your project. This package is implicitly referenced by the .NET SDK and you do not typically need to reference it from your project. For more information, see https://aka.ms/sdkimplicitrefs [/home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj]
    /home/directhex/Projects/dotnet/.dotnet/sdk/7.0.100-rc.2.22477.23/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.DefaultItems.Shared.targets(152,5): error NETSDK1023: A PackageReference for 'Microsoft.NETCore.App.Crossgen2.linux-x64' was included in your project. This package is implicitly referenced by the .NET SDK and you do not typically need to reference it from your project. For more information, see https://aka.ms/sdkimplicitrefs [/home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj]
    /home/directhex/Projects/dotnet/.dotnet/sdk/7.0.100-rc.2.22477.23/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.DefaultItems.Shared.targets(152,5): error NETSDK1023: A PackageReference for 'Microsoft.NETCore.App.Crossgen2.linux-x64' was included in your project. This package is implicitly referenced by the .NET SDK and you do not typically need to reference it from your project. For more information, see https://aka.ms/sdkimplicitrefs [/home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj]
    /home/directhex/Projects/dotnet/.dotnet/sdk/7.0.100-rc.2.22477.23/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.DefaultItems.Shared.targets(152,5): error NETSDK1023: A PackageReference for 'Microsoft.NETCore.App.Crossgen2.linux-x64' was included in your project. This package is implicitly referenced by the .NET SDK and you do not typically need to reference it from your project. For more information, see https://aka.ms/sdkimplicitrefs [/home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj]
    /home/directhex/Projects/dotnet/.dotnet/sdk/7.0.100-rc.2.22477.23/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.DefaultItems.Shared.targets(152,5): error NETSDK1023: A PackageReference for 'Microsoft.NETCore.App.Crossgen2.linux-x64' was included in your project. This package is implicitly referenced by the .NET SDK and you do not typically need to reference it from your project. For more information, see https://aka.ms/sdkimplicitrefs [/home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj]
    /home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj(434,5): error : Could not find crossgen2 tools/crossgen2

@directhex
Copy link
Author

Adding DisableImplicitFrameworkReferences to Microsoft.AspNetCore.App.Runtime.csproj reduces the error count from 17 to 13, which is not very meaningful or useful :|

@omajid
Copy link
Member

omajid commented Nov 30, 2022

cc @tmds

@tmds
Copy link
Member

tmds commented Nov 30, 2022

There are some changes on aspnetcore 7.0 branch for eliminating the portable build.

The PRs against the installer repo haven't been backported yet (dotnet/installer#14549 and dotnet/installer#14938). Backporting those may fix this issue.

cc @crummel @MichaelSimons

@directhex directhex changed the title Build failure with portable RID (e.g. /p:TargetRid=linux-x64) [VMR] Build failure with portable RID (e.g. /p:TargetRid=linux-x64) Nov 30, 2022
@directhex
Copy link
Author

Funny story, I think the 7.0 aspnetcore changes (specifically dotnet/aspnetcore#44978) are the problem, not the cure.

Dependencies.props includes a hardcoded dependency on Microsoft.NETCore.App.Runtime.linux-x64 et al, and also includes a dependency on Microsoft.NETCore.App.Runtime.$(TargetRuntimeIdentifier) which (when building on Mac, or on Linux with /p:TargetRid=linux-x64) resolves to a duplicate dependency. This causes the CheckForImplicitPackageReferenceOverrides task to freak out, remove both copies of the reference, and fail the build.

Locally, removing one of the duplicated dependencies progresses the build further. Without the duplicate dependency, DisableImplicitFrameworkReferences has no impact at all either enabled or disabled, so it turns out this was a bad area of investigation.

@directhex
Copy link
Author

directhex commented Dec 1, 2022 •
edited
Loading

The aspnetcore build still fails, mind you, but later in the build:

      Unhandled exception. System.CommandLine.CommandLineException: Multiple input files matching same simple name /home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/artifacts/bin/Microsoft.AspNetCore.App.Runtime/Release/net7.0/linux-x64/System.Formats.Asn1.dll /home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/package-cache/microsoft.netcore.app.runtime.linux-x64/8.0.0-alpha.1.22559.2/runtimes/linux-x64/lib/net7.0/System.Formats.Asn1.dll
         at System.CommandLine.Helpers.AppendExpandedPaths(Dictionary`2 dictionary, String pattern, Boolean strict)
         at System.CommandLine.Helpers.BuildPathDictionary(IReadOnlyList`1 tokens, Boolean strict)
         at System.CommandLine.Argument`1.<>c__DisplayClass5_0.<.ctor>b__1(ArgumentResult argumentResult, Object& value)
         at System.CommandLine.Parsing.ArgumentResult.Convert(Argument argument)
         at System.CommandLine.Parsing.ArgumentResult.GetArgumentConversionResult()
         at System.CommandLine.Parsing.ParseResultVisitor.ValidateAndConvertArgumentResult(ArgumentResult argumentResult)
         at System.CommandLine.Parsing.ParseResultVisitor.ValidateAndConvertArgumentResults(IReadOnlyList`1 arguments, Int32 commandArgumentResultCount)
         at System.CommandLine.Parsing.ParseResultVisitor.Stop()
         at System.CommandLine.Parsing.Parser.Parse(IReadOnlyList`1 arguments, String rawInput)
         at ILCompiler.Program.Main(String[] args)
      Aborted (core dumped)
    /home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj(496,5): error MSB3073: The command ""/home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/package-cache/microsoft.netcore.app.crossgen2.linux-x64/8.0.0-alpha.1.22559.2/tools/crossgen2" --composite --targetarch:x64 --targetos:linux /home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/package-cache/microsoft.netcore.app.runtime.linux-x64/8.0.0-alpha.1.22559.2/runtimes/linux-x64/lib/net7.0/*.dll /home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/artifacts/bin/Microsoft.AspNetCore.App.Runtime/Release/net7.0/linux-x64/*.dll --out:"/home/directhex/Projects/dotnet/src/aspnetcore/artifacts/source-build/self/src/artifacts/obj/Composite/Release/full-composite.r2r.dll"" exited with code 134.

@directhex
Copy link
Author

directhex commented Dec 1, 2022 •
edited
Loading

Well that's fascinating. In my non-portable build, those three assemblies exist only in microsoft.netcore.app.runtime.ubuntu.22.04-x64 but in the portable build, they're in both microsoft.netcore.app.runtime.linux-x64 and artifacts/bin/Microsoft.AspNetCore.App.Runtime

@directhex
Copy link
Author

OK, no, the 2-assembly thing is a weird problem with a dirty build. Doing a 100% clean build, this is enough to make the build work:

index 6c44830a0..904a23489 100644
--- a/src/aspnetcore/eng/Dependencies.props
+++ b/src/aspnetcore/eng/Dependencies.props
@@ -100,7 +100,6 @@ and are generated based on the last package release.
     <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.win-arm64" />
     <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.osx-x64" />
     <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.osx-arm64" />
-    <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.linux-x64" />
     <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.linux-arm" />
     <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.linux-arm64" />
     <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.linux-musl-x64" />
@@ -117,7 +116,6 @@ and are generated based on the last package release.
     <LatestPackageReference Include="Microsoft.NETCore.App.Crossgen2.linux-musl-arm" />
     <LatestPackageReference Include="Microsoft.NETCore.App.Crossgen2.linux-musl-arm64" />
     <LatestPackageReference Include="Microsoft.NETCore.App.Crossgen2.linux-musl-x64" />
-    <LatestPackageReference Include="Microsoft.NETCore.App.Crossgen2.linux-x64" />
     <LatestPackageReference Include="Microsoft.NETCore.App.Crossgen2.win-x64" />
     <LatestPackageReference Include="Microsoft.NETCore.App.Crossgen2.win-x86" />
     <LatestPackageReference Include="Microsoft.NETCore.App.Crossgen2.win-arm" />

@directhex
Copy link
Author

Got a Mac build. I'll post patches & file issues tomorrow morning, since changes are needed.

@directhex
Copy link
Author

I'm going to close this. I've split the issues I encountered into separate issues.

#3149

#3150

#3151

@corngood
Copy link

corngood commented Apr 1, 2024 •
edited
Loading

I'm still getting this problem on 9.0.0-preview.2 unless I make the same change described in #3131 (comment) (removing the duplicate references).

I also had a similar problem in 8.0 releases, which I described here: dotnet/installer#15163 (comment). I should have probably posted that comment on this issue instead.

My fix from 8.0 no longer applies, but I verified that aspnetcore is building with /p:PortableBuild=true /p:TargetRuntimeIdentifier=linux-x64, so I'm not sure why it's failing again. I'll try to find out.

@corngood corngood mentioned this issue Apr 1, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area-build Improvements in source-build's own build process
Projects
.NET Docker
Status: Done
Milestone
No milestone
Development

No branches or pull requests
5 participants
@omajid @directhex @tmds @corngood @MichaelSimons