Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build fails with "eng/common/tools.sh: line 474: 537 Segmentation fault" #76759

Closed
jkotas opened this issue Oct 7, 2022 · 10 comments
Closed
Labels
area-VM-coreclr Known Build Error Use this to report build issues in the .NET Helix tab
Milestone

Comments

@jkotas
Copy link
Member

jkotas commented Oct 7, 2022

2022-10-07T15:14:51.2484448Z Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli
2022-10-07T15:14:51.2485788Z --------------------------------------------------------------------------------------
2022-10-07T15:14:51.6083403Z /__w/1/s/.dotnet/sdk/7.0.100-rc.1.22431.12/MSBuild.dll /nologo -maxcpucount /m -verbosity:m /v:minimal /bl:/__w/1/s/artifacts/log/Release/ToolsetRestore.binlog /clp:Summary /clp:ErrorsOnly;NoSummary /nr:false /p:TreatWarningsAsErrors=true /p:ContinuousIntegrationBuild=true /p:__ToolsetLocationOutputFile=/__w/1/s/artifacts/toolset/8.0.0-beta.22503.1.txt /t:__WriteToolsetLocation /warnaserror /__w/1/s/artifacts/toolset/restore.proj
2022-10-07T15:14:55.6156152Z /__w/1/s/eng/common/tools.sh: line 474:   537 Segmentation fault      (core dumped) "$_InitializeBuildTool" "$@"
2022-10-07T15:14:55.6156978Z Build failed with exit code 139. Check errors above.

Failed in #76737. Full log: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_apis/build/builds/44249/logs/166

{
  "ErrorPattern": "eng/common/tools.sh: line [0-9]+:   [0-9]+ Segmentation fault",
  "BuildRetry": true
}

Report

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 0

Known issue validation

Build: 🔎
Result validation: ⚠️ Validation could not be done without an Azure DevOps build URL on the issue. Please add it to the "Build: 🔎" line.

@jkotas jkotas added area-Infrastructure Known Build Error Use this to report build issues in the .NET Helix tab labels Oct 7, 2022
@ghost
Copy link

ghost commented Oct 7, 2022

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details
2022-10-07T15:14:51.2484448Z Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli
2022-10-07T15:14:51.2485788Z --------------------------------------------------------------------------------------
2022-10-07T15:14:51.6083403Z /__w/1/s/.dotnet/sdk/7.0.100-rc.1.22431.12/MSBuild.dll /nologo -maxcpucount /m -verbosity:m /v:minimal /bl:/__w/1/s/artifacts/log/Release/ToolsetRestore.binlog /clp:Summary /clp:ErrorsOnly;NoSummary /nr:false /p:TreatWarningsAsErrors=true /p:ContinuousIntegrationBuild=true /p:__ToolsetLocationOutputFile=/__w/1/s/artifacts/toolset/8.0.0-beta.22503.1.txt /t:__WriteToolsetLocation /warnaserror /__w/1/s/artifacts/toolset/restore.proj
2022-10-07T15:14:55.6156152Z /__w/1/s/eng/common/tools.sh: line 474:   537 Segmentation fault      (core dumped) "$_InitializeBuildTool" "$@"
2022-10-07T15:14:55.6156978Z Build failed with exit code 139. Check errors above.

Failed in #76737. Full log: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_apis/build/builds/44249/logs/166

{
  "ErrorMessage": "eng/common/tools.sh: line 474:   537 Segmentation fault",
  "BuildRetry": true
}
Author: jkotas
Assignees: -
Labels:

area-Infrastructure, Known Build Error

Milestone: -

@ghost ghost added the untriaged New issue has not been triaged by the area owner label Oct 7, 2022
@EgorBo
Copy link
Member

EgorBo commented Oct 7, 2022

I hit the same issue in #76112 yesterday, disappeared after re-run

@janvorli
Copy link
Member

I've hit the same issue in #76972 and noticed it in another PR few days ago.

@janvorli
Copy link
Member

In both the case @jkotas referenced and in my PR, the failure was in arm64 cross build of native aot. Not sure if that's just a coincidence though:
/__w/1/s/build.sh -ci -arch arm64 -os Linux -s clr.aot+libs+libs.tests -c Release /p:TestNativeAot=true /p:RunSmokeTestsOnly=true /p:ArchiveTests=true /p:CrossBuild=true /p:MonoLLVMUseCxx11Abi=true

@MichalStrehovsky
Copy link
Member

In both the case @jkotas referenced and in my PR, the failure was in arm64 cross build of native aot. Not sure if that's just a coincidence though: /__w/1/s/build.sh -ci -arch arm64 -os Linux -s clr.aot+libs+libs.tests -c Release /p:TestNativeAot=true /p:RunSmokeTestsOnly=true /p:ArchiveTests=true /p:CrossBuild=true /p:MonoLLVMUseCxx11Abi=true

Coincidence.

This one is failing while restoring host+packs: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_apis/build/builds/118290/logs/995

2022-12-22T01:40:48.7387119Z /__w/1/s/build.sh -subset host+packs -ci  -arch arm /p:CrossBuild=true /p:PortableBuild=true /p:SkipTests=True  /p:RuntimeArtifactsPath=/__w/1/s/artifacts/transport/coreclr /p:RuntimeConfiguration=release  /p:LibrariesConfiguration=Release /p:Configuration=Debug /p:TargetOS=Linux /p:TargetArchitecture=arm /p:RuntimeFlavor=coreclr 
2022-12-22T01:40:48.7388339Z ========================== Starting Command Output ===========================
2022-12-22T01:40:48.7433628Z [command]/bin/bash --noprofile --norc /__w/_temp/b3185808-2639-4823-907b-8265987e8dd7.sh
2022-12-22T01:40:48.8261028Z __DistroRid: linux-musl-arm
2022-12-22T01:40:48.9440350Z Downloading 'https://dotnet.microsoft.com/download/dotnet/scripts/v1/dotnet-install.sh'
2022-12-22T01:40:49.0004221Z Attempting to install dotnet from public_location.
2022-12-22T01:40:49.0104471Z dotnet-install: Note that the intended use of this script is for Continuous Integration (CI) scenarios, where:
2022-12-22T01:40:49.0107504Z dotnet-install: - The SDK needs to be installed without user interaction and without admin rights.
2022-12-22T01:40:49.0109451Z dotnet-install: - The SDK installation doesn't need to persist across multiple CI runs.
2022-12-22T01:40:49.0112241Z dotnet-install: To set up a development environment or to run apps, use installers rather than this script. Visit https://dotnet.microsoft.com/download to get the installer.
2022-12-22T01:40:49.0113208Z 
2022-12-22T01:40:49.4297716Z dotnet-install: Attempting to download using primary link https://dotnetcli.azureedge.net/dotnet/Sdk/7.0.100/dotnet-sdk-7.0.100-linux-x64.tar.gz
2022-12-22T01:40:50.7548065Z dotnet-install: Extracting zip from https://dotnetcli.azureedge.net/dotnet/Sdk/7.0.100/dotnet-sdk-7.0.100-linux-x64.tar.gz
[REDACTED FOR BREVITY]
2022-12-22T01:40:57.3692043Z /__w/1/s/.dotnet/sdk/7.0.100/MSBuild.dll /nologo -maxcpucount /m -verbosity:m /v:minimal /bl:/__w/1/s/artifacts/log/Debug/ToolsetRestore.binlog /clp:Summary /clp:ErrorsOnly;NoSummary /nr:false /p:TreatWarningsAsErrors=true /p:ContinuousIntegrationBuild=true /p:__ToolsetLocationOutputFile=/__w/1/s/artifacts/toolset/8.0.0-beta.22606.1.txt /t:__WriteToolsetLocation /warnaserror /__w/1/s/artifacts/toolset/restore.proj
2022-12-22T01:40:59.5418606Z /__w/1/s/eng/common/tools.sh: line 467:   637 Segmentation fault      (core dumped) "$_InitializeBuildTool" "$@"

@agocke
Copy link
Member

agocke commented Aug 17, 2023

Changing to area-CoreCLR. This is likely a product issue, not an infra issue.

@jkotas
Copy link
Member Author

jkotas commented Aug 25, 2023

Closing. This stopped occurring after updating .NET SDK LKG to .NET 8 P7.

@jkotas jkotas closed this as completed Aug 25, 2023
@ghost ghost removed the untriaged New issue has not been triaged by the area owner label Aug 25, 2023
@jkotas jkotas reopened this Sep 5, 2023
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Sep 5, 2023
@jkotas
Copy link
Member Author

jkotas commented Sep 5, 2023

Happened again in #91554 on osx-x64 release Runtime_Debug build

Full log https://dev.azure.com/dnceng-public/public/_build/results?buildId=395071&view=logs&jobId=74a332a6-4c62-563e-0293-081993a1bb75&j=74a332a6-4c62-563e-0293-081993a1bb75&t=26d9c75b-b006-57b7-96be-abd7e0e03816

...
  System.Security.Cryptography.Pkcs -> /Users/runner/work/1/s/artifacts/bin/System.Security.Cryptography.Pkcs/ref/Release/net9.0/System.Security.Cryptography.Pkcs.dll
  System.ComponentModel.Composition -> /Users/runner/work/1/s/artifacts/bin/System.ComponentModel.Composition/ref/Release/net9.0/System.ComponentModel.Composition.dll
  System.Composition.Runtime -> /Users/runner/work/1/s/artifacts/bin/System.Composition.Runtime/Release/net9.0/System.Composition.Runtime.dll
/Users/runner/work/1/s/eng/common/tools.sh: line 498:  2233 Segmentation fault: 11  "$_InitializeBuildTool" "$@"
Build failed with exit code 139. Check errors above.

@mangod9
Copy link
Member

mangod9 commented Sep 5, 2023

Looks like there is overall flakiness on macOS x64 recently with many different failure modes.

@mangod9 mangod9 removed the untriaged New issue has not been triaged by the area owner label Sep 5, 2023
@mangod9 mangod9 added this to the 9.0.0 milestone Sep 5, 2023
@build-analysis build-analysis bot removed this from the 9.0.0 milestone Nov 15, 2023
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Nov 15, 2023
@akoeplinger akoeplinger added this to the 9.0.0 milestone Nov 24, 2023
@ghost ghost removed the untriaged New issue has not been triaged by the area owner label Nov 24, 2023
@mangod9
Copy link
Member

mangod9 commented Jul 8, 2024

@jkotas guessing this is ok to close again if this hasnt occurred recently?

@jkotas jkotas closed this as completed Jul 8, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Aug 9, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-VM-coreclr Known Build Error Use this to report build issues in the .NET Helix tab
Projects
Archived in project
Development

No branches or pull requests

7 participants