Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test failure readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd #93639

Closed
v-wenyuxu opened this issue Oct 18, 2023 · 3 comments
Closed

Test failure readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd #93639

v-wenyuxu opened this issue Oct 18, 2023 · 3 comments
Labels
arch-arm64 arch-x64 area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI blocking-clean-ci-optional Blocking optional rolling runs JitStress CLR JIT issues involving JIT internal stress modes os-linux Linux OS (any supported distro) os-mac-os-x macOS aka OSX os-windows
Milestone
9.0.0

Comments

@v-wenyuxu
Copy link

Failed in: runtime-coreclr jitstress 20231016.1

Failed tests:

coreclr linux arm Checked jitstress1 @ (Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux x64 Checked jitstress2 @ Ubuntu.2204.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux x64 Checked jitstress1 @ Ubuntu.2204.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux arm Checked jitstress2_tiered @ (Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux arm Checked jitstress1_tiered @ (Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux arm Checked jitstress2 @ (Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux arm64 Checked jitstress2 @ (Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux arm64 Checked jitstress1_tiered @ (Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux x64 Checked jitstress2_tiered @ Ubuntu.2204.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr osx x64 Checked jitstress2 @ OSX.1200.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux x64 Checked jitstress1_tiered @ Ubuntu.2204.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr osx x64 Checked jitstress2_tiered @ OSX.1200.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr osx x64 Checked jitstress1_tiered @ OSX.1200.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr osx x64 Checked jitstress1 @ OSX.1200.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr windows x64 Checked jitstress1 @ Windows.10.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr windows x64 Checked jitstress2 @ Windows.10.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr windows x64 Checked jitstress1_tiered @ Windows.10.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr windows x64 Checked jitstress2_tiered @ Windows.10.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr osx arm64 Checked jitstress2 @ OSX.1200.ARM64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr osx arm64 Checked jitstress2_tiered @ OSX.1200.ARM64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr osx arm64 Checked jitstress1 @ OSX.1200.ARM64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr osx arm64 Checked jitstress1_tiered @ OSX.1200.ARM64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr windows arm64 Checked jitstress1 @ Windows.11.Arm64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr windows arm64 Checked jitstress2_tiered @ Windows.11.Arm64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr windows arm64 Checked jitstress2 @ Windows.11.Arm64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr windows arm64 Checked jitstress1_tiered @ Windows.11.Arm64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux arm64 Checked jitstress2_tiered @ (Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd

Error message:

1 / 1 (100%, 1 failed): failed in 7392 msecs, exit code 134, expected 0: corerun /root/helix/work/correlation/crossgen2/crossgen2.dll @/root/helix/work/workitem/e/readytorun/coreroot_determinism/coreroot_determinism/seed1/CPAOT-ret.out/System.Private.CoreLib.dll.rsp
  Hit an illegal branch between funclets![createdump] Gathering state for process 59 corerun
  [createdump] Crashing thread 003b signal 6 (0006)
!! Assert failure(PID 59 [0x0000003b], Thread: 59 [0x003b]): Assertion failed 'tgtIG->igFuncIdx == jmpIG->igFuncIdx' in 'System.Xml.Linq.XHashtable`1[System.__Canon]:Add(System.__Canon):System.__Canon:this' during 'Generate code' (IL size 66; hash 0xa0e13e54; FullOpts)
!!     File: /__w/1/s/src/coreclr/jit/emit.cpp Line: 6540
!!     Image: /root/helix/work/correlation/corerun
  [createdump] Writing crash report to file /home/helixbot/dotnetbuild/dumps/coredump.59.dmp.crashreport.json
  [createdump] Crash report successfully written
  [createdump] Writing minidump with heap to file /home/helixbot/dotnetbuild/dumps/coredump.59.dmp
  [createdump] Written 192278528 bytes (46943 pages) to core file
  [createdump] Target process is alive
  [createdump] Dump successfully written in 310ms
!! waitpid() returned successfully (wstatus 00000000) WEXITSTATUS 0 WTERMSIG 0

Return code:      1
Raw output file:      /root/helix/work/workitem/uploads/coreroot_determinism/output.txt
Raw output:
BEGIN EXECUTION
/root/helix/work/correlation/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false -p System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true coreroot_determinism.dll ''
================================== Compiling with seed 1 ==================================
Locating output folder /root/helix/work/workitem/e/readytorun/coreroot_determinism/coreroot_determinism/seed1
Deleting 0 output folders
Successfully deleted 0 output folders in 108 msecs
Using dotnet: /root/helix/work/correlation/corerun
1 / 1 (0%, 0 failed): launching: /root/helix/work/correlation/corerun /root/helix/work/correlation/crossgen2/crossgen2.dll @/root/helix/work/workitem/e/readytorun/coreroot_determinism/coreroot_determinism/seed1/CPAOT-ret.out/System.Private.CoreLib.dll.rsp
Compilation failed. /root/helix/work/correlation/corerun /root/helix/work/correlation/R2RTest/R2RTest.dll compile-directory -cr /root/helix/work/correlation -in TestAssemblies --nojit --noexe --large-bubble --release --nocleanup -out seed1 failed with exit code 1
Xunit.Sdk.EqualException: Assert.Equal() Failure
Expected: 100
Actual:   1
   at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer) in /_/src/xunit.assert/Asserts/EqualityAsserts.cs:line 96
   at __GeneratedMainWrapper.Main()
Expected: 100
Actual: 101
END EXECUTION - FAILED
Test failed. Trying to see if dump file was created in /home/helixbot/dotnetbuild/dumps since 10/17/2023 5:43:08 AM
Processing /home/helixbot/dotnetbuild/dumps/coredump.59.dmp.crashreport.json
Printing stacktrace from '/home/helixbot/dotnetbuild/dumps/coredump.59.dmp.crashreport.json'
Invoking llvm-symbolizer --pretty-print
Errors while running llvm-symbolizer --pretty-print
System.ComponentModel.Win32Exception (2): An error occurred trying to start process 'llvm-symbolizer' with working directory '/root/helix/work/workitem/e/readytorun/coreroot_determinism/readytorun_coreroot_determinism'. No such file or directory
   at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec) in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs:line 499
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs:line 456
   at System.Diagnostics.P

Stack trace:

   at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 132
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)
   at Program.<<Main>$>g__TestExecutor1|0_2(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)
Queued Pipeline Pipeline_Configuration OS Arch Test Outcome Properties BuildReason
2023-10-17T05:43:01.947Z runtime-coreclr jitstress Checked-jitstress2_tiered osx.1200.amd64.open x64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd Failed {"System.JobAttempt":"1","architecture":"x64","DefinitionId":"109","System.StageAttempt":"1","System.JobName":"__default","System.PhaseName":"run_test_p1__osx_x64_checked","AzurePipelinesTestRunId":"9815420","BuildNumber":"20231016.1","System.JobId":"bf644a0b-2dbe-5a03-d8c6-a86fdd97c823","BuildId":"440585","configuration":"Checked-jitstress2_tiered","System.StageName":"Build","DefinitionName":"runtime-coreclr jitstress","Project":"public","CollectionUri":"https://dev.azure.com/dnceng-public/","operatingSystem":"OSX.1200.Amd64.Open","System.PhaseAttempt":"1","Reason":"Schedule"} Schedule
2023-10-17T05:42:59.118Z runtime-coreclr jitstress Checked-jitstress2 osx.1200.amd64.open x64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd Failed {"BuildId":"440585","DefinitionId":"109","System.JobAttempt":"1","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231016.1","configuration":"Checked-jitstress2","System.PhaseName":"run_test_p1__osx_x64_checked","AzurePipelinesTestRunId":"9815416","DefinitionName":"runtime-coreclr jitstress","architecture":"x64","operatingSystem":"OSX.1200.Amd64.Open","Project":"public","System.JobId":"bf644a0b-2dbe-5a03-d8c6-a86fdd97c823","System.JobName":"__default","Reason":"Schedule","System.PhaseAttempt":"1","System.StageAttempt":"1","System.StageName":"Build"} Schedule
2023-10-17T05:42:57.009Z runtime-coreclr jitstress Checked-jitstress1 osx.1200.amd64.open x64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd Failed {"System.JobAttempt":"1","BuildId":"440585","Reason":"Schedule","Project":"public","System.PhaseName":"run_test_p1__osx_x64_checked","architecture":"x64","AzurePipelinesTestRunId":"9815426","DefinitionName":"runtime-coreclr jitstress","System.PhaseAttempt":"1","CollectionUri":"https://dev.azure.com/dnceng-public/","DefinitionId":"109","operatingSystem":"OSX.1200.Amd64.Open","System.StageName":"Build","BuildNumber":"20231016.1","System.JobId":"bf644a0b-2dbe-5a03-d8c6-a86fdd97c823","System.JobName":"__default","configuration":"Checked-jitstress1","System.StageAttempt":"1"} Schedule
2023-10-17T05:42:54.492Z runtime-coreclr jitstress Checked-jitstress1_tiered osx.1200.amd64.open x64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd Failed {"BuildNumber":"20231016.1","CollectionUri":"https://dev.azure.com/dnceng-public/","DefinitionId":"109","Reason":"Schedule","System.JobId":"bf644a0b-2dbe-5a03-d8c6-a86fdd97c823","System.JobAttempt":"1","System.JobName":"__default","configuration":"Checked-jitstress1_tiered","System.StageName":"Build","operatingSystem":"OSX.1200.Amd64.Open","AzurePipelinesTestRunId":"9815424","System.PhaseAttempt":"1","Project":"public","System.StageAttempt":"1","architecture":"x64","System.PhaseName":"run_test_p1__osx_x64_checked","DefinitionName":"runtime-coreclr jitstress","BuildId":"440585"} Schedule
2023-10-17T05:27:23.318Z runtime-coreclr jitstress Checked-jitstress2 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd Failed {"Reason":"Schedule","AzurePipelinesTestRunId":"9815054","operatingSystem":"OSX.1200.ARM64.Open","BuildNumber":"20231016.1","System.JobAttempt":"1","System.PhaseAttempt":"1","System.StageAttempt":"1","System.JobName":"__default","System.PhaseName":"run_test_p1__osx_arm64_checked","Project":"public","CollectionUri":"https://dev.azure.com/dnceng-public/","configuration":"Checked-jitstress2","BuildId":"440585","architecture":"arm64","System.StageName":"Build","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","DefinitionId":"109","DefinitionName":"runtime-coreclr jitstress"} Schedule
More failures
Queued Pipeline Pipeline_Configuration OS Arch Test Outcome Properties BuildReason
2023-10-17T05:27:20.347Z runtime-coreclr jitstress Checked-jitstress2_tiered osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd Failed {"AzurePipelinesTestRunId":"9815032","System.PhaseName":"run_test_p1__osx_arm64_checked","operatingSystem":"OSX.1200.ARM64.Open","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","DefinitionId":"109","System.JobName":"__default","System.StageAttempt":"1","BuildId":"440585","Reason":"Schedule","configuration":"Checked-jitstress2_tiered","Project":"public","System.StageName":"Build","BuildNumber":"20231016.1","System.PhaseAttempt":"1","DefinitionName":"runtime-coreclr jitstress","System.JobAttempt":"1","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"arm64"} Schedule
2023-10-17T05:27:19.182Z runtime-coreclr jitstress Checked-jitstress1_tiered osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd Failed {"configuration":"Checked-jitstress1_tiered","System.PhaseAttempt":"1","Reason":"Schedule","Project":"public","DefinitionId":"109","DefinitionName":"runtime-coreclr jitstress","architecture":"arm64","System.JobName":"__default","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231016.1","BuildId":"440585","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","AzurePipelinesTestRunId":"9815030","System.StageAttempt":"1","System.StageName":"Build","System.PhaseName":"run_test_p1__osx_arm64_checked","operatingSystem":"OSX.1200.ARM64.Open","System.JobAttempt":"1"} Schedule
2023-10-17T05:27:18.206Z runtime-coreclr jitstress Checked-jitstress1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd Failed {"Reason":"Schedule","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","System.StageName":"Build","configuration":"Checked-jitstress1","System.JobName":"__default","System.JobAttempt":"1","DefinitionId":"109","BuildId":"440585","System.PhaseAttempt":"1","System.StageAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","Project":"public","DefinitionName":"runtime-coreclr jitstress","AzurePipelinesTestRunId":"9815034","operatingSystem":"OSX.1200.ARM64.Open","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231016.1"} Schedule
2023-10-02T07:24:59.077Z coreclr-release-outerloop-nightly Release-no_tiered_compilation ubuntu.1804.armarch.open arm readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"AzurePipelinesTestRunId":"9328590","operatingSystem":"(Alpine.315.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm32v7","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.StageAttempt":"1","BuildNumber":"20231001.1","architecture":"arm","System.JobName":"__default","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","Reason":"Schedule","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","BuildId":"424226","DefinitionId":"135","System.JobAttempt":"1","System.StageName":"Build"} Schedule
2023-10-02T07:24:59.077Z coreclr-release-outerloop-nightly Release-no_tiered_compilation ubuntu.1804.armarch.open arm readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"AzurePipelinesTestRunId":"9328590","operatingSystem":"(Alpine.315.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm32v7","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.StageAttempt":"1","BuildNumber":"20231001.1","architecture":"arm","System.JobName":"__default","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","Reason":"Schedule","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","BuildId":"424226","DefinitionId":"135","System.JobAttempt":"1","System.StageName":"Build"} Schedule
2023-10-02T07:24:59.077Z coreclr-release-outerloop-nightly Release-no_tiered_compilation ubuntu.1804.armarch.open arm readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"AzurePipelinesTestRunId":"9328590","operatingSystem":"(Alpine.315.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm32v7","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.StageAttempt":"1","BuildNumber":"20231001.1","architecture":"arm","System.JobName":"__default","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","Reason":"Schedule","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","BuildId":"424226","DefinitionId":"135","System.JobAttempt":"1","System.StageName":"Build"} Schedule
2023-10-02T07:24:59.077Z coreclr-release-outerloop-nightly Release-no_tiered_compilation ubuntu.1804.armarch.open arm readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"AzurePipelinesTestRunId":"9328590","operatingSystem":"(Alpine.315.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm32v7","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.StageAttempt":"1","BuildNumber":"20231001.1","architecture":"arm","System.JobName":"__default","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","Reason":"Schedule","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","BuildId":"424226","DefinitionId":"135","System.JobAttempt":"1","System.StageName":"Build"} Schedule
2023-10-02T07:24:59.077Z coreclr-release-outerloop-nightly Release-no_tiered_compilation ubuntu.1804.armarch.open arm readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"AzurePipelinesTestRunId":"9328590","operatingSystem":"(Alpine.315.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm32v7","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.StageAttempt":"1","BuildNumber":"20231001.1","architecture":"arm","System.JobName":"__default","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","Reason":"Schedule","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","BuildId":"424226","DefinitionId":"135","System.JobAttempt":"1","System.StageName":"Build"} Schedule
2023-10-02T07:24:59.077Z coreclr-release-outerloop-nightly Release-no_tiered_compilation ubuntu.1804.armarch.open arm readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"AzurePipelinesTestRunId":"9328590","operatingSystem":"(Alpine.315.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm32v7","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.StageAttempt":"1","BuildNumber":"20231001.1","architecture":"arm","System.JobName":"__default","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","Reason":"Schedule","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","BuildId":"424226","DefinitionId":"135","System.JobAttempt":"1","System.StageName":"Build"} Schedule
2023-10-02T07:24:59.077Z coreclr-release-outerloop-nightly Release-no_tiered_compilation ubuntu.1804.armarch.open arm readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"AzurePipelinesTestRunId":"9328590","operatingSystem":"(Alpine.315.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm32v7","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.StageAttempt":"1","BuildNumber":"20231001.1","architecture":"arm","System.JobName":"__default","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","Reason":"Schedule","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","BuildId":"424226","DefinitionId":"135","System.JobAttempt":"1","System.StageName":"Build"} Schedule
2023-10-02T07:24:59.077Z coreclr-release-outerloop-nightly Release-no_tiered_compilation ubuntu.1804.armarch.open arm readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"AzurePipelinesTestRunId":"9328590","operatingSystem":"(Alpine.315.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm32v7","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.StageAttempt":"1","BuildNumber":"20231001.1","architecture":"arm","System.JobName":"__default","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","Reason":"Schedule","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","BuildId":"424226","DefinitionId":"135","System.JobAttempt":"1","System.StageName":"Build"} Schedule
2023-10-02T07:24:59.077Z coreclr-release-outerloop-nightly Release-no_tiered_compilation ubuntu.1804.armarch.open arm readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"AzurePipelinesTestRunId":"9328590","operatingSystem":"(Alpine.315.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm32v7","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.StageAttempt":"1","BuildNumber":"20231001.1","architecture":"arm","System.JobName":"__default","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","Reason":"Schedule","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","BuildId":"424226","DefinitionId":"135","System.JobAttempt":"1","System.StageName":"Build"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
@v-wenyuxu v-wenyuxu added arch-arm64 os-linux Linux OS (any supported distro) os-mac-os-x macOS aka OSX os-windows JitStress CLR JIT issues involving JIT internal stress modes arch-x64 labels Oct 18, 2023
@dotnet-issue-labeler dotnet-issue-labeler bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Oct 18, 2023
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Oct 18, 2023
@ghost
Copy link

ghost commented Oct 18, 2023

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

Failed in: runtime-coreclr jitstress 20231016.1

Failed tests:

coreclr linux arm Checked jitstress1 @ (Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux x64 Checked jitstress2 @ Ubuntu.2204.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux x64 Checked jitstress1 @ Ubuntu.2204.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux arm Checked jitstress2_tiered @ (Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux arm Checked jitstress1_tiered @ (Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux arm Checked jitstress2 @ (Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux arm64 Checked jitstress2 @ (Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux arm64 Checked jitstress1_tiered @ (Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux x64 Checked jitstress2_tiered @ Ubuntu.2204.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr osx x64 Checked jitstress2 @ OSX.1200.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux x64 Checked jitstress1_tiered @ Ubuntu.2204.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr osx x64 Checked jitstress2_tiered @ OSX.1200.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr osx x64 Checked jitstress1_tiered @ OSX.1200.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr osx x64 Checked jitstress1 @ OSX.1200.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr windows x64 Checked jitstress1 @ Windows.10.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr windows x64 Checked jitstress2 @ Windows.10.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr windows x64 Checked jitstress1_tiered @ Windows.10.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr windows x64 Checked jitstress2_tiered @ Windows.10.Amd64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr osx arm64 Checked jitstress2 @ OSX.1200.ARM64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr osx arm64 Checked jitstress2_tiered @ OSX.1200.ARM64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr osx arm64 Checked jitstress1 @ OSX.1200.ARM64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr osx arm64 Checked jitstress1_tiered @ OSX.1200.ARM64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr windows arm64 Checked jitstress1 @ Windows.11.Arm64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr windows arm64 Checked jitstress2_tiered @ Windows.11.Arm64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr windows arm64 Checked jitstress2 @ Windows.11.Arm64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr windows arm64 Checked jitstress1_tiered @ Windows.11.Arm64.Open
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd
coreclr linux arm64 Checked jitstress2_tiered @ (Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8
    - readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd

Error message:

1 / 1 (100%, 1 failed): failed in 7392 msecs, exit code 134, expected 0: corerun /root/helix/work/correlation/crossgen2/crossgen2.dll @/root/helix/work/workitem/e/readytorun/coreroot_determinism/coreroot_determinism/seed1/CPAOT-ret.out/System.Private.CoreLib.dll.rsp
  Hit an illegal branch between funclets![createdump] Gathering state for process 59 corerun
  [createdump] Crashing thread 003b signal 6 (0006)
!! Assert failure(PID 59 [0x0000003b], Thread: 59 [0x003b]): Assertion failed 'tgtIG->igFuncIdx == jmpIG->igFuncIdx' in 'System.Xml.Linq.XHashtable`1[System.__Canon]:Add(System.__Canon):System.__Canon:this' during 'Generate code' (IL size 66; hash 0xa0e13e54; FullOpts)
!!     File: /__w/1/s/src/coreclr/jit/emit.cpp Line: 6540
!!     Image: /root/helix/work/correlation/corerun
  [createdump] Writing crash report to file /home/helixbot/dotnetbuild/dumps/coredump.59.dmp.crashreport.json
  [createdump] Crash report successfully written
  [createdump] Writing minidump with heap to file /home/helixbot/dotnetbuild/dumps/coredump.59.dmp
  [createdump] Written 192278528 bytes (46943 pages) to core file
  [createdump] Target process is alive
  [createdump] Dump successfully written in 310ms
!! waitpid() returned successfully (wstatus 00000000) WEXITSTATUS 0 WTERMSIG 0

Return code:      1
Raw output file:      /root/helix/work/workitem/uploads/coreroot_determinism/output.txt
Raw output:
BEGIN EXECUTION
/root/helix/work/correlation/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false -p System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true coreroot_determinism.dll ''
================================== Compiling with seed 1 ==================================
Locating output folder /root/helix/work/workitem/e/readytorun/coreroot_determinism/coreroot_determinism/seed1
Deleting 0 output folders
Successfully deleted 0 output folders in 108 msecs
Using dotnet: /root/helix/work/correlation/corerun
1 / 1 (0%, 0 failed): launching: /root/helix/work/correlation/corerun /root/helix/work/correlation/crossgen2/crossgen2.dll @/root/helix/work/workitem/e/readytorun/coreroot_determinism/coreroot_determinism/seed1/CPAOT-ret.out/System.Private.CoreLib.dll.rsp
Compilation failed. /root/helix/work/correlation/corerun /root/helix/work/correlation/R2RTest/R2RTest.dll compile-directory -cr /root/helix/work/correlation -in TestAssemblies --nojit --noexe --large-bubble --release --nocleanup -out seed1 failed with exit code 1
Xunit.Sdk.EqualException: Assert.Equal() Failure
Expected: 100
Actual:   1
   at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer) in /_/src/xunit.assert/Asserts/EqualityAsserts.cs:line 96
   at __GeneratedMainWrapper.Main()
Expected: 100
Actual: 101
END EXECUTION - FAILED
Test failed. Trying to see if dump file was created in /home/helixbot/dotnetbuild/dumps since 10/17/2023 5:43:08 AM
Processing /home/helixbot/dotnetbuild/dumps/coredump.59.dmp.crashreport.json
Printing stacktrace from '/home/helixbot/dotnetbuild/dumps/coredump.59.dmp.crashreport.json'
Invoking llvm-symbolizer --pretty-print
Errors while running llvm-symbolizer --pretty-print
System.ComponentModel.Win32Exception (2): An error occurred trying to start process 'llvm-symbolizer' with working directory '/root/helix/work/workitem/e/readytorun/coreroot_determinism/readytorun_coreroot_determinism'. No such file or directory
   at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec) in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs:line 499
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) in /_/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Unix.cs:line 456
   at System.Diagnostics.P

Stack trace:

   at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 132
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath)
   at Program.<<Main>$>g__TestExecutor1|0_2(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)
Queued Pipeline Pipeline_Configuration OS Arch Test Outcome Properties BuildReason
2023-10-17T05:43:01.947Z runtime-coreclr jitstress Checked-jitstress2_tiered osx.1200.amd64.open x64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd Failed {"System.JobAttempt":"1","architecture":"x64","DefinitionId":"109","System.StageAttempt":"1","System.JobName":"__default","System.PhaseName":"run_test_p1__osx_x64_checked","AzurePipelinesTestRunId":"9815420","BuildNumber":"20231016.1","System.JobId":"bf644a0b-2dbe-5a03-d8c6-a86fdd97c823","BuildId":"440585","configuration":"Checked-jitstress2_tiered","System.StageName":"Build","DefinitionName":"runtime-coreclr jitstress","Project":"public","CollectionUri":"https://dev.azure.com/dnceng-public/","operatingSystem":"OSX.1200.Amd64.Open","System.PhaseAttempt":"1","Reason":"Schedule"} Schedule
2023-10-17T05:42:59.118Z runtime-coreclr jitstress Checked-jitstress2 osx.1200.amd64.open x64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd Failed {"BuildId":"440585","DefinitionId":"109","System.JobAttempt":"1","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231016.1","configuration":"Checked-jitstress2","System.PhaseName":"run_test_p1__osx_x64_checked","AzurePipelinesTestRunId":"9815416","DefinitionName":"runtime-coreclr jitstress","architecture":"x64","operatingSystem":"OSX.1200.Amd64.Open","Project":"public","System.JobId":"bf644a0b-2dbe-5a03-d8c6-a86fdd97c823","System.JobName":"__default","Reason":"Schedule","System.PhaseAttempt":"1","System.StageAttempt":"1","System.StageName":"Build"} Schedule
2023-10-17T05:42:57.009Z runtime-coreclr jitstress Checked-jitstress1 osx.1200.amd64.open x64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd Failed {"System.JobAttempt":"1","BuildId":"440585","Reason":"Schedule","Project":"public","System.PhaseName":"run_test_p1__osx_x64_checked","architecture":"x64","AzurePipelinesTestRunId":"9815426","DefinitionName":"runtime-coreclr jitstress","System.PhaseAttempt":"1","CollectionUri":"https://dev.azure.com/dnceng-public/","DefinitionId":"109","operatingSystem":"OSX.1200.Amd64.Open","System.StageName":"Build","BuildNumber":"20231016.1","System.JobId":"bf644a0b-2dbe-5a03-d8c6-a86fdd97c823","System.JobName":"__default","configuration":"Checked-jitstress1","System.StageAttempt":"1"} Schedule
2023-10-17T05:42:54.492Z runtime-coreclr jitstress Checked-jitstress1_tiered osx.1200.amd64.open x64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd Failed {"BuildNumber":"20231016.1","CollectionUri":"https://dev.azure.com/dnceng-public/","DefinitionId":"109","Reason":"Schedule","System.JobId":"bf644a0b-2dbe-5a03-d8c6-a86fdd97c823","System.JobAttempt":"1","System.JobName":"__default","configuration":"Checked-jitstress1_tiered","System.StageName":"Build","operatingSystem":"OSX.1200.Amd64.Open","AzurePipelinesTestRunId":"9815424","System.PhaseAttempt":"1","Project":"public","System.StageAttempt":"1","architecture":"x64","System.PhaseName":"run_test_p1__osx_x64_checked","DefinitionName":"runtime-coreclr jitstress","BuildId":"440585"} Schedule
2023-10-17T05:27:23.318Z runtime-coreclr jitstress Checked-jitstress2 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd Failed {"Reason":"Schedule","AzurePipelinesTestRunId":"9815054","operatingSystem":"OSX.1200.ARM64.Open","BuildNumber":"20231016.1","System.JobAttempt":"1","System.PhaseAttempt":"1","System.StageAttempt":"1","System.JobName":"__default","System.PhaseName":"run_test_p1__osx_arm64_checked","Project":"public","CollectionUri":"https://dev.azure.com/dnceng-public/","configuration":"Checked-jitstress2","BuildId":"440585","architecture":"arm64","System.StageName":"Build","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","DefinitionId":"109","DefinitionName":"runtime-coreclr jitstress"} Schedule
More failures
Queued Pipeline Pipeline_Configuration OS Arch Test Outcome Properties BuildReason
2023-10-17T05:27:20.347Z runtime-coreclr jitstress Checked-jitstress2_tiered osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd Failed {"AzurePipelinesTestRunId":"9815032","System.PhaseName":"run_test_p1__osx_arm64_checked","operatingSystem":"OSX.1200.ARM64.Open","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","DefinitionId":"109","System.JobName":"__default","System.StageAttempt":"1","BuildId":"440585","Reason":"Schedule","configuration":"Checked-jitstress2_tiered","Project":"public","System.StageName":"Build","BuildNumber":"20231016.1","System.PhaseAttempt":"1","DefinitionName":"runtime-coreclr jitstress","System.JobAttempt":"1","CollectionUri":"https://dev.azure.com/dnceng-public/","architecture":"arm64"} Schedule
2023-10-17T05:27:19.182Z runtime-coreclr jitstress Checked-jitstress1_tiered osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd Failed {"configuration":"Checked-jitstress1_tiered","System.PhaseAttempt":"1","Reason":"Schedule","Project":"public","DefinitionId":"109","DefinitionName":"runtime-coreclr jitstress","architecture":"arm64","System.JobName":"__default","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231016.1","BuildId":"440585","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","AzurePipelinesTestRunId":"9815030","System.StageAttempt":"1","System.StageName":"Build","System.PhaseName":"run_test_p1__osx_arm64_checked","operatingSystem":"OSX.1200.ARM64.Open","System.JobAttempt":"1"} Schedule
2023-10-17T05:27:18.206Z runtime-coreclr jitstress Checked-jitstress1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.cmd Failed {"Reason":"Schedule","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","System.StageName":"Build","configuration":"Checked-jitstress1","System.JobName":"__default","System.JobAttempt":"1","DefinitionId":"109","BuildId":"440585","System.PhaseAttempt":"1","System.StageAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","Project":"public","DefinitionName":"runtime-coreclr jitstress","AzurePipelinesTestRunId":"9815034","operatingSystem":"OSX.1200.ARM64.Open","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231016.1"} Schedule
2023-10-02T07:24:59.077Z coreclr-release-outerloop-nightly Release-no_tiered_compilation ubuntu.1804.armarch.open arm readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"AzurePipelinesTestRunId":"9328590","operatingSystem":"(Alpine.315.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm32v7","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.StageAttempt":"1","BuildNumber":"20231001.1","architecture":"arm","System.JobName":"__default","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","Reason":"Schedule","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","BuildId":"424226","DefinitionId":"135","System.JobAttempt":"1","System.StageName":"Build"} Schedule
2023-10-02T07:24:59.077Z coreclr-release-outerloop-nightly Release-no_tiered_compilation ubuntu.1804.armarch.open arm readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"AzurePipelinesTestRunId":"9328590","operatingSystem":"(Alpine.315.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm32v7","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.StageAttempt":"1","BuildNumber":"20231001.1","architecture":"arm","System.JobName":"__default","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","Reason":"Schedule","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","BuildId":"424226","DefinitionId":"135","System.JobAttempt":"1","System.StageName":"Build"} Schedule
2023-10-02T07:24:59.077Z coreclr-release-outerloop-nightly Release-no_tiered_compilation ubuntu.1804.armarch.open arm readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"AzurePipelinesTestRunId":"9328590","operatingSystem":"(Alpine.315.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm32v7","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.StageAttempt":"1","BuildNumber":"20231001.1","architecture":"arm","System.JobName":"__default","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","Reason":"Schedule","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","BuildId":"424226","DefinitionId":"135","System.JobAttempt":"1","System.StageName":"Build"} Schedule
2023-10-02T07:24:59.077Z coreclr-release-outerloop-nightly Release-no_tiered_compilation ubuntu.1804.armarch.open arm readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"AzurePipelinesTestRunId":"9328590","operatingSystem":"(Alpine.315.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm32v7","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.StageAttempt":"1","BuildNumber":"20231001.1","architecture":"arm","System.JobName":"__default","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","Reason":"Schedule","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","BuildId":"424226","DefinitionId":"135","System.JobAttempt":"1","System.StageName":"Build"} Schedule
2023-10-02T07:24:59.077Z coreclr-release-outerloop-nightly Release-no_tiered_compilation ubuntu.1804.armarch.open arm readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"AzurePipelinesTestRunId":"9328590","operatingSystem":"(Alpine.315.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm32v7","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.StageAttempt":"1","BuildNumber":"20231001.1","architecture":"arm","System.JobName":"__default","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","Reason":"Schedule","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","BuildId":"424226","DefinitionId":"135","System.JobAttempt":"1","System.StageName":"Build"} Schedule
2023-10-02T07:24:59.077Z coreclr-release-outerloop-nightly Release-no_tiered_compilation ubuntu.1804.armarch.open arm readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"AzurePipelinesTestRunId":"9328590","operatingSystem":"(Alpine.315.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm32v7","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.StageAttempt":"1","BuildNumber":"20231001.1","architecture":"arm","System.JobName":"__default","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","Reason":"Schedule","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","BuildId":"424226","DefinitionId":"135","System.JobAttempt":"1","System.StageName":"Build"} Schedule
2023-10-02T07:24:59.077Z coreclr-release-outerloop-nightly Release-no_tiered_compilation ubuntu.1804.armarch.open arm readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"AzurePipelinesTestRunId":"9328590","operatingSystem":"(Alpine.315.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm32v7","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.StageAttempt":"1","BuildNumber":"20231001.1","architecture":"arm","System.JobName":"__default","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","Reason":"Schedule","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","BuildId":"424226","DefinitionId":"135","System.JobAttempt":"1","System.StageName":"Build"} Schedule
2023-10-02T07:24:59.077Z coreclr-release-outerloop-nightly Release-no_tiered_compilation ubuntu.1804.armarch.open arm readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"AzurePipelinesTestRunId":"9328590","operatingSystem":"(Alpine.315.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm32v7","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.StageAttempt":"1","BuildNumber":"20231001.1","architecture":"arm","System.JobName":"__default","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","Reason":"Schedule","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","BuildId":"424226","DefinitionId":"135","System.JobAttempt":"1","System.StageName":"Build"} Schedule
2023-10-02T07:24:59.077Z coreclr-release-outerloop-nightly Release-no_tiered_compilation ubuntu.1804.armarch.open arm readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"AzurePipelinesTestRunId":"9328590","operatingSystem":"(Alpine.315.Arm32.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm32v7","System.PhaseName":"run_test_p1_R2R_linux_musl_arm_release","DefinitionName":"coreclr-release-outerloop-nightly","Project":"public","System.StageAttempt":"1","BuildNumber":"20231001.1","architecture":"arm","System.JobName":"__default","configuration":"Release-no_tiered_compilation","System.PhaseAttempt":"1","Reason":"Schedule","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobId":"c8f31202-1062-5d6a-be4d-45312a30ae25","BuildId":"424226","DefinitionId":"135","System.JobAttempt":"1","System.StageName":"Build"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
2023-10-01T19:51:38.315Z runtime-coreclr jitstress-random Checked-jitstress_random_1 osx.1200.arm64.open arm64 readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh Failed {"configuration":"Checked-jitstress_random_1","System.JobId":"765fd5e9-e47f-5cb0-4cbf-499eaaaee93e","Reason":"Schedule","System.StageName":"Build","DefinitionId":"159","DefinitionName":"runtime-coreclr jitstress-random","operatingSystem":"OSX.1200.ARM64.Open","AzurePipelinesTestRunId":"9324632","Project":"public","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"arm64","CollectionUri":"https://dev.azure.com/dnceng-public/","BuildNumber":"20231001.1","System.StageAttempt":"1","System.JobAttempt":"1","System.PhaseName":"run_test_p1__osx_arm64_checked","BuildId":"424103"} Schedule
Author: v-wenyuxu
Assignees: -
Labels:

arch-arm64, os-linux, os-mac-os-x, os-windows, JitStress, arch-x64, area-CodeGen-coreclr
Milestone: -

@jakobbotsch
Copy link
Member

The assertion also happens in Methodical_d1

set DOTNET_TieredCompilation=0
set DOTNET_JitStress=1
...
05:56:38.146 Running test: JIT/Methodical/eh/deadcode/badcodeafterfault_d/badcodeafterfault_d.dll
Hit an illegal branch between funclets!
Assert failure(PID 3680 [0x00000e60], Thread: 6048 [0x17a0]): Assertion failed 'tgtIG->igFuncIdx == jmpIG->igFuncIdx' in 'Test_badcodeafterfault:GetObj():System.Object' during 'Generate code' (IL size 18; hash 0xdfcaa346; MinOpts)

    File: D:\a\_work\1\s\src\coreclr\jit\emit.cpp Line: 6540
    Image: C:\h\w\BB0209FD\p\corerun.exe

App Exit Code: -1073740286
Expected: 100
Actual: -1073740286
END EXECUTION - FAILED
FAILED

I bisected it to #93429, cc @EgorBo

@jakobbotsch jakobbotsch mentioned this issue Oct 18, 2023
Closed
@jakobbotsch jakobbotsch added the blocking-clean-ci-optional Blocking optional rolling runs label Oct 18, 2023
@jakobbotsch jakobbotsch added this to the 9.0.0 milestone Oct 18, 2023
@ghost ghost removed the untriaged New issue has not been triaged by the area owner label Oct 18, 2023
@jakobbotsch jakobbotsch mentioned this issue Oct 18, 2023
Closed
@jakobbotsch
Copy link
Member

Fixed by #93429

@ghost ghost locked as resolved and limited conversation to collaborators Nov 22, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
arch-arm64 arch-x64 area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI blocking-clean-ci-optional Blocking optional rolling runs JitStress CLR JIT issues involving JIT internal stress modes os-linux Linux OS (any supported distro) os-mac-os-x macOS aka OSX os-windows
Projects
None yet
Milestone
9.0.0
Development

No branches or pull requests
2 participants
@jakobbotsch @v-wenyuxu