Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test failure: readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh #45326

Closed
v-haren opened this issue Nov 30, 2020 · 16 comments · Fixed by #49066 or #50676
Closed
Assignees
Milestone

Comments

@v-haren
Copy link

v-haren commented Nov 30, 2020

failed in job: https://dev.azure.com/dnceng/public/_build/results?buildId=901492&view=ms.vss-test-web.build-test-results-tab&runId=28795396&resultId=106816&paneView=debug

CoreCLR Linux x64 Checked jitstress1_tiered @ Ubuntu.1804.Amd64.Open

Error message

1 / 1 (100%, 1 failed): failed in 15981 msecs, exit code 134, expected 0: corerun /home/helixbot/work/A64D08B1/p/crossgen2/crossgen2.dll @/home/helixbot/work/A64D08B1/w/A68C0979/e/readytorun/coreroot_determinism/coreroot_determinism/seed1/CPAOT-ret.out/System.Private.CoreLib.dll.rsp\n\nReturn code:      1
Raw output file:      /home/helixbot/work/A64D08B1/w/A68C0979/e/readytorun/coreroot_determinism/Reports/readytorun.coreroot_determinism/coreroot_determinism/coreroot_determinism.output.txt
Raw output:
BEGIN EXECUTION
/home/helixbot/work/A64D08B1/p/corerun coreroot_determinism.dll ''
================================== Compiling with seed 1 ==================================
Locating output folder /home/helixbot/work/A64D08B1/w/A68C0979/e/readytorun/coreroot_determinism/coreroot_determinism/seed1
Deleting 0 output folders
Successfully deleted 0 output folders in 60 msecs
Using dotnet: /home/helixbot/work/A64D08B1/p/corerun
1 / 1 (0%, 0 failed): launching: /home/helixbot/work/A64D08B1/p/corerun /home/helixbot/work/A64D08B1/p/crossgen2/crossgen2.dll @/home/helixbot/work/A64D08B1/w/A68C0979/e/readytorun/coreroot_determinism/coreroot_determinism/seed1/CPAOT-ret.out/System.Private.CoreLib.dll.rsp
Compilation failed. /home/helixbot/work/A64D08B1/p/corerun /home/helixbot/work/A64D08B1/p/R2RTest/R2RTest.dll compile-directory -cr /home/helixbot/work/A64D08B1/p -in TestAssemblies --nojit --noexe --large-bubble --release --nocleanup -out seed1 failed with exit code 1
Expected: 100
Actual: 1
END EXECUTION - FAILED
Test Harness Exitcode is : 1
To run the test:
> set CORE_ROOT=/home/helixbot/work/A64D08B1/p
> /home/helixbot/work/A64D08B1/w/A68C0979/e/readytorun/coreroot_determinism/coreroot_determinism/coreroot_determinism.sh
Expected: True
Actual:   False

Stack trace
   at readytorun_coreroot_determinism._coreroot_determinism_coreroot_determinism_._coreroot_determinism_coreroot_determinism_sh() in /__w/1/s/artifacts/tests/coreclr/Linux.x64.Checked/TestWrappers/readytorun.coreroot_determinism/readytorun.coreroot_determinism.XUnitWrapper.cs:line 153

Runfo Tracking Issue: coreroot_determinism.cmd

Build Definition Kind Run Name Console Core Dump Test Results Run Client
1079379 runtime PR 50922 CoreCLR windows x86 Checked @ Windows.10.Amd64.Open console.log runclient.py
1079379 runtime PR 50922 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1079379 runtime PR 50922 CoreCLR windows arm64 Checked @ Windows.10.Arm64v8.Open console.log runclient.py
1079379 runtime PR 50922 CoreCLR windows arm64 Checked no_tiered_compilation @ Windows.10.Arm64v8.Open console.log runclient.py
1079379 runtime PR 50922 CoreCLR windows x64 Checked @ Windows.10.Amd64.Open console.log runclient.py
1079379 runtime PR 50922 CoreCLR windows x64 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1079379 runtime PR 50922 CoreCLR windows x86 Checked @ Windows.10.Amd64.Open console.log runclient.py
1079379 runtime PR 50922 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1079379 runtime PR 50922 CoreCLR windows arm64 Checked @ Windows.10.Arm64v8.Open console.log runclient.py
1079379 runtime PR 50922 CoreCLR windows arm64 Checked no_tiered_compilation @ Windows.10.Arm64v8.Open console.log runclient.py
1079379 runtime PR 50922 CoreCLR windows arm64 Checked no_tiered_compilation @ Windows.10.Arm64v8.Open console.log runclient.py
1079379 runtime PR 50922 CoreCLR windows x64 Checked @ Windows.10.Amd64.Open console.log runclient.py
1079379 runtime PR 50922 CoreCLR windows x64 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1078291 runtime PR 50922 CoreCLR windows x64 Checked @ Windows.10.Amd64.Open console.log runclient.py
1078291 runtime PR 50922 CoreCLR windows x64 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1078291 runtime PR 50922 CoreCLR windows arm64 Checked @ Windows.10.Arm64v8.Open console.log runclient.py
1078291 runtime PR 50922 CoreCLR windows arm64 Checked no_tiered_compilation @ Windows.10.Arm64v8.Open console.log runclient.py
1059736 runtime PR 50232 CoreCLR windows x64 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1059736 runtime PR 50232 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1059736 runtime PR 50232 CoreCLR windows arm64 Checked no_tiered_compilation @ Windows.10.Arm64v8.Open console.log
1057171 runtime PR 50082 CoreCLR windows x64 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1057171 runtime PR 50082 CoreCLR windows arm64 Checked no_tiered_compilation @ Windows.10.Arm64v8.Open console.log runclient.py
1057171 runtime PR 50082 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1037848 runtime PR 49576 CoreCLR windows x64 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1028832 runtime PR 49295 CoreCLR windows arm64 Checked no_tiered_compilation @ Windows.10.Arm64v8.Open console.log
1028832 runtime PR 49295 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log
1028832 runtime PR 49295 CoreCLR windows x64 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log
1027318 runtime PR 47901 CoreCLR windows x64 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1025677 runtime PR 49257 CoreCLR windows x64 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log
1025113 runtime PR 49183 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1023780 runtime PR 48936 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1023759 runtime PR 49178 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1023260 runtime PR 49066 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1022125 runtime PR 48936 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1021719 runtime PR 49093 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1021643 runtime PR 48589 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1021240 runtime PR 49074 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1020340 runtime PR 48589 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1019099 runtime PR 49021 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1017879 runtime PR 48923 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1017716 runtime PR 45058 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1017614 runtime PR 48923 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1013297 runtime Rolling CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1013095 runtime PR 48677 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1012960 runtime PR 48171 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1012535 runtime PR 48536 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1011419 runtime PR 48746 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1011339 runtime PR 48601 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1010968 runtime PR 48731 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1010324 runtime PR 48435 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1009843 runtime PR 47639 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1009838 runtime PR 48698 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1009500 runtime PR 48691 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1007749 runtime PR 44905 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py
1007653 runtime PR 48642 CoreCLR windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open console.log runclient.py

Build Result Summary

Day Hit Count Week Hit Count Month Hit Count
0 2 4
@Dotnet-GitSync-Bot Dotnet-GitSync-Bot added area-crossgen2-coreclr untriaged New issue has not been triaged by the area owner labels Nov 30, 2020
@sandreenko sandreenko added the blocking-outerloop Blocking the 'runtime-coreclr outerloop' and 'runtime-libraries-coreclr outerloop' runs label Dec 17, 2020
@sandreenko
Copy link
Contributor

PTAL @dotnet/crossgen-contrib

@mangod9 mangod9 removed the untriaged New issue has not been triaged by the area owner label Dec 17, 2020
@mangod9 mangod9 added this to the 6.0.0 milestone Dec 17, 2020
@sandreenko
Copy link
Contributor

disabled in #46350

@briansull
Copy link
Contributor

briansull commented Feb 23, 2021

Seeing this test fail:
CoreCLR Pri0 Runtime Tests Run windows x86 checked failed
https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-48503-merge-207a8f793fe04202a9/PayloadGroup0/console.ea9e278d.log?sv=2019-07-07&se=2021-03-14T23%3A17%3A41Z&sr=c&sp=rl&sig=7xSihSCdtFD9g58mtQMlPSn4DnFqJJMa6iU9JSiVyhM%3D

    readytorun\coreroot_determinism\coreroot_determinism\coreroot_determinism.cmd [FAIL]
      1 / 1 (100%, 1 failed): failed in 34712 msecs, exit code -1073740286 = 0xC0000602, expected 0: corerun.exe C:\h\w\ACFF09DC\p\crossgen2\crossgen2.dll @C:\h\w\ACFF09DC\w\A4000920\e\readytorun\coreroot_determinism\coreroot_determinism\seed2\CPAOT-ret.out\System.Private.CoreLib.dll.rsp
      !! Assert failure(PID 7736 [0x00001e38], Thread: 7740 [0x1e3c]): CONTRACT VIOLATION by Thread::SetFrame at "D:\workspace\_work\1\s\src\coreclr\vm\threads.cpp" @ 212
      !! MODE_COOPERATIVE encountered while thread is in preemptive state.
      !!                         CONTRACT in Thread::SetFrame at "D:\workspace\_work\1\s\src\coreclr\vm\threads.cpp" @ 212
      !!                         CONTRACT in HijackFrame::HijackFrame at "D:\workspace\_work\1\s\src\coreclr\vm\threadsuspend.cpp" @ 4970
      !!                         CONTRACT in OnHijackWorker at "D:\workspace\_work\1\s\src\coreclr\vm\threadsuspend.cpp" @ 4984
      !!                         CONTRACT in MethodDescCallSite::CallTargetWorker at "D:\workspace\_work\1\s\src\coreclr\vm\callhelpers.cpp" @ 287
      !!                         GCX_COOP in Assembly::ExecuteMainMethod at "D:\workspace\_work\1\s\src\coreclr\vm\assembly.cpp" @ 1670
      !!                         CONTRACT in Assembly::ExecuteMainMethod at "D:\workspace\_work\1\s\src\coreclr\vm\assembly.cpp" @ 1654
      !!                         GCX_COOP in CorHost2::ExecuteAssembly at "D:\workspace\_work\1\s\src\coreclr\vm\corhost.cpp" @ 357
      !!                         CONTRACT in CorHost2::ExecuteAssembly at "D:\workspace\_work\1\s\src\coreclr\vm\corhost.cpp" @ 306
      !! We can't find the violated contract. Look for an old-style non-holder-based contract.
      !! CORECLR! CONTRACT_ASSERT + 0x2C8 (0x70164cf9)
      !! CORECLR! EEContract::DoChecks + 0x144 (0x70403d0e)
      !! CORECLR! Thread::SetFrame + 0x208 (0x70378430)
      !! CORECLR! HijackFrame::HijackFrame + 0xD8 (0x70552d09)
      !! CORECLR! OnHijackWorker + 0xBF (0x7055b34e)
      !! CORECLR! OnHijackTripThread + 0x11 (0x7069b9ee)
      !! CLRJIT_WIN_X86_X86! Compiler::eeGetMethodName + 0x102 (0x6f1fa356)
      !! CLRJIT_WIN_X86_X86! Compiler::compCompile + 0x3F5 (0x6f1f30bb)
      !! CLRJIT_WIN_X86_X86! ``jitNativeCode'::`8'::__Body::Run'::`5'::__Body::Run + 0xC5 (0x6f1f1c08)
      !! CLRJIT_WIN_X86_X86! `jitNativeCode'::`8'::__Body::Run + 0x38 (0x6f1f1c48)
      !!     File: D:\workspace\_work\1\s\src\coreclr\vm\threads.cpp Line: 212
      !!     Image: C:\h\w\ACFF09DC\p\corerun.exe

@trylek
Copy link
Member

trylek commented Feb 23, 2021

I'm afraid that the only person capable of understanding all the gory details of this issue - JanV - is right now busy celebrating the birth of his sixth baby ;-).

@hoyosjs hoyosjs added the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label Feb 24, 2021
@BruceForstall
Copy link
Member

@trylek : @sandreenko disabled this in #46350 but just for jitstress. Can we just disable it for all runs for now?

@jkotas
Copy link
Member

jkotas commented Mar 2, 2021

The x86 SetFrame crash is regression introduced by #46238.

The problem is that EECodeManager::GetReturnAddressHijackInfo is missing return false for reverse PInvoke methods on x86.

@jkoritzinsky Could you please take a look?

@jkoritzinsky
Copy link
Member

I'll take a look tomorrow.

@BruceForstall
Copy link
Member

I see a failure today's in outerloop win-x86:

https://dev.azure.com/dnceng/public/_build/results?buildId=1022865&view=ms.vss-test-web.build-test-results-tab&runId=31815932&resultId=110541&paneView=debug

        1 / 1 (100%, 1 failed): failed in 22592 msecs, exit code -1073740286 = 0xC0000602, expected 0: corerun.exe C:\h\w\B45109C6\p\crossgen2\crossgen2.dll @C:\h\w\B45109C6\w\B48D09E4\e\readytorun\coreroot_determinism\coreroot_determinism\seed2\CPAOT-ret.out\System.Private.CoreLib.dll.rsp
        !! Assert failure(PID 6176 [0x00001820], Thread: 4668 [0x123c]): CONTRACT VIOLATION by Thread::SetFrame at "D:\workspace\_work\1\s\src\coreclr\vm\threads.cpp" @ 212
        !! MODE_COOPERATIVE encountered while thread is in preemptive state.
        !!                         CONTRACT in Thread::SetFrame at "D:\workspace\_work\1\s\src\coreclr\vm\threads.cpp" @ 212
        !!                         CONTRACT in HijackFrame::HijackFrame at "D:\workspace\_work\1\s\src\coreclr\vm\threadsuspend.cpp" @ 4970
        !!                         CONTRACT in OnHijackWorker at "D:\workspace\_work\1\s\src\coreclr\vm\threadsuspend.cpp" @ 4984
        !!                         CONTRACT in MethodDescCallSite::CallTargetWorker at "D:\workspace\_work\1\s\src\coreclr\vm\callhelpers.cpp" @ 287
        !!                         GCX_COOP in Assembly::ExecuteMainMethod at "D:\workspace\_work\1\s\src\coreclr\vm\assembly.cpp" @ 1670
        !!                         CONTRACT in Assembly::ExecuteMainMethod at "D:\workspace\_work\1\s\src\coreclr\vm\assembly.cpp" @ 1654
        !!                         GCX_COOP in CorHost2::ExecuteAssembly at "D:\workspace\_work\1\s\src\coreclr\vm\corhost.cpp" @ 357
        !!                         CONTRACT in CorHost2::ExecuteAssembly at "D:\workspace\_work\1\s\src\coreclr\vm\corhost.cpp" @ 306
        !! We can't find the violated contract. Look for an old-style non-holder-based contract.
        !! CORECLR! CONTRACT_ASSERT + 0x2C8 (0x6fea4cf9)
        !! CORECLR! EEContract::DoChecks + 0x144 (0x70143935)
        !! CORECLR! Thread::SetFrame + 0x208 (0x700b83b0)
        !! CORECLR! HijackFrame::HijackFrame + 0xD8 (0x70292989)
        !! CORECLR! OnHijackWorker + 0xBF (0x7029afce)
        !! CORECLR! OnHijackTripThread + 0x11 (0x703db77e)
        !! CLRJIT_WIN_X86_X86! emitter::emitEndCodeGen + 0x22E (0x6ef6dc2e)
        !! CLRJIT_WIN_X86_X86! CodeGen::genEmitMachineCode + 0xB1 (0x6ef53631)
        !! CLRJIT_WIN_X86_X86! CodeGenPhase::DoPhase + 0x14 (0x6ef51754)
        !! CLRJIT_WIN_X86_X86! Phase::Run + 0x38 (0x6f02e938)
        !!     File: D:\workspace\_work\1\s\src\coreclr\vm\threads.cpp Line: 212
        !!     Image: C:\h\w\B45109C6\p\corerun.exe

@trylek
Copy link
Member

trylek commented Mar 4, 2021

Sorry, accidentally added a circular reference, intended to refer to #48719 (comment) where I recently hit this.

@VincentBu
Copy link
Contributor

Failed again in runtime-coreclr r2r 20210303.1

Failed test:

R2R windows x86 Checked no_tiered_compilation @ Windows.10.Amd64.Open
 -readytorun\\coreroot_determinism\\coreroot_determinism\\coreroot_determinism.cmd

Error message:

1 / 1 (100%, 1 failed): failed in 34825 msecs, exit code -1073740286 = 0xC0000602, expected 0: corerun.exe C:\h\w\B9440A48\p\crossgen2\crossgen2.dll @C:\h\w\B9440A48\w\A18A08EA\e\readytorun\coreroot_determinism\coreroot_determinism\seed1\CPAOT-ret.out\System.Private.CoreLib.dll.rsp
!! Assert failure(PID 3840 [0x00000f00], Thread: 3796 [0x0ed4]): CONTRACT VIOLATION by Thread::SetFrame at "D:\workspace\_work\1\s\src\coreclr\vm\threads.cpp" @ 212
!! MODE_COOPERATIVE encountered while thread is in preemptive state.
!! CONTRACT in Thread::SetFrame at "D:\workspace\_work\1\s\src\coreclr\vm\threads.cpp" @ 212
!! CONTRACT in HijackFrame::HijackFrame at "D:\workspace\_work\1\s\src\coreclr\vm\threadsuspend.cpp" @ 4970
!! CONTRACT in OnHijackWorker at "D:\workspace\_work\1\s\src\coreclr\vm\threadsuspend.cpp" @ 4984
!! CONTRACT in MethodDescCallSite::CallTargetWorker at "D:\workspace\_work\1\s\src\coreclr\vm\callhelpers.cpp" @ 287
!! GCX_COOP in Assembly::ExecuteMainMethod at "D:\workspace\_work\1\s\src\coreclr\vm\assembly.cpp" @ 1670
!! CONTRACT in Assembly::ExecuteMainMethod at "D:\workspace\_work\1\s\src\coreclr\vm\assembly.cpp" @ 1654
!! GCX_COOP in CorHost2::ExecuteAssembly at "D:\workspace\_work\1\s\src\coreclr\vm\corhost.cpp" @ 357
!! CONTRACT in CorHost2::ExecuteAssembly at "D:\workspace\_work\1\s\src\coreclr\vm\corhost.cpp" @ 306
!! We can't find the violated contract. Look for an old-style non-holder-based contract.
!! CORECLR! CONTRACT_ASSERT + 0x2C8 (0x70e14cf9)
!! CORECLR! EEContract::DoChecks + 0x144 (0x710b3935)
!! CORECLR! Thread::SetFrame + 0x208 (0x710283b0)
!! CORECLR! HijackFrame::HijackFrame + 0xD8 (0x71202989)
!! CORECLR! OnHijackWorker + 0xBF (0x7120afce)
!! CORECLR! OnHijackTripThread + 0x11 (0x7134b77e)
!! CLRJIT_WIN_X86_X86! Compiler::eeGetMethodName + 0x102 (0x6ee4a2a6)
!! CLRJIT_WIN_X86_X86! Compiler::eeGetMethodFullName + 0x20 (0x6ee4b0bd)
!! CLRJIT_WIN_X86_X86! <lambda_201b456a578c759327e46be22656edae>::operator() + 0xB6 (0x6ee60e04)
!! JITINTERFACE_X86! JitInterfaceWrapper::runWithErrorTrap + 0x1F (0x6fa6119f)
!! File: D:\workspace\work\1\s\src\coreclr\vm\threads.cpp Line: 212
!! Image: C:\h\w\B9440A48\p\corerun.exe

Return code: 1
Raw output file: C:\h\w\B9440A48\w\A18A08EA\e\readytorun\coreroot_determinism\Reports\readytorun.coreroot_determinism\coreroot_determinism\coreroot_determinism.output.txt
Raw output:
BEGIN EXECUTION
 "C:\h\w\B9440A48\p\corerun.exe" coreroot_determinism.dll 
================================== Compiling with seed 1 ==================================
Locating output folder C:\h\w\B9440A48\w\A18A08EA\e\readytorun\coreroot_determinism\coreroot_determinism\seed1
Deleting 0 output folders
Successfully deleted 0 output folders in 152 msecs
Using dotnet: C:\h\w\B9440A48\p\corerun.exe
1 / 1 (0%, 0 failed): launching: C:\h\w\B9440A48\p\corerun.exe C:\h\w\B9440A48\p\crossgen2\crossgen2.dll @C:\h\w\B9440A48\w\A18A08EA\e\readytorun\coreroot_determinism\coreroot_determinism\seed1\CPAOT-ret.out\System.Private.CoreLib.dll.rsp
Compilation failed. C:\h\w\B9440A48\p\corerun.exe C:\h\w\B9440A48\p\R2RTest\R2RTest.dll compile-directory -cr C:\h\w\B9440A48\p -in TestAssemblies --nojit --noexe --large-bubble --release --nocleanup -out seed1 failed with exit code 1
Expected: 100
Actual: 1
END EXECUTION - FAILED
FAILED
Test Harness Exitcode is : 1
To run the test:
> set CORE_ROOT=C:\h\w\B9440A48\p
> C:\h\w\B9440A48\w\A18A08EA\e\readytorun\coreroot_determinism\coreroot

Stack trace
   at readytorun_coreroot_determinism._coreroot_determinism_coreroot_determinism_._coreroot_determinism_coreroot_determinism_cmd()

@BruceForstall
Copy link
Member

@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Mar 5, 2021
@jkotas
Copy link
Member

jkotas commented Mar 5, 2021

#49066 fixed the x86-specific failure. The test is still disabled for JitStress against this issue.

@jkotas jkotas reopened this Mar 5, 2021
@jkotas jkotas removed blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' blocking-outerloop Blocking the 'runtime-coreclr outerloop' and 'runtime-libraries-coreclr outerloop' runs labels Mar 5, 2021
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Apr 3, 2021
@ghost ghost closed this as completed in #50676 Apr 3, 2021
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Apr 3, 2021
@ghost ghost locked as resolved and limited conversation to collaborators May 12, 2021
@VincentBu
Copy link
Contributor

Failed again in runtime-coreclr jitstress 20210623.2

Failed test:

CoreCLR windows x86 Checked jitstress1 @ Windows.10.Amd64.Open

- readytorun\\coreroot_determinism\\coreroot_determinism\\coreroot_determinism.cmd

Error message:

1 / 1 (100%, 1 failed): failed in 51020 msecs, exit code -1073740286 = 0xC0000602, expected 0: corerun.exe C:\h\w\B28D0968\p\crossgen2\crossgen2.dll @C:\h\w\B28D0968\w\BFBC0A5A\e\readytorun\coreroot_determinism\coreroot_determinism\seed2\CPAOT-ret.out\System.Private.CoreLib.dll.rsp
!! Assert failure(PID 6716 [0x00001a3c], Thread: 7148 [0x1bec]): !PreemptiveGCDisabled()
!! CORECLR! Thread::DetachThread + 0xCF (0x70699d7d)
!! CORECLR! TlsDestructionMonitor::~TlsDestructionMonitor + 0x8A (0x70b37920)
!! CORECLR! _dyn_tls_dtor + 0x86 (0x70b496e6)
!! NTDLL! RtlDecompressBuffer + 0xDE (0x7763ea4e)
!! NTDLL! LdrShutdownThread + 0x386 (0x7760eea6)
!! NTDLL! LdrSetAppCompatDllRedirectionCallback + 0x1052A (0x77663c5a)
!! NTDLL! LdrShutdownProcess + 0x15F (0x77622d7f)
!! NTDLL! RtlExitUserProcess + 0x96 (0x77623886)
!! KERNEL32! ExitProcess + 0x13 (0x7747b3e3)
!! CORECLR! exit_or_terminate_process + 0x38 (0x70b7f0e8)
!!     File: D:\workspace_work\1\s\src\coreclr\vm\threads.cpp Line: 981
!!     Image: C:\h\w\B28D0968\p\corerun.exe

Return code:      1
Raw output file:      C:\h\w\B28D0968\w\BFBC0A5A\uploads\Reports\readytorun.coreroot_determinism\coreroot_determinism\coreroot_determinism.output.txt
Raw output:
BEGIN EXECUTION
"C:\h\w\B28D0968\p\corerun.exe"  coreroot_determinism.dll
================================== Compiling with seed 1 ==================================
Locating output folder C:\h\w\B28D0968\w\BFBC0A5A\e\readytorun\coreroot_determinism\coreroot_determinism\seed1
Deleting 0 output folders
Successfully deleted 0 output folders in 164 msecs
Using dotnet: C:\h\w\B28D0968\p\corerun.exe
1 / 1 (0%, 0 failed): launching: C:\h\w\B28D0968\p\corerun.exe C:\h\w\B28D0968\p\crossgen2\crossgen2.dll @C:\h\w\B28D0968\w\BFBC0A5A\e\readytorun\coreroot_determinism\coreroot_determinism\seed1\CPAOT-ret.out\System.Private.CoreLib.dll.rsp
1 / 1 (100%, 0 failed): succeeded in 51009 msecs: corerun.exe C:\h\w\B28D0968\p\crossgen2\crossgen2.dll @C:\h\w\B28D0968\w\BFBC0A5A\e\readytorun\coreroot_determinism\coreroot_determinism\seed1\CPAOT-ret.out\System.Private.CoreLib.dll.rsp
================================== Compiling with seed 2 ==================================
Locating output folder C:\h\w\B28D0968\w\BFBC0A5A\e\readytorun\coreroot_determinism\coreroot_determinism\seed2
Deleting 0 output folders
Successfully deleted 0 output folders in 181 msecs
Using dotnet: C:\h\w\B28D0968\p\corerun.exe
1 / 1 (0%, 0 failed): launching: C:\h\w\B28D0968\p\corerun.exe C:\h\w\B28D0968\p\crossgen2\crossgen2.dll @C:\h\w\B28D0968\w\BFBC0A5A\e\readytorun\coreroot_determinism\coreroot_determinism\seed2\CPAOT-ret.out\System.Private.CoreLib.dll.rsp
Compilation failed. C:\h\w\B28D0968\p\corerun.exe C:\h\w\B28D0968\p\R2RTest\R2RTest.dll compile-directory -cr C:\h\w\B28D0968\p -in TestAssemblies --nojit --noexe --large-bubble --release --nocleanup -out seed2 failed with exit code 1
Expected: 100
Actual: 1
END EXECUTION - FAILED
FAILED
Test Harness Exitcode is : 1
To run the test:

set CORE_ROOT=C:\h\w\B28D0968\p
C:\h\w\B28D0968\w\BFBC0A5A\e\readytorun\coreroot_determinism\coreroot_determinism\coreroot_determinism.cmd
Expected: True
Actual:   False


Stack trace
   at readytorun_coreroot_determinism._coreroot_determinism_coreroot_determinism_._coreroot_determinism_coreroot_determinism_cmd() in readytorun.coreroot_determinism.XUnitWrapper.dll:token 0x6000004+0x284

@davidwrighton davidwrighton reopened this Jul 1, 2021
@davidwrighton
Copy link
Member

I've identified the Windows X86 assert here. It is an assert caused by incorrect handling of stack_limit. See upcoming PR for details

@davidwrighton
Copy link
Member

PR to fix the intermittent failure is #54986

@davidwrighton
Copy link
Member

I've checked in #54986 which should fix the intermittent assertion failure seen on x86, and hopefully some of the intemittent failures seen on other architectures.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.