Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Assertion failed '!"Missing flags on tree"' #95042

Closed
BruceForstall opened this issue Nov 20, 2023 · 4 comments
Closed

Assertion failed '!"Missing flags on tree"' #95042

BruceForstall opened this issue Nov 20, 2023 · 4 comments
Assignees
@AndyAyersMS
Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI JitStress CLR JIT issues involving JIT internal stress modes
Milestone
9.0.0

Comments

@BruceForstall
Copy link
Member

runtime-coreclr libraries-pgo
net9.0-windows-Release-x64-CoreCLR_checked-jitosr_stress_random-Windows.10.Amd64.Open
System.Configuration.ConfigurationManager.Tests

https://dev.azure.com/dnceng-public/public/_build/results?buildId=475007&view=ms.vss-test-web.build-test-results-tab&runId=10851678&paneView=debug&resultId=203727

C:\h\w\B80F097F\w\9E5F08FF\e>set DOTNET 
DOTNET_JitRandomOnStackReplacement=15
DOTNET_OSR_HitLimit=2
DOTNET_TC_OnStackReplacement=1
DOTNET_TC_OnStackReplacement_InitialCounter=1
DOTNET_TC_QuickJitForLoops=1
DOTNET_TieredCompilation=1

C:\h\w\B80F097F\w\9E5F08FF\e>call RunTests.cmd --runtime-path C:\h\w\B80F097F\p 
----- start Sun 11/19/2023 17:12:04.78 ===============  To repro directly: =====================================================
pushd C:\h\w\B80F097F\w\9E5F08FF\e\
"C:\h\w\B80F097F\p\dotnet.exe" exec --runtimeconfig System.Configuration.ConfigurationManager.Tests.runtimeconfig.json --depsfile System.Configuration.ConfigurationManager.Tests.deps.json xunit.console.dll System.Configuration.ConfigurationManager.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing 
popd
===========================================================================================================

C:\h\w\B80F097F\w\9E5F08FF\e>"C:\h\w\B80F097F\p\dotnet.exe" exec --runtimeconfig System.Configuration.ConfigurationManager.Tests.runtimeconfig.json --depsfile System.Configuration.ConfigurationManager.Tests.deps.json xunit.console.dll System.Configuration.ConfigurationManager.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing  
  Discovering: System.Configuration.ConfigurationManager.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Configuration.ConfigurationManager.Tests (found 512 of 515 test cases)
  Starting:    System.Configuration.ConfigurationManager.Tests (parallel test collections = on, max threads = 4)
Missing flags on tree [000189]: --X-------
               [000189] --C-G------                         *  CALL      void   System.Xml.XmlTextReaderImpl:.ctor(System.Xml.XmlNameTable):this
               [000145] ----------- this                    +--*  LCL_VAR   ref    V18 tmp11        
               [000166] ----------- arg1                    \--*  LCL_VAR   ref    V19 tmp12        

Assert failure(PID 10928 [0x00002ab0], Thread: 10932 [0x2ab4]): Assertion failed '!"Missing flags on tree"' in 'System.Configuration.XmlUtil:.ctor(System.IO.Stream,System.String,ubyte,System.Configuration.ConfigurationSchemaErrors):this' during 'Post-Morph' (IL size 169; hash 0x3139d2a3; Tier1-OSR)

    File: D:\a\_work\1\s\src\coreclr\jit\fgdiagnostic.cpp Line: 3399
    Image: C:\h\w\B80F097F\p\dotnet.exe

----- end Sun 11/19/2023 17:12:24.03 ----- exit code -1073740286 ----------------------------------------------------------

Also fails linux-arm64:

+ grep DOTNET
DOTNET_TC_QuickJitForLoops=1
DOTNET_EnableCrashReport=1
DOTNET_TieredCompilation=1
DOTNET_JitRandomOnStackReplacement=15
DOTNET_TC_OnStackReplacement=1
DOTNET_OSR_HitLimit=2
DOTNET_DbgMiniDumpName=/home/helixbot/dotnetbuild/dumps/coredump.%d.dmp
DOTNET_DbgEnableMiniDump=1
DOTNET_TC_OnStackReplacement_InitialCounter=1
+ ./RunTests.sh --runtime-path /root/helix/work/correlation
----- start Sun Nov 19 16:58:39 UTC 2023 =============== To repro directly: =====================================================
pushd .
/root/helix/work/correlation/dotnet exec --runtimeconfig System.Configuration.ConfigurationManager.Tests.runtimeconfig.json --depsfile System.Configuration.ConfigurationManager.Tests.deps.json xunit.console.dll System.Configuration.ConfigurationManager.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing 
popd
===========================================================================================================
/root/helix/work/workitem/e /root/helix/work/workitem/e
  Discovering: System.Configuration.ConfigurationManager.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Configuration.ConfigurationManager.Tests (found 510 of 515 test cases)
  Starting:    System.Configuration.ConfigurationManager.Tests (parallel test collections = on, max threads = 2)
Missing flags on tree [000189]: --X-------
               [000189] --C-G------                         *  CALL      void   System.Xml.XmlTextReaderImpl:.ctor(System.Xml.XmlNameTable):this
               [000145] ----------- this                    +--*  LCL_VAR   ref    V18 tmp11        
               [000166] ----------- arg1                    \--*  LCL_VAR   ref    V19 tmp12        

Assert failure(PID 28 [0x0000001c], Thread: 28 [0x001c]): Assertion failed '!"Missing flags on tree"' in 'System.Configuration.XmlUtil:.ctor(System.IO.Stream,System.String,ubyte,System.Configuration.ConfigurationSchemaErrors):this' during 'Post-Morph' (IL size 169; hash 0x3139d2a3; Tier1-OSR)

    File: /__w/1/s/src/coreclr/jit/fgdiagnostic.cpp Line: 3399
    Image: /root/helix/work/correlation/dotnet

[createdump] Gathering state for process 28 dotnet
[createdump] Crashing thread 001c signal 6 (0006)
apply_reg_state: ip and cfa unchanged; stopping here (ip=0xff728e53f228)
apply_reg_state: ip and cfa unchanged; stopping here (ip=0xff728e53f228)
[createdump] Writing crash report to file /home/helixbot/dotnetbuild/dumps/coredump.28.dmp.crashreport.json
[createdump] Crash report successfully written
[createdump] Writing minidump with heap to file /home/helixbot/dotnetbuild/dumps/coredump.28.dmp
[createdump] Written 181579776 bytes (44331 pages) to core file
[createdump] Target process is alive
[createdump] Dump successfully written in 421ms
waitpid() returned successfully (wstatus 00000000) WEXITSTATUS 0 WTERMSIG 0
./RunTests.sh: line 204:    28 Aborted                 (core dumped) "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Configuration.ConfigurationManager.Tests.runtimeconfig.json --depsfile System.Configuration.ConfigurationManager.Tests.deps.json xunit.console.dll System.Configuration.ConfigurationManager.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing $RSP_FILE
/root/helix/work/workitem/e
----- end Sun Nov 19 16:58:49 UTC 2023 ----- exit code 134 ----------------------------------------------------------

@AndyAyersMS @dotnet/jit-contrib
@BruceForstall BruceForstall added JitStress CLR JIT issues involving JIT internal stress modes area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI blocking-clean-ci-optional Blocking optional rolling runs labels Nov 20, 2023
@BruceForstall BruceForstall added this to the 9.0.0 milestone Nov 20, 2023
@ghost
Copy link

ghost commented Nov 20, 2023

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

runtime-coreclr libraries-pgo
net9.0-windows-Release-x64-CoreCLR_checked-jitosr_stress_random-Windows.10.Amd64.Open
System.Configuration.ConfigurationManager.Tests

https://dev.azure.com/dnceng-public/public/_build/results?buildId=475007&view=ms.vss-test-web.build-test-results-tab&runId=10851678&paneView=debug&resultId=203727

C:\h\w\B80F097F\w\9E5F08FF\e>set DOTNET 
DOTNET_JitRandomOnStackReplacement=15
DOTNET_OSR_HitLimit=2
DOTNET_TC_OnStackReplacement=1
DOTNET_TC_OnStackReplacement_InitialCounter=1
DOTNET_TC_QuickJitForLoops=1
DOTNET_TieredCompilation=1

C:\h\w\B80F097F\w\9E5F08FF\e>call RunTests.cmd --runtime-path C:\h\w\B80F097F\p 
----- start Sun 11/19/2023 17:12:04.78 ===============  To repro directly: =====================================================
pushd C:\h\w\B80F097F\w\9E5F08FF\e\
"C:\h\w\B80F097F\p\dotnet.exe" exec --runtimeconfig System.Configuration.ConfigurationManager.Tests.runtimeconfig.json --depsfile System.Configuration.ConfigurationManager.Tests.deps.json xunit.console.dll System.Configuration.ConfigurationManager.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing 
popd
===========================================================================================================

C:\h\w\B80F097F\w\9E5F08FF\e>"C:\h\w\B80F097F\p\dotnet.exe" exec --runtimeconfig System.Configuration.ConfigurationManager.Tests.runtimeconfig.json --depsfile System.Configuration.ConfigurationManager.Tests.deps.json xunit.console.dll System.Configuration.ConfigurationManager.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing  
  Discovering: System.Configuration.ConfigurationManager.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Configuration.ConfigurationManager.Tests (found 512 of 515 test cases)
  Starting:    System.Configuration.ConfigurationManager.Tests (parallel test collections = on, max threads = 4)
Missing flags on tree [000189]: --X-------
               [000189] --C-G------                         *  CALL      void   System.Xml.XmlTextReaderImpl:.ctor(System.Xml.XmlNameTable):this
               [000145] ----------- this                    +--*  LCL_VAR   ref    V18 tmp11        
               [000166] ----------- arg1                    \--*  LCL_VAR   ref    V19 tmp12        

Assert failure(PID 10928 [0x00002ab0], Thread: 10932 [0x2ab4]): Assertion failed '!"Missing flags on tree"' in 'System.Configuration.XmlUtil:.ctor(System.IO.Stream,System.String,ubyte,System.Configuration.ConfigurationSchemaErrors):this' during 'Post-Morph' (IL size 169; hash 0x3139d2a3; Tier1-OSR)

    File: D:\a\_work\1\s\src\coreclr\jit\fgdiagnostic.cpp Line: 3399
    Image: C:\h\w\B80F097F\p\dotnet.exe

----- end Sun 11/19/2023 17:12:24.03 ----- exit code -1073740286 ----------------------------------------------------------

Also fails linux-arm64:

+ grep DOTNET
DOTNET_TC_QuickJitForLoops=1
DOTNET_EnableCrashReport=1
DOTNET_TieredCompilation=1
DOTNET_JitRandomOnStackReplacement=15
DOTNET_TC_OnStackReplacement=1
DOTNET_OSR_HitLimit=2
DOTNET_DbgMiniDumpName=/home/helixbot/dotnetbuild/dumps/coredump.%d.dmp
DOTNET_DbgEnableMiniDump=1
DOTNET_TC_OnStackReplacement_InitialCounter=1
+ ./RunTests.sh --runtime-path /root/helix/work/correlation
----- start Sun Nov 19 16:58:39 UTC 2023 =============== To repro directly: =====================================================
pushd .
/root/helix/work/correlation/dotnet exec --runtimeconfig System.Configuration.ConfigurationManager.Tests.runtimeconfig.json --depsfile System.Configuration.ConfigurationManager.Tests.deps.json xunit.console.dll System.Configuration.ConfigurationManager.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing 
popd
===========================================================================================================
/root/helix/work/workitem/e /root/helix/work/workitem/e
  Discovering: System.Configuration.ConfigurationManager.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Configuration.ConfigurationManager.Tests (found 510 of 515 test cases)
  Starting:    System.Configuration.ConfigurationManager.Tests (parallel test collections = on, max threads = 2)
Missing flags on tree [000189]: --X-------
               [000189] --C-G------                         *  CALL      void   System.Xml.XmlTextReaderImpl:.ctor(System.Xml.XmlNameTable):this
               [000145] ----------- this                    +--*  LCL_VAR   ref    V18 tmp11        
               [000166] ----------- arg1                    \--*  LCL_VAR   ref    V19 tmp12        

Assert failure(PID 28 [0x0000001c], Thread: 28 [0x001c]): Assertion failed '!"Missing flags on tree"' in 'System.Configuration.XmlUtil:.ctor(System.IO.Stream,System.String,ubyte,System.Configuration.ConfigurationSchemaErrors):this' during 'Post-Morph' (IL size 169; hash 0x3139d2a3; Tier1-OSR)

    File: /__w/1/s/src/coreclr/jit/fgdiagnostic.cpp Line: 3399
    Image: /root/helix/work/correlation/dotnet

[createdump] Gathering state for process 28 dotnet
[createdump] Crashing thread 001c signal 6 (0006)
apply_reg_state: ip and cfa unchanged; stopping here (ip=0xff728e53f228)
apply_reg_state: ip and cfa unchanged; stopping here (ip=0xff728e53f228)
[createdump] Writing crash report to file /home/helixbot/dotnetbuild/dumps/coredump.28.dmp.crashreport.json
[createdump] Crash report successfully written
[createdump] Writing minidump with heap to file /home/helixbot/dotnetbuild/dumps/coredump.28.dmp
[createdump] Written 181579776 bytes (44331 pages) to core file
[createdump] Target process is alive
[createdump] Dump successfully written in 421ms
waitpid() returned successfully (wstatus 00000000) WEXITSTATUS 0 WTERMSIG 0
./RunTests.sh: line 204:    28 Aborted                 (core dumped) "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Configuration.ConfigurationManager.Tests.runtimeconfig.json --depsfile System.Configuration.ConfigurationManager.Tests.deps.json xunit.console.dll System.Configuration.ConfigurationManager.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing $RSP_FILE
/root/helix/work/workitem/e
----- end Sun Nov 19 16:58:49 UTC 2023 ----- exit code 134 ----------------------------------------------------------

@AndyAyersMS @dotnet/jit-contrib

Author: BruceForstall
Assignees: -
Labels:

JitStress, area-CodeGen-coreclr, blocking-clean-ci-optional
Milestone: 9.0.0

@AndyAyersMS
Copy link
Member

So far I can't repro this, it also did not recur in last weekend's lab runs. Will keep trying.

@AndyAyersMS AndyAyersMS removed the blocking-clean-ci-optional Blocking optional rolling runs label Nov 28, 2023
@AndyAyersMS
Copy link
Member

Got it to repro -- looks like the same issue that I fixed over in #94689 via 3882492 -- the fgEntryBB was getting lost.

@AndyAyersMS
Copy link
Member

Fixed via #94689

@github-actions github-actions bot locked and limited conversation to collaborators Dec 29, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@AndyAyersMS AndyAyersMS

Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI JitStress CLR JIT issues involving JIT internal stress modes
Projects
None yet
Milestone
9.0.0
Development

No branches or pull requests
2 participants
@AndyAyersMS @BruceForstall