[wasm] System.Runtime.Tests exiting silently on main

[kg]

When I do a clean build on commits 2b185ac, 3816ed1 or a78f9bd, System.Runtime.Tests silently exits with an exit code of 0 (well, to be precise, v8 exits) during execution (not immediately after starting the test run, it's part way in somewhere). The output looks like this:

like so:

  info: Using js engine V8 from path /home/kate/.jsvu/v8
  info: V8 version: 11.3.175
  info: Running /home/kate/.jsvu/v8 --expose_wasm --stack-trace-limit=1000 --experimental-wasm-bigint test-main.js -- --run WasmTestRunner.dll System.Runtime.Tests.dll -notrait category=AdditionalTimezoneChecks -notrait category=OuterLoop -notrait category=failing
        
  info: Warning: unknown flag --experimental-wasm-bigint.
  info: Incoming arguments: --run WasmTestRunner.dll System.Runtime.Tests.dll -notrait category=AdditionalTimezoneChecks -notrait category=OuterLoop -notrait category=failing
  info: Application arguments: --run WasmTestRunner.dll System.Runtime.Tests.dll -notrait category=AdditionalTimezoneChecks -notrait category=OuterLoop -notrait category=failing
  info: console.info: Initializing dotnet version 8.0.0-dev commit hash 5629069f8f1c597d382997ad948233b9fb8688f5
  info: Discovering: System.Runtime.Tests.dll (method display = ClassAndMethod, method display options = None)
  info: Discovered:  System.Runtime.Tests.dll (found 8979 of 9172 test cases)
  info: Using random seed for test cases: 1887416109
  info: Using random seed for collections: 1887416109
  info: Starting:    System.Runtime.Tests.dll
  info: Process v8 exited with 0
        
  info: Waiting to flush log messages with a timeout of 120 secs ..
  info: Application has finished with exit code: 0
  XHarness exit code: 0
  ~/Projects/dotnet-runtime-wasm/artifacts/bin/System.Runtime.Tests/Release/net8.0-browser/browser-wasm/AppBundle ~/Projects/dotnet-runtime-wasm/src/libraries/System.Runtime/tests
  ----- end Tue 14 Mar 2023 11:11:12 PM PDT ----- exit code 0 ----------------------------------------------------------
  XHarness artifacts: /home/kate/Projects/dotnet-runtime-wasm/artifacts/bin/System.Runtime.Tests/Release/net8.0-browser/browser-wasm/AppBundle/xharness-output

Build succeeded.

I'm able to successfully run browser-bench and dotnet/performance benchmarks on current main so it's not completely broken, but something seems seriously wrong if tests are able to "succeed" when crashing. Historically our infra has been good about always exiting with a nonzero exit code when something goes wrong.

I thought it might be v8, so I updated my previous build of v8 to latest using jsvu and saw no difference.

When changing commits I deleted .dotnet and artifacts, re-provisioned emsdk, and pkill -9'd dotnet.

Does not appear to reproduce if I check out 8d9ca3f, so it might be broken by #81215 ?

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

---

When I do a clean build on commits 2b185ac, 3816ed1 or a78f9bd, System.Runtime.Tests silently exits with an exit code of 0 (well, to be precise, v8 exits) during execution (not immediately after starting the test run, it's part way in somewhere). The output looks like this:

like so:

  info: Using js engine V8 from path /home/kate/.jsvu/v8
  info: V8 version: 11.3.175
  info: Running /home/kate/.jsvu/v8 --expose_wasm --stack-trace-limit=1000 --experimental-wasm-bigint test-main.js -- --run WasmTestRunner.dll System.Runtime.Tests.dll -notrait category=AdditionalTimezoneChecks -notrait category=OuterLoop -notrait category=failing
        
  info: Warning: unknown flag --experimental-wasm-bigint.
  info: Incoming arguments: --run WasmTestRunner.dll System.Runtime.Tests.dll -notrait category=AdditionalTimezoneChecks -notrait category=OuterLoop -notrait category=failing
  info: Application arguments: --run WasmTestRunner.dll System.Runtime.Tests.dll -notrait category=AdditionalTimezoneChecks -notrait category=OuterLoop -notrait category=failing
  info: console.info: Initializing dotnet version 8.0.0-dev commit hash 5629069f8f1c597d382997ad948233b9fb8688f5
  info: Discovering: System.Runtime.Tests.dll (method display = ClassAndMethod, method display options = None)
  info: Discovered:  System.Runtime.Tests.dll (found 8979 of 9172 test cases)
  info: Using random seed for test cases: 1887416109
  info: Using random seed for collections: 1887416109
  info: Starting:    System.Runtime.Tests.dll
  info: Process v8 exited with 0
        
  info: Waiting to flush log messages with a timeout of 120 secs ..
  info: Application has finished with exit code: 0
  XHarness exit code: 0
  ~/Projects/dotnet-runtime-wasm/artifacts/bin/System.Runtime.Tests/Release/net8.0-browser/browser-wasm/AppBundle ~/Projects/dotnet-runtime-wasm/src/libraries/System.Runtime/tests
  ----- end Tue 14 Mar 2023 11:11:12 PM PDT ----- exit code 0 ----------------------------------------------------------
  XHarness artifacts: /home/kate/Projects/dotnet-runtime-wasm/artifacts/bin/System.Runtime.Tests/Release/net8.0-browser/browser-wasm/AppBundle/xharness-output

Build succeeded.

I'm able to successfully run browser-bench and dotnet/performance benchmarks on current main so it's not completely broken, but something seems seriously wrong if tests are able to "succeed" when crashing. Historically our infra has been good about always exiting with a nonzero exit code when something goes wrong.

I thought it might be v8, so I updated my previous build of v8 to latest using jsvu and saw no difference.

When changing commits I deleted .dotnet and artifacts, re-provisioned emsdk, and pkill -9'd dotnet.

Does not appear to reproduce if I check out 8d9ca3f, so it might be broken by #81215 ?

Author:	kg
Assignees:	-
Labels:	arch-wasm
Milestone:	-

[kg]

I run System.Runtime.Tests with this command line:
/home/kate/Projects/dotnet-runtime-wasm/dotnet.sh build /bl /t:Test /p:TargetOS=browser /p:TargetArchitecture=wasm /p:Configuration=Release /home/kate/Projects/dotnet-runtime-wasm/src/libraries/System.Runtime/tests/System.Runtime.Tests.csproj /p:EnableAggressiveTrimming=true /p:RunAOTCompilation=false

[ghost]

Tagging subscribers to this area: @dotnet/area-system-runtime
See info in area-owners.md if you want to be subscribed.

Issue Details

---

When I do a clean build on commits 2b185ac, 3816ed1 or a78f9bd, System.Runtime.Tests silently exits with an exit code of 0 (well, to be precise, v8 exits) during execution (not immediately after starting the test run, it's part way in somewhere). The output looks like this:

like so:

  info: Using js engine V8 from path /home/kate/.jsvu/v8
  info: V8 version: 11.3.175
  info: Running /home/kate/.jsvu/v8 --expose_wasm --stack-trace-limit=1000 --experimental-wasm-bigint test-main.js -- --run WasmTestRunner.dll System.Runtime.Tests.dll -notrait category=AdditionalTimezoneChecks -notrait category=OuterLoop -notrait category=failing
        
  info: Warning: unknown flag --experimental-wasm-bigint.
  info: Incoming arguments: --run WasmTestRunner.dll System.Runtime.Tests.dll -notrait category=AdditionalTimezoneChecks -notrait category=OuterLoop -notrait category=failing
  info: Application arguments: --run WasmTestRunner.dll System.Runtime.Tests.dll -notrait category=AdditionalTimezoneChecks -notrait category=OuterLoop -notrait category=failing
  info: console.info: Initializing dotnet version 8.0.0-dev commit hash 5629069f8f1c597d382997ad948233b9fb8688f5
  info: Discovering: System.Runtime.Tests.dll (method display = ClassAndMethod, method display options = None)
  info: Discovered:  System.Runtime.Tests.dll (found 8979 of 9172 test cases)
  info: Using random seed for test cases: 1887416109
  info: Using random seed for collections: 1887416109
  info: Starting:    System.Runtime.Tests.dll
  info: Process v8 exited with 0
        
  info: Waiting to flush log messages with a timeout of 120 secs ..
  info: Application has finished with exit code: 0
  XHarness exit code: 0
  ~/Projects/dotnet-runtime-wasm/artifacts/bin/System.Runtime.Tests/Release/net8.0-browser/browser-wasm/AppBundle ~/Projects/dotnet-runtime-wasm/src/libraries/System.Runtime/tests
  ----- end Tue 14 Mar 2023 11:11:12 PM PDT ----- exit code 0 ----------------------------------------------------------
  XHarness artifacts: /home/kate/Projects/dotnet-runtime-wasm/artifacts/bin/System.Runtime.Tests/Release/net8.0-browser/browser-wasm/AppBundle/xharness-output

Build succeeded.

I'm able to successfully run browser-bench and dotnet/performance benchmarks on current main so it's not completely broken, but something seems seriously wrong if tests are able to "succeed" when crashing. Historically our infra has been good about always exiting with a nonzero exit code when something goes wrong.

I thought it might be v8, so I updated my previous build of v8 to latest using jsvu and saw no difference.

When changing commits I deleted .dotnet and artifacts, re-provisioned emsdk, and pkill -9'd dotnet.

Does not appear to reproduce if I check out 8d9ca3f, so it might be broken by #81215 ?

Author:	kg
Assignees:	-
Labels:	arch-wasm, area-System.Runtime, untriaged
Milestone:	-

[kg]

This appears to be https://bugs.chromium.org/p/v8/issues/detail?id=13823 somehow