[libraries-jitstress] System.Runtime.Intrinsics.Tests.Vectors.Vector512Tests.Vector512DoubleEqualsNonCanonicalNaNTest fails in CI on win-x64

[jakobbotsch]

Example pipeline run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=727164&view=results
Example console log: https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-heads-main-f251c096fcce46c899/System.Runtime.Intrinsics.Tests/1/console.8f31b9de.log?helixlogtype=result

  Starting:    System.Runtime.Intrinsics.Tests (parallel test collections = on [4 threads], stop on fail = off)
    System.Runtime.Intrinsics.Tests.Vectors.Vector512Tests.Vector512DoubleEqualsNonCanonicalNaNTest [FAIL]
      Assert.True() Failure
      Expected: True
      Actual:   False
      Stack Trace:
        /_/src/libraries/System.Runtime.Intrinsics/tests/Vectors/Vector512Tests.cs(5013,0): at System.Runtime.Intrinsics.Tests.Vectors.Vector512Tests.Vector512DoubleEqualsNonCanonicalNaNTest()
           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
        /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs(57,0): at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
    System.Runtime.Intrinsics.Tests.Vectors.Vector512Tests.Vector512DoubleEqualsNaNTest [FAIL]
      Assert.True() Failure
      Expected: True
      Actual:   False
      Stack Trace:
        /_/src/libraries/System.Runtime.Intrinsics/tests/Vectors/Vector512Tests.cs(4985,0): at System.Runtime.Intrinsics.Tests.Vectors.Vector512Tests.Vector512DoubleEqualsNaNTest()
           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
        /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs(57,0): at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
  Finished:    System.Runtime.Intrinsics.Tests

cc @tannergooding @dotnet/jit-contrib

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[jakobbotsch]

This does not repro on my CPU. Presumably it needs AVX512 to repro. @tannergooding @EgorBo can one of you take a look?

[tannergooding]

We're generating the wrong size kortest here (picking kortestw when we should be picking kortestb), looking to see what the issue is

[tannergooding]

PR is up here: #104364

[jakobbotsch]

Thanks @tannergooding!

[v-wenyuxu]

Failed in: runtime-coreclr libraries-jitstress 20240703.1

Failed tests:

net9.0-linux-Release-x64-tailcallstress-Ubuntu.2204.Amd64.Open
    - System.Runtime.Intrinsics.Tests.Vectors.Vector512Tests.Vector512DoubleEqualsNonCanonicalNaNTest
    - System.Runtime.Intrinsics.Tests.Vectors.Vector512Tests.Vector512DoubleEqualsNaNTest

Error message:

 Assert.True() Failure
Expected: True
Actual:   False

Stack trace:

   at System.Runtime.Intrinsics.Tests.Vectors.Vector512Tests.Vector512DoubleEqualsNonCanonicalNaNTest() in /_/src/libraries/System.Runtime.Intrinsics/tests/Vectors/Vector512Tests.cs:line 5013
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs:line 57