System.Net.Security.Tests.SslStreamTlsResumeTests failing

[kunalspathak]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=705166
Build error leg or test failing: System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds
Pull request: #103159

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": ["ClientCert_DefaultValue_ResumeSucceeds", "Assert.Equal() Failure: Values differ"],
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Mostly due to #102656
cc: @rzikm

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=705166
Error message validated: [ClientCert_DefaultValue_ResumeSucceeds Assert.Equal() Failure: Values differ]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/13/2024 7:49:03 AM UTC

Report

Build	Definition	Test	Pull Request
705793	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#101095
705631	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103369
706214	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103382
706355	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103226
705744	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#102834
706320	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#102523
706349	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103399
706344	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103375
706113	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#102736
706298	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#101969
706149	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103143
706263	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103331
705570	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103332
705479	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103241
706227	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103394
706187	dotnet/runtime	System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Proxy_Test.MultiProxy_PAC_Failover_Succeeds	#103364
706174	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientDisableTlsResume_Succeeds	#103392
706155	dotnet/runtime	JIT/opt/ObjectStackAllocation/ObjectStackAllocationTests/ObjectStackAllocationTests.cmd	#103361
706094	dotnet/runtime	JIT/Regression/JitBlue/GitHub_18770/GitHub_18770/GitHub_18770.dll	#103382
706138	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103391
706088	dotnet/runtime	JIT/Regression/JitBlue/GitHub_18770/GitHub_18770/GitHub_18770.dll	#103382
706127	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103390
706061	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103387
705940	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103206
706039	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103358
706025	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103386
705911	dotnet/runtime	WasmTestOnV8-ST-System.Runtime.Tests.WorkItemExecution	#103381
705977	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103076
705958	dotnet/runtime	System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Proxy_Test.MultiProxy_PAC_Failover_Succeeds	#103364
705952	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#102364
705662	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#99183
705881	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103375
705762	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#100334
705726	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103374
705672	dotnet/runtime	JIT/opt/ObjectStackAllocation/ObjectStackAllocationTests/ObjectStackAllocationTests.cmd	#103361
705719	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103322
705463	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103356
705559	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103279
705536	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103358
705451	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#101876
705416	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103346
705403	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103058
705367	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103352
705344	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#102882
705232	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103195
705321	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103191
705312	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#100356
705283	dotnet/runtime	System.Net.Security.Tests.SslStreamTlsResumeTests.ClientCert_DefaultValue_ResumeSucceeds	#103346

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	48

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

[rzikm]

hmm, looks like TLS Resume is not working on TLS 1.3 on Ubuntu 18.04? I will take a look

[rzikm]

@v-wenyuxu is it really the same issue? The test have been disabled, and I don't see this failure in the text of your comment.

[rzikm]

Added some debug print on the related platform and there seems to be something fishy happening in OpenSSL

helixbot@77d5709caac1:/test$ ./dotnet exec --runtimeconfig System.Net.Security.Tests.runtimeconfig.json --depsfile System.Net.Security.Tests.deps.json xunit.console.dll System.Net.Security.Tests.dll -xml testResults.xml -nologo -notrait category=OuterLoop -notrait category=failing -method *NoClientCert_DefaultValue_ResumeSucceeds
  Discovering: System.Net.Security.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Net.Security.Tests (found 1 of 514 test case)
  Starting:    System.Net.Security.Tests (parallel test collections = on [8 threads], stop on fail = off)
Session reused: False
Session reused: False
Adding session to dade675368e24722aa72162292629e68
Adding session to dade675368e24722aa72162292629e68
removing session to dade675368e24722aa72162292629e68
Session reused: False
Session reused: False
setting session
removing session to dade675368e24722aa72162292629e68
Session reused: True
Session reused: True
Adding session to dade675368e24722aa72162292629e68
Session reused: True
Session reused: True
setting session
removing session to @_f�
Session reused: True
Session reused: True
Adding session to @_f�
Session reused: True
Session reused: True
setting session
Session reused: False
Session reused: False
    System.Net.Security.Tests.SslStreamTlsResumeTests.NoClientCert_DefaultValue_ResumeSucceeds(sslProtocol: Tls13) [FAIL]
      Assert.Equal() Failure: Values differ
      Expected: True
      Actual:   False

[rzikm]

The same tests work fine on Ubuntu 20.04 ARM, I believe this is due to a bug in OpenSSL, the version in 18.04 image is stuck on OpenSSL 1.1.1 (no letter in the version) and the image does not receive updates due to being out of support. I think we should keep the problematic combination disabled for this test.