SignedXmlTest.VerifyXmlResolver sometimes failing due to failing to connect to localhost. #74115

danmoseley · 2022-08-18T00:31:20Z

This was just introduced in #73676 and is failing at least 4 times a day. @GrabYourPitchforks

History: https://dev.azure.com/dnceng/public/_build/results?buildId=1949830&view=ms.vss-test-web.build-test-results-tab&runId=50186248&resultId=193531&paneView=history

example https://dev.azure.com/dnceng/public/_build/results?buildId=1949830&view=ms.vss-test-web.build-test-results-tab&runId=50186248&resultId=193531&paneView=debug

System.Security.Cryptography.Xml.Tests.SignedXmlTest.VerifyXmlResolver [[Console]](https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-74094-merge-bd71821e4e9c4ed7aa/System.Security.Cryptography.Xml.Tests/1/console.5d095986.log?helixlogtype=result) [[Details]](https://dev.azure.com/dnceng//public/_build/results?buildId=1949830&view=ms.vss-test-web.build-test-results-tab&runId=50186248&resultId=193531&paneView=debug) [[Artifacts]](https://dev.azure.com/dnceng//public/_build/results?buildId=1949830&view=ms.vss-test-web.build-test-results-tab&runId=50186248&resultId=193531&paneView=dotnet-dnceng.dnceng-build-release-tasks.helix-test-information-tab) [[3.18% failure rate]](https://dev.azure.com/dnceng//public/_build/results?buildId=1949830&view=ms.vss-test-web.build-test-results-tab&runId=50186248&resultId=193531&paneView=history)
Data Driven SubResults
❌ System.Security.Cryptography.Xml.Tests.SignedXmlTest.VerifyXmlResolver(provideResolver: True)
Exception Message
System.Xml.XmlException : An error has occurred while opening external entity 'http://127.0.0.1:9000/': The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
---- System.Threading.Tasks.TaskCanceledException : The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
-------- System.TimeoutException : The operation was canceled.
------------ System.Threading.Tasks.TaskCanceledException : The operation was canceled.
---------------- System.IO.IOException : Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
-------------------- System.Net.Sockets.SocketException : The I/O operation has been aborted because of either a thread exit or an application request.
Stack Trace
   at System.Xml.XmlTextReaderImpl.Throw(Exception e) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 2778
   at System.Xml.XmlTextReaderImpl.PushExternalEntityOrSubset(String publicId, String systemId, Uri baseUri, String entityName) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 7984
   at System.Xml.XmlTextReaderImpl.PushExternalEntity(IDtdEntityInfo entity) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 8052
   at System.Xml.XmlTextReaderImpl.HandleGeneralEntityReference(String name, Boolean isInAttributeValue, Boolean pushFakeEntityIfNullResolver, Int32 entityStartLinePos) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 6144
   at System.Xml.XmlTextReaderImpl.HandleEntityReference(Boolean isInAttributeValue, EntityExpandType expandType, Int32& charRefEndPos) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 6069
   at System.Xml.XmlTextReaderImpl.ParseText(Int32& startPos, Int32& endPos, Int32& outOrChars) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 5690
   at System.Xml.XmlTextReaderImpl.FinishPartialValue() in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 5834
   at System.Xml.XmlTextReaderImpl.get_Value() in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 919
   at System.Xml.XmlLoader.LoadNode(Boolean skipOverWhitespace) in /_/src/libraries/System.Private.Xml/src/System/Xml/Dom/XmlLoader.cs:line 174
   at System.Xml.XmlLoader.LoadDocSequence(XmlDocument parentDoc) in /_/src/libraries/System.Private.Xml/src/System/Xml/Dom/XmlLoader.cs:line 60
   at System.Xml.XmlLoader.Load(XmlDocument doc, XmlReader reader, Boolean preserveWhitespace) in /_/src/libraries/System.Private.Xml/src/System/Xml/Dom/XmlLoader.cs:line 51
   at System.Xml.XmlDocument.Load(XmlReader reader) in /_/src/libraries/System.Private.Xml/src/System/Xml/Dom/XmlDocument.cs:line 1323
   at System.Security.Cryptography.Xml.Utils.PreProcessDocumentInput(XmlDocument document, XmlResolver xmlResolver, String baseUri) in /_/src/libraries/System.Security.Cryptography.Xml/src/System/Security/Cryptography/Xml/Utils.cs:line 222
   at System.Security.Cryptography.Xml.Reference.CalculateHashValue(XmlDocument document, CanonicalXmlNodeList refList) in /_/src/libraries/System.Security.Cryptography.Xml/src/System/Security/Cryptography/Xml/Reference.cs:line 386
   at System.Security.Cryptography.Xml.SignedXml.CheckDigestedReferences() in /_/src/libraries/System.Security.Cryptography.Xml/src/System/Security/Cryptography/Xml/SignedXml.cs:line 917
   at System.Security.Cryptography.Xml.SignedXml.CheckSignature(KeyedHashAlgorithm macAlg) in /_/src/libraries/System.Security.Cryptography.Xml/src/System/Security/Cryptography/Xml/SignedXml.cs:line 312
   at System.Security.Cryptography.Xml.Tests.SignedXmlTest.VerifyXmlResolver(Boolean provideResolver) in /_/src/libraries/System.Security.Cryptography.Xml/tests/SignedXmlTest.cs:line 1658
   at InvokeStub_SignedXmlTest.VerifyXmlResolver(Object, Object, IntPtr*)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
----- Inner Stack Trace -----
   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 632
   at System.Net.Http.HttpClient.GetStreamAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 352
   at System.Xml.XmlDownloadManager.GetNonFileStreamAsync(Uri uri, ICredentials credentials, IWebProxy proxy) in /_/src/libraries/System.Private.Xml/src/System/Xml/XmlDownloadManagerAsync.cs:line 43
   at System.Xml.XmlDownloadManager.GetStream(Uri uri, ICredentials credentials, IWebProxy proxy) in /_/src/libraries/System.Private.Xml/src/System/Xml/XmlDownloadManager.cs:line 22
   at System.Xml.XmlUrlResolver.GetEntity(Uri absoluteUri, String role, Type ofObjectToReturn) in /_/src/libraries/System.Private.Xml/src/System/Xml/XmlUrlResolver.cs:line 47
   at System.Xml.XmlTextReaderImpl.OpenAndPush(Uri uri) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 8023
   at System.Xml.XmlTextReaderImpl.PushExternalEntityOrSubset(String publicId, String systemId, Uri baseUri, String entityName) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 7969
----- Inner Stack Trace -----
----- Inner Stack Trace -----

at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnection.cs:line 836

at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 1091

at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/RedirectHandler.cs:line 30

at System.Net.Http.HttpClient.GetStreamAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 343

----- Inner Stack Trace -----

at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.Tasks.cs:line 1451

at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token) in /_/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.Tasks.cs:line 1346

at System.Net.Http.HttpConnection.InitialFillAsync(Boolean async) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnection.cs:line 1676

at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnection.cs:line 599

----- Inner Stack Trace -----


Failing Configuration
net7.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open

HttpListener was successfully started on 127.0.0.1:9000, but apparently did not respond promptly. That was on Windows. Looking at history, we also see failures on Android and Ubuntu, which may have a slightly different error:

                   Starting:    System.Security.Cryptography.Xml.Tests (parallel test collections = on, max threads = 1)
                       System.Xml.XmlException : An error has occurred while opening external entity 'http://127.0.0.1:9000/': The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
                       ---- System.Threading.Tasks.TaskCanceledException : The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
                       -------- System.TimeoutException : The operation was canceled.
                       ------------ System.Threading.Tasks.TaskCanceledException : The operation was canceled.
                       ---------------- System.IO.IOException : The response ended prematurely.

It's probably significant that this is not OS specific. @dotnet/ncl do you see anything obvious about the test here? Code:

runtime/src/libraries/System.Security.Cryptography.Xml/tests/SignedXmlTest.cs

Line 1603 in f140a12

listener = new HttpListener();

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

The text was updated successfully, but these errors were encountered:

ghost · 2022-08-18T00:31:27Z

Tagging subscribers to this area: @dotnet/area-system-xml
See info in area-owners.md if you want to be subscribed.

Issue Details

This was just introduced in #73676 and is failing at least 4 times a day. @GrabYourPitchforks

HttpListener was successfully started on 127.0.0.1:9000, but apparently did not respond promptly. Looking at history, we also see failures on Android, Ubuntu, which may have a slightly different error:

                   Starting:    System.Security.Cryptography.Xml.Tests (parallel test collections = on, max threads = 1)
                       System.Xml.XmlException : An error has occurred while opening external entity 'http://127.0.0.1:9000/': The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
                       ---- System.Threading.Tasks.TaskCanceledException : The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
                       -------- System.TimeoutException : The operation was canceled.
                       ------------ System.Threading.Tasks.TaskCanceledException : The operation was canceled.
                       ---------------- System.IO.IOException : The response ended prematurely.

https://dev.azure.com/dnceng/public/_build/results?buildId=1949830&view=ms.vss-test-web.build-test-results-tab&runId=50186248&resultId=193531&paneView=debug

System.Security.Cryptography.Xml.Tests.SignedXmlTest.VerifyXmlResolver [[Console]](https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-74094-merge-bd71821e4e9c4ed7aa/System.Security.Cryptography.Xml.Tests/1/console.5d095986.log?helixlogtype=result) [[Details]](https://dev.azure.com/dnceng//public/_build/results?buildId=1949830&view=ms.vss-test-web.build-test-results-tab&runId=50186248&resultId=193531&paneView=debug) [[Artifacts]](https://dev.azure.com/dnceng//public/_build/results?buildId=1949830&view=ms.vss-test-web.build-test-results-tab&runId=50186248&resultId=193531&paneView=dotnet-dnceng.dnceng-build-release-tasks.helix-test-information-tab) [[3.18% failure rate]](https://dev.azure.com/dnceng//public/_build/results?buildId=1949830&view=ms.vss-test-web.build-test-results-tab&runId=50186248&resultId=193531&paneView=history)
Data Driven SubResults
❌ System.Security.Cryptography.Xml.Tests.SignedXmlTest.VerifyXmlResolver(provideResolver: True)
Exception Message
System.Xml.XmlException : An error has occurred while opening external entity 'http://127.0.0.1:9000/': The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
---- System.Threading.Tasks.TaskCanceledException : The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
-------- System.TimeoutException : The operation was canceled.
------------ System.Threading.Tasks.TaskCanceledException : The operation was canceled.
---------------- System.IO.IOException : Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
-------------------- System.Net.Sockets.SocketException : The I/O operation has been aborted because of either a thread exit or an application request.
Stack Trace
   at System.Xml.XmlTextReaderImpl.Throw(Exception e) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 2778
   at System.Xml.XmlTextReaderImpl.PushExternalEntityOrSubset(String publicId, String systemId, Uri baseUri, String entityName) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 7984
   at System.Xml.XmlTextReaderImpl.PushExternalEntity(IDtdEntityInfo entity) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 8052
   at System.Xml.XmlTextReaderImpl.HandleGeneralEntityReference(String name, Boolean isInAttributeValue, Boolean pushFakeEntityIfNullResolver, Int32 entityStartLinePos) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 6144
   at System.Xml.XmlTextReaderImpl.HandleEntityReference(Boolean isInAttributeValue, EntityExpandType expandType, Int32& charRefEndPos) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 6069
   at System.Xml.XmlTextReaderImpl.ParseText(Int32& startPos, Int32& endPos, Int32& outOrChars) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 5690
   at System.Xml.XmlTextReaderImpl.FinishPartialValue() in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 5834
   at System.Xml.XmlTextReaderImpl.get_Value() in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 919
   at System.Xml.XmlLoader.LoadNode(Boolean skipOverWhitespace) in /_/src/libraries/System.Private.Xml/src/System/Xml/Dom/XmlLoader.cs:line 174
   at System.Xml.XmlLoader.LoadDocSequence(XmlDocument parentDoc) in /_/src/libraries/System.Private.Xml/src/System/Xml/Dom/XmlLoader.cs:line 60
   at System.Xml.XmlLoader.Load(XmlDocument doc, XmlReader reader, Boolean preserveWhitespace) in /_/src/libraries/System.Private.Xml/src/System/Xml/Dom/XmlLoader.cs:line 51
   at System.Xml.XmlDocument.Load(XmlReader reader) in /_/src/libraries/System.Private.Xml/src/System/Xml/Dom/XmlDocument.cs:line 1323
   at System.Security.Cryptography.Xml.Utils.PreProcessDocumentInput(XmlDocument document, XmlResolver xmlResolver, String baseUri) in /_/src/libraries/System.Security.Cryptography.Xml/src/System/Security/Cryptography/Xml/Utils.cs:line 222
   at System.Security.Cryptography.Xml.Reference.CalculateHashValue(XmlDocument document, CanonicalXmlNodeList refList) in /_/src/libraries/System.Security.Cryptography.Xml/src/System/Security/Cryptography/Xml/Reference.cs:line 386
   at System.Security.Cryptography.Xml.SignedXml.CheckDigestedReferences() in /_/src/libraries/System.Security.Cryptography.Xml/src/System/Security/Cryptography/Xml/SignedXml.cs:line 917
   at System.Security.Cryptography.Xml.SignedXml.CheckSignature(KeyedHashAlgorithm macAlg) in /_/src/libraries/System.Security.Cryptography.Xml/src/System/Security/Cryptography/Xml/SignedXml.cs:line 312
   at System.Security.Cryptography.Xml.Tests.SignedXmlTest.VerifyXmlResolver(Boolean provideResolver) in /_/src/libraries/System.Security.Cryptography.Xml/tests/SignedXmlTest.cs:line 1658
   at InvokeStub_SignedXmlTest.VerifyXmlResolver(Object, Object, IntPtr*)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
----- Inner Stack Trace -----
   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 632
   at System.Net.Http.HttpClient.GetStreamAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 352
   at System.Xml.XmlDownloadManager.GetNonFileStreamAsync(Uri uri, ICredentials credentials, IWebProxy proxy) in /_/src/libraries/System.Private.Xml/src/System/Xml/XmlDownloadManagerAsync.cs:line 43
   at System.Xml.XmlDownloadManager.GetStream(Uri uri, ICredentials credentials, IWebProxy proxy) in /_/src/libraries/System.Private.Xml/src/System/Xml/XmlDownloadManager.cs:line 22
   at System.Xml.XmlUrlResolver.GetEntity(Uri absoluteUri, String role, Type ofObjectToReturn) in /_/src/libraries/System.Private.Xml/src/System/Xml/XmlUrlResolver.cs:line 47
   at System.Xml.XmlTextReaderImpl.OpenAndPush(Uri uri) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 8023
   at System.Xml.XmlTextReaderImpl.PushExternalEntityOrSubset(String publicId, String systemId, Uri baseUri, String entityName) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 7969
----- Inner Stack Trace -----
----- Inner Stack Trace -----

at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnection.cs:line 836

at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 1091

at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/RedirectHandler.cs:line 30

at System.Net.Http.HttpClient.GetStreamAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 343

----- Inner Stack Trace -----

at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.Tasks.cs:line 1451

at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token) in /_/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.Tasks.cs:line 1346

at System.Net.Http.HttpConnection.InitialFillAsync(Boolean async) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnection.cs:line 1676

at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnection.cs:line 599

----- Inner Stack Trace -----


Failing Configuration
net7.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open

Author:	danmoseley
Assignees:	-
Labels:	`area-System.Xml`
Milestone:	-

jeffhandley · 2022-08-22T22:29:31Z

Since this test was just added and it's failing often, I'd like to see the test fix get addressed in the release/7.0 branch to prevent permanent noise there.

eiriktsarpalis · 2022-08-23T11:36:10Z

Slightly unrelated to the test failure itself -- shouldn't this test be part of outerloop?

ghost · 2022-08-23T12:29:28Z

Tagging subscribers to this area: @dotnet/area-system-security, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

This was just introduced in #73676 and is failing at least 4 times a day. @GrabYourPitchforks

History: https://dev.azure.com/dnceng/public/_build/results?buildId=1949830&view=ms.vss-test-web.build-test-results-tab&runId=50186248&resultId=193531&paneView=history

example https://dev.azure.com/dnceng/public/_build/results?buildId=1949830&view=ms.vss-test-web.build-test-results-tab&runId=50186248&resultId=193531&paneView=debug

System.Security.Cryptography.Xml.Tests.SignedXmlTest.VerifyXmlResolver [[Console]](https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-74094-merge-bd71821e4e9c4ed7aa/System.Security.Cryptography.Xml.Tests/1/console.5d095986.log?helixlogtype=result) [[Details]](https://dev.azure.com/dnceng//public/_build/results?buildId=1949830&view=ms.vss-test-web.build-test-results-tab&runId=50186248&resultId=193531&paneView=debug) [[Artifacts]](https://dev.azure.com/dnceng//public/_build/results?buildId=1949830&view=ms.vss-test-web.build-test-results-tab&runId=50186248&resultId=193531&paneView=dotnet-dnceng.dnceng-build-release-tasks.helix-test-information-tab) [[3.18% failure rate]](https://dev.azure.com/dnceng//public/_build/results?buildId=1949830&view=ms.vss-test-web.build-test-results-tab&runId=50186248&resultId=193531&paneView=history)
Data Driven SubResults
❌ System.Security.Cryptography.Xml.Tests.SignedXmlTest.VerifyXmlResolver(provideResolver: True)
Exception Message
System.Xml.XmlException : An error has occurred while opening external entity 'http://127.0.0.1:9000/': The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
---- System.Threading.Tasks.TaskCanceledException : The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
-------- System.TimeoutException : The operation was canceled.
------------ System.Threading.Tasks.TaskCanceledException : The operation was canceled.
---------------- System.IO.IOException : Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
-------------------- System.Net.Sockets.SocketException : The I/O operation has been aborted because of either a thread exit or an application request.
Stack Trace
   at System.Xml.XmlTextReaderImpl.Throw(Exception e) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 2778
   at System.Xml.XmlTextReaderImpl.PushExternalEntityOrSubset(String publicId, String systemId, Uri baseUri, String entityName) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 7984
   at System.Xml.XmlTextReaderImpl.PushExternalEntity(IDtdEntityInfo entity) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 8052
   at System.Xml.XmlTextReaderImpl.HandleGeneralEntityReference(String name, Boolean isInAttributeValue, Boolean pushFakeEntityIfNullResolver, Int32 entityStartLinePos) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 6144
   at System.Xml.XmlTextReaderImpl.HandleEntityReference(Boolean isInAttributeValue, EntityExpandType expandType, Int32& charRefEndPos) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 6069
   at System.Xml.XmlTextReaderImpl.ParseText(Int32& startPos, Int32& endPos, Int32& outOrChars) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 5690
   at System.Xml.XmlTextReaderImpl.FinishPartialValue() in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 5834
   at System.Xml.XmlTextReaderImpl.get_Value() in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 919
   at System.Xml.XmlLoader.LoadNode(Boolean skipOverWhitespace) in /_/src/libraries/System.Private.Xml/src/System/Xml/Dom/XmlLoader.cs:line 174
   at System.Xml.XmlLoader.LoadDocSequence(XmlDocument parentDoc) in /_/src/libraries/System.Private.Xml/src/System/Xml/Dom/XmlLoader.cs:line 60
   at System.Xml.XmlLoader.Load(XmlDocument doc, XmlReader reader, Boolean preserveWhitespace) in /_/src/libraries/System.Private.Xml/src/System/Xml/Dom/XmlLoader.cs:line 51
   at System.Xml.XmlDocument.Load(XmlReader reader) in /_/src/libraries/System.Private.Xml/src/System/Xml/Dom/XmlDocument.cs:line 1323
   at System.Security.Cryptography.Xml.Utils.PreProcessDocumentInput(XmlDocument document, XmlResolver xmlResolver, String baseUri) in /_/src/libraries/System.Security.Cryptography.Xml/src/System/Security/Cryptography/Xml/Utils.cs:line 222
   at System.Security.Cryptography.Xml.Reference.CalculateHashValue(XmlDocument document, CanonicalXmlNodeList refList) in /_/src/libraries/System.Security.Cryptography.Xml/src/System/Security/Cryptography/Xml/Reference.cs:line 386
   at System.Security.Cryptography.Xml.SignedXml.CheckDigestedReferences() in /_/src/libraries/System.Security.Cryptography.Xml/src/System/Security/Cryptography/Xml/SignedXml.cs:line 917
   at System.Security.Cryptography.Xml.SignedXml.CheckSignature(KeyedHashAlgorithm macAlg) in /_/src/libraries/System.Security.Cryptography.Xml/src/System/Security/Cryptography/Xml/SignedXml.cs:line 312
   at System.Security.Cryptography.Xml.Tests.SignedXmlTest.VerifyXmlResolver(Boolean provideResolver) in /_/src/libraries/System.Security.Cryptography.Xml/tests/SignedXmlTest.cs:line 1658
   at InvokeStub_SignedXmlTest.VerifyXmlResolver(Object, Object, IntPtr*)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
----- Inner Stack Trace -----
   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 632
   at System.Net.Http.HttpClient.GetStreamAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 352
   at System.Xml.XmlDownloadManager.GetNonFileStreamAsync(Uri uri, ICredentials credentials, IWebProxy proxy) in /_/src/libraries/System.Private.Xml/src/System/Xml/XmlDownloadManagerAsync.cs:line 43
   at System.Xml.XmlDownloadManager.GetStream(Uri uri, ICredentials credentials, IWebProxy proxy) in /_/src/libraries/System.Private.Xml/src/System/Xml/XmlDownloadManager.cs:line 22
   at System.Xml.XmlUrlResolver.GetEntity(Uri absoluteUri, String role, Type ofObjectToReturn) in /_/src/libraries/System.Private.Xml/src/System/Xml/XmlUrlResolver.cs:line 47
   at System.Xml.XmlTextReaderImpl.OpenAndPush(Uri uri) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 8023
   at System.Xml.XmlTextReaderImpl.PushExternalEntityOrSubset(String publicId, String systemId, Uri baseUri, String entityName) in /_/src/libraries/System.Private.Xml/src/System/Xml/Core/XmlTextReaderImpl.cs:line 7969
----- Inner Stack Trace -----
----- Inner Stack Trace -----

at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnection.cs:line 836

at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 1091

at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/RedirectHandler.cs:line 30

at System.Net.Http.HttpClient.GetStreamAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 343

----- Inner Stack Trace -----

at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.Tasks.cs:line 1451

at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token) in /_/src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.Tasks.cs:line 1346

at System.Net.Http.HttpConnection.InitialFillAsync(Boolean async) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnection.cs:line 1676

at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnection.cs:line 599

----- Inner Stack Trace -----


Failing Configuration
net7.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open

HttpListener was successfully started on 127.0.0.1:9000, but apparently did not respond promptly. That was on Windows. Looking at history, we also see failures on Android and Ubuntu, which may have a slightly different error:

                   Starting:    System.Security.Cryptography.Xml.Tests (parallel test collections = on, max threads = 1)
                       System.Xml.XmlException : An error has occurred while opening external entity 'http://127.0.0.1:9000/': The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
                       ---- System.Threading.Tasks.TaskCanceledException : The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
                       -------- System.TimeoutException : The operation was canceled.
                       ------------ System.Threading.Tasks.TaskCanceledException : The operation was canceled.
                       ---------------- System.IO.IOException : The response ended prematurely.

It's probably significant that this is not OS specific. @dotnet/ncl do you see anything obvious about the test here? Code:

runtime/src/libraries/System.Security.Cryptography.Xml/tests/SignedXmlTest.cs

Line 1603 in f140a12

listener = new HttpListener();

Author:	danmoseley
Assignees:	-
Labels:	`area-System.Security`, `area-System.Xml`, `test-failure`
Milestone:	7.0.0

bartonjs · 2022-08-23T16:44:31Z

shouldn't this test be part of outerloop?

That's probably reasonable. It seems odd that there's enough load going on that these tests can't reliably use HttpListner / loopback networking... but c'est la vie.

wfurt · 2022-08-23T16:47:26Z

HttpListner is finicky and we don't use it for HTTP tests. As long as this runs on loopback e.g. within test host I don't see reason to make it outer loop. In general, We do that for anything that needs to reach out to Internet.

My suggestion would be to re-write the test.

GrabYourPitchforks · 2022-08-23T17:16:03Z

@bartonjs Maybe we can use a static file on disk rather than an http listener? I remember when we discussed this a few months ago during the original CVE fix the issue of test flakiness was brought up, but at the time it was deemed that this would be stable enough for innerloop use. Do we know what was wrong with our analysis?

bartonjs · 2022-08-23T17:20:30Z

Do we know what was wrong with our analysis?

Nope. The X509 revocation tests and AIA tests all use HttpListener, and while there is an occasional problem they don't fail all that often. Of course, they're all OuterLoop, not because of the HttpListener, but because they take several minutes to run. So maybe it's just that innerloop runs so much more often that it's showing up here.

danmoseley · 2022-08-23T19:01:07Z

If we disable this have we lost protection against regressing the security feature?

GrabYourPitchforks · 2022-08-23T19:35:03Z

If we disable this have we lost protection against regressing the security feature?

We're discussing in email some other ways of making a less flaky test. Those other mechanisms would give us varying degrees of confidence that we didn't regress this.

jeffhandley · 2022-09-08T22:25:52Z

@bartonjs and @GrabYourPitchforks reported progress on these discussions and work is underway to refactor the tests. While this isn't blocking the release, we will want to get the text changes into the release/7.0 branch so that we have regression protection there for servicing.

PR #75369 tried to fix the problem and re-enabled the test, but it is still failing. Tracked by #74115

karelz · 2022-09-24T17:12:10Z

@bartonjs looks like your PR didn't fix the test yet. Re-disabling it again in PR #76129

Failures in last 30 days

First in Rolling run 24630 on 9/21 AM
9/24 - 14x failures, i.e. 3x per day -- all on net7.0-Linux-Release-x64-Mono_Release_LinuxBionic-Ubuntu.1804.Amd64.Android.29.Open

PR #75369 tried to fix the problem and re-enabled the test, but it is still failing. Tracked by #74115

karelz · 2022-09-24T20:59:33Z

Test disabled in PR #76129 -- it can be probably narrowed down to the specific platform, or fixed for the platform.

krwq · 2024-02-09T12:03:55Z

I think the problem here is that the listener doesn't start by the time we send request. I think the order should be following:

testcase:

start a task
wait until notification that task is ready to process <-- this is missing currently
parse XML, wait until task 1 is started

task:

start server
notify it's started <-- this is missing
process requests

possibly can be simplified by maybe doing everything until listener.Start(); synchronously.

dotnet-issue-labeler bot added the area-System.Xml label Aug 18, 2022

ghost added the untriaged New issue has not been triaged by the area owner label Aug 18, 2022

danmoseley mentioned this issue Aug 18, 2022

[release/7.0-rc1] Don't fail the test when the file is in use #74094

Merged

jeffhandley added the test-failure label Aug 22, 2022

jeffhandley added this to the 7.0.0 milestone Aug 22, 2022

ghost removed the untriaged New issue has not been triaged by the area owner label Aug 22, 2022

eiriktsarpalis mentioned this issue Aug 23, 2022

Disable VerifyXmlResolver test. #74417

Merged

krwq added area-System.Security and removed area-System.Xml labels Aug 23, 2022

This was referenced Aug 23, 2022

[release/7.0-rc1] Disable VerifyXmlResolver test. #74446

Closed

[release/7.0] Disable VerifyXmlResolver test. #74447

Merged

bartonjs mentioned this issue Sep 9, 2022

Change VerifyXmlResolver to use TcpListener over HttpListener #75369

Merged

jeffhandley linked a pull request Sep 16, 2022 that will close this issue

Change VerifyXmlResolver to use TcpListener over HttpListener #75369

Merged

jeffhandley added the in-pr There is an active PR which will close this issue when it is merged label Sep 16, 2022

bartonjs closed this as completed in #75369 Sep 20, 2022

karelz added a commit that referenced this issue Sep 24, 2022

Disable failing test SignedXmlTest.VerifyXmlResolver

e03fd16

PR #75369 tried to fix the problem and re-enabled the test, but it is still failing. Tracked by #74115

karelz mentioned this issue Sep 24, 2022

Disable failing test SignedXmlTest.VerifyXmlResolver #76129

Merged

karelz reopened this Sep 24, 2022

karelz added disabled-test The test is disabled in source code against the issue and removed in-pr There is an active PR which will close this issue when it is merged labels Sep 24, 2022

karelz removed this from the 7.0.0 milestone Sep 24, 2022

ghost added the untriaged New issue has not been triaged by the area owner label Sep 24, 2022

karelz added this to the 7.0.0 milestone Sep 24, 2022

karelz added blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' and removed untriaged New issue has not been triaged by the area owner disabled-test The test is disabled in source code against the issue labels Sep 24, 2022

karelz added a commit that referenced this issue Sep 24, 2022

Disable failing test SignedXmlTest.VerifyXmlResolver (#76129)

2600909

PR #75369 tried to fix the problem and re-enabled the test, but it is still failing. Tracked by #74115

karelz added disabled-test The test is disabled in source code against the issue and removed blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' labels Sep 24, 2022

bartonjs modified the milestones: 7.0.0, Future Sep 26, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SignedXmlTest.VerifyXmlResolver sometimes failing due to failing to connect to localhost. #74115

SignedXmlTest.VerifyXmlResolver sometimes failing due to failing to connect to localhost. #74115

danmoseley commented Aug 18, 2022 •

edited by build-analysis bot

Loading

ghost commented Aug 18, 2022

jeffhandley commented Aug 22, 2022

eiriktsarpalis commented Aug 23, 2022

ghost commented Aug 23, 2022

bartonjs commented Aug 23, 2022

wfurt commented Aug 23, 2022 •

edited

Loading

GrabYourPitchforks commented Aug 23, 2022

bartonjs commented Aug 23, 2022

danmoseley commented Aug 23, 2022

GrabYourPitchforks commented Aug 23, 2022

jeffhandley commented Sep 8, 2022

karelz commented Sep 24, 2022 •

edited

Loading

karelz commented Sep 24, 2022

krwq commented Feb 9, 2024

SignedXmlTest.VerifyXmlResolver sometimes failing due to failing to connect to localhost. #74115

SignedXmlTest.VerifyXmlResolver sometimes failing due to failing to connect to localhost. #74115

Comments

danmoseley commented Aug 18, 2022 • edited by build-analysis bot Loading

Report

Summary

ghost commented Aug 18, 2022

jeffhandley commented Aug 22, 2022

eiriktsarpalis commented Aug 23, 2022

ghost commented Aug 23, 2022

bartonjs commented Aug 23, 2022

wfurt commented Aug 23, 2022 • edited Loading

GrabYourPitchforks commented Aug 23, 2022

bartonjs commented Aug 23, 2022

danmoseley commented Aug 23, 2022

GrabYourPitchforks commented Aug 23, 2022

jeffhandley commented Sep 8, 2022

karelz commented Sep 24, 2022 • edited Loading

karelz commented Sep 24, 2022

krwq commented Feb 9, 2024

danmoseley commented Aug 18, 2022 •

edited by build-analysis bot

Loading

wfurt commented Aug 23, 2022 •

edited

Loading

karelz commented Sep 24, 2022 •

edited

Loading