Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Harden and re-enable DualMode socket tests #80715

Merged
merged 2 commits into from
Feb 7, 2023

Conversation

antonfirsov
Copy link
Member

@antonfirsov antonfirsov commented Jan 16, 2023

Fixes #1481.
Fixes #79820.

Throwing DisableParallelization at all tests in DualModeSocketTest increased the total execution time of those tests by approximately 1 minute on my Windows machine, so I decided to go with a more sophisticated approach.

  • UDP tests are moved to a non-parallel collection, since there is no way to prevent parallel UDP tests from intercepting each other's packets. This only increases the execution time by ~2 seconds.
  • TCP interferences are prevented by a utility PortBlocker, which creates and binds a "shadow" socket of the opposite address family to prevent sockets in parallel tests to bind to the same port.
  • Accept and SendTo tests are refactored to use SocketTestHelperBase<T>, reducing duplicate code.

@ghost
Copy link

ghost commented Jan 16, 2023

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes #1481.
Fixes #21853.

Dropping DisableParallelization at all tests in DualModeSocketTest increased the total execution time of those tests by approximately 1 minute on my Windows machine, so I decided to go with a more sophisticated approach.

  • UDP tests are moved to a non-parallel collection, since there is no way to prevent parallel UDP tests from intercepting each other's packets. This only increases the execution time by ~2 seconds.
  • TCP interferences are prevented by a utility PortBlocker, which creates and binds a "shadow" socket of the opposite address family to prevent sockets in parallel tests to bind to the same port.
  • Accept and SendTo tests are refactored to use SocketTestHelperBase<T>, reducing duplicate code.
Author: antonfirsov
Assignees: -
Labels:

area-System.Net.Sockets

Milestone: -

@liveans
Copy link
Member

liveans commented Jan 17, 2023

/azp run runtime-libraries-coreclr outerloop

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Copy link
Member

@liveans liveans left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@antonfirsov
Copy link
Member Author

/azp run runtime-libraries-coreclr outerloop

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@antonfirsov
Copy link
Member Author

/azp run runtime

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@antonfirsov
Copy link
Member Author

@wfurt do you want to take a look, or should I go ahead and merge this?

Copy link
Member

@wfurt wfurt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I would probably go with disabled parallelization but I'm fine with the helper if you feel it would stabilize the test. Time will tell and keeping the test disable does not provide any benefits.

@karelz karelz added this to the 8.0.0 milestone Feb 7, 2023
@antonfirsov antonfirsov merged commit 5b31064 into dotnet:main Feb 7, 2023
@ManickaP
Copy link
Member

ManickaP commented Feb 9, 2023

I think that some of the re-enabled / shuffled tests are failing on extra platform pipeline:
https://dev.azure.com/dnceng-public/public/_build/results?buildId=165856&view=ms.vss-test-web.build-test-results-tab&runId=3369863&resultId=168462&paneView=debug
https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-81481-merge-79b8ca868c0d49858f/System.Net.Sockets.Tests/3/testResults.xml?helixlogtype=result

<test name="System.Net.Sockets.Tests.DualModeConnectionlessReceiveMessageFrom.ReceiveMessageFromAsyncV4BoundToSpecificV6_NotReceived" type="System.Net.Sockets.Tests.DualModeConnectionlessReceiveMessageFrom" method="ReceiveMessageFromAsyncV4BoundToSpecificV6_NotReceived" time="0.0168523" result="Fail">
<traits>
<trait name="IPv4" value="true"/>
<trait name="IPv6" value="true"/>
</traits>
<failure exception-type="Xunit.Sdk.ThrowsException">
<message>
Assert.Throws() Failure\nExpected: typeof(System.TimeoutException)\nActual: typeof(System.ArgumentException): The supplied EndPoint of AddressFamily InterNetwork is not valid for this Socket, use InterNetworkV6 instead. (Parameter 'e')\n---- System.ArgumentException : The supplied EndPoint of AddressFamily InterNetwork is not valid for this Socket, use InterNetworkV6 instead. (Parameter 'e')
</message>
<stack-trace>
at System.Net.Sockets.Socket.ReceiveMessageFromAsync(SocketAsyncEventArgs e, CancellationToken cancellationToken) at System.Net.Sockets.Socket.ReceiveMessageFromAsync(SocketAsyncEventArgs e) at System.Net.Sockets.Tests.DualModeConnectionlessReceiveMessageFrom.ReceiveMessageFromAsync_Helper(IPAddress listenOn, IPAddress connectTo, Boolean expectedToTimeout) in /_/src/libraries/System.Net.Sockets/tests/FunctionalTests/DualModeSocketTest.cs:line 1666 at System.Net.Sockets.Tests.DualModeConnectionlessReceiveMessageFrom.<ReceiveMessageFromAsyncV4BoundToSpecificV6_NotReceived>b__39_0() in /_/src/libraries/System.Net.Sockets/tests/FunctionalTests/DualModeSocketTest.cs:line 1613 ----- Inner Stack Trace ----- at System.Net.Sockets.Socket.ReceiveMessageFromAsync(SocketAsyncEventArgs e, CancellationToken cancellationToken) at System.Net.Sockets.Socket.ReceiveMessageFromAsync(SocketAsyncEventArgs e) at System.Net.Sockets.Tests.DualModeConnectionlessReceiveMessageFrom.ReceiveMessageFromAsync_Helper(IPAddress listenOn, IPAddress connectTo, Boolean expectedToTimeout) in /_/src/libraries/System.Net.Sockets/tests/FunctionalTests/DualModeSocketTest.cs:line 1666 at System.Net.Sockets.Tests.DualModeConnectionlessReceiveMessageFrom.<ReceiveMessageFromAsyncV4BoundToSpecificV6_NotReceived>b__39_0() in /_/src/libraries/System.Net.Sockets/tests/FunctionalTests/DualModeSocketTest.cs:line 1613
</stack-trace>
</failure>
</test>

cc: @antonfirsov

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
5 participants