refactor old APM [Begin/End]Accept methods on top of Task APIs, and enable for Unix as well #51212

geoffkizer · 2021-04-14T05:11:07Z

Note, the old Begin/EndAccept methods support doing an accept and receive in a single operation. Unfortunately the API for this is terrible and forces allocation. It's also not currently supported on non-Windows platforms. So, replace this with a helper routine that performs an accept followed by a receive, and works across all platforms.

Contributes to #43845

ghost · 2021-04-14T05:11:15Z

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

Note, the old Begin/EndAccept methods support doing an accept and receive in a single operation. Unfortunately the API for this is terrible and forces allocation. It's also not currently supported on non-Windows platforms. So, replace this with a helper routine that performs an accept followed by a receive, and works across all platforms.

Contributes to #43845

Author:	geoffkizer
Assignees:	-
Labels:	`area-System.Net.Sockets`
Milestone:	-

geoffkizer · 2021-04-14T05:51:28Z

/azp run runtime-libraries-coreclr outerloop

azure-pipelines · 2021-04-14T05:51:41Z

Azure Pipelines successfully started running 1 pipeline(s).

src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs

src/libraries/System.Net.Sockets/tests/FunctionalTests/SocketTestHelper.cs

stephentoub · 2021-04-14T13:12:05Z

the old Begin/EndAccept methods support doing an accept and receive in a single operation. Unfortunately the API for this is terrible and forces allocation

In theory there's a perf benefit possible from the existing accept+receive on Windows, but as we only expose it from an old APM-based API we discourage using, it incurs non-trivial allocation overheads, and it's complicated, non-portable logic, I'm fine seeing it go away.

src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs

geoffkizer · 2021-04-18T01:07:42Z

I also pushed a change to remove the CallbackClosure cache. This is now only used for BeginSendFile, and that will go away eventually as well.

Saves 8 bytes on the Socket object.

geoffkizer · 2021-04-18T01:07:58Z

/azp run runtime-libraries-coreclr outerloop

azure-pipelines · 2021-04-18T01:08:11Z

Azure Pipelines successfully started running 1 pipeline(s).

…nable for Unix as well

geoffkizer · 2021-04-18T02:46:16Z

/azp run runtime-libraries-coreclr outerloop

azure-pipelines · 2021-04-18T02:46:35Z

Azure Pipelines successfully started running 1 pipeline(s).

…orted on Windows

geoffkizer · 2021-04-18T06:14:19Z

/azp run runtime-libraries-coreclr outerloop

azure-pipelines · 2021-04-18T06:14:31Z

Azure Pipelines successfully started running 1 pipeline(s).

stephentoub · 2021-04-18T20:58:44Z

src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs

-            return null; // unreachable
-        }
+        public IAsyncResult BeginAccept(AsyncCallback? callback, object? state) =>
+            TaskToApmBeginWithSyncExceptions(AcceptAsync(), callback, state);


When was this added? I don't remember seeing it. Is this actually providing compatibility with what was there before? It's not just throwing exceptions that may have occurred on the initiation path, it's throwing all exceptions from the whole operation if it happens to complete quickly, by the time we get to the IsFaulted check. I don't have a strong opinion, but if our goal is compatibility, i think this likely makes it worse, potentially throwing new exceptions that previously weren't possible from Begin. I suggest we just accept the small difference in behavior we've accepted everywhere else we've used TaskToApm, rather than trying to special-case this. If you want to keep it, though, ok.

This got added in #51213 (which I just merged last night) to address the compat concerns from @antonfirsov. I didn't think it would be controversial, so I merged it without additional review.

It provides compat with what was there before in the sense that sync exceptions now propagate synchronously. It also, as you point out, will throw synchronously if the task completes asynchronously but before we hit the IsFaulted check. AFAIK there is no way to avoid this (right?).

I suggest we just accept the small difference in behavior we've accepted everywhere else we've used TaskToApm, rather than trying to special-case this.

I'm fine with this. I'll just rip this code out. @antonfirsov any concerns here? Based on this I think we should close #47905 without action as well.

@stephentoub le me know, if my definition of breaking changes, and/or my expectations around maintaining docs are too strict, or if I'm misunderstanding the rules in other means, but this is how I view it:

AFAIK we do not document async exceptions the same way we do with we sync exceptions since they are not being actually thrown by calling the method, but rather by awaiting it.

This means that if we close #47905 as wontfix, we need to add a "breaking change" label to all these PR-s, and make sure we propagate and address all the related documentation work in 6.0. If the goal is to save time, I'm don't think this path will result in less work. I would just add a tiny layer of compat code instead.

Whether the behavior was documented or not, it can still be a breaking change. And my point is that in this case, it's a breaking change whether or not this TaskToApmBeginWithSyncExceptions is used, and from my perspective, it's a larger break with TaskToApmBeginWithSyncExceptions (it's non-deterministically causing more exceptions to be thrown out of the BeginXx method). I'm not convinced either way it rises to the level that requires being documented as a breaking change, but if you feel it needs to be, feel free. To my knowledge we haven't in the past when we've ripped out APM implementations and replaced them with TaskToApm wrappers around XxAsync methods.

I'm not convinced either way it rises to the level that requires being documented as a breaking change

Do we have any formal guidelines about the bar for breaking changes?

My main concern is around the API docs. What is our process then for making sure that the exceptions which are no longer thrown synchronously will be removed from the list? If there is none, we will create inconsistency in our docs which doesn't look good, even if these particular API-s are marginal.

We can still accomplish it with a trick like the one below

Can you share a full example? I think it's much more complicated than that, and would deoptimize the XxAsync methods we're trying to reuse.

Opened #51693 to show a more complete example.

Thanks. I left a few comments. That only partially addresses the issue, and adds non-trivial complication. I don't believe it's worth it. If you believe this rises to the level of requiring a breaking change notification, please feel free to do so. But FYI we've made such changes throughout .NET Core incrementally over the last several years, ripping out lots of custom IAsyncResult code across lots of libraries, and I don't believe we've documented any of them as being breaking.

Thanks! I see now the more complicated cases, and agree with your analysis, that it's not worth it.

ripping out lots of custom IAsyncResult code across lots of libraries, and I don't believe we've documented any of them as being breaking.

When it comes to the process that is triggered by applying the "breaking change" labels today, I don't think it makes sense to deviate from an established practice in this particular case of sockets. I opened #6658 to track the API docs work.

I opened #6658 to track the API docs work.

This looks like it's linked to the wrong repo, perhaps?

geoffkizer · 2021-04-18T23:51:00Z

/azp run runtime-libraries-coreclr outerloop

azure-pipelines · 2021-04-18T23:51:14Z

Azure Pipelines successfully started running 1 pipeline(s).

antonfirsov · 2021-04-19T11:31:30Z

src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs

@@ -2124,7 +2114,7 @@ public void EndConnect(IAsyncResult asyncResult)
        }

        public IAsyncResult BeginDisconnect(bool reuseSocket, AsyncCallback? callback, object? state) =>
-            TaskToApmBeginWithSyncExceptions(DisconnectAsync(reuseSocket).AsTask(), callback, state);
+            TaskToApm.Begin(DisconnectAsync(reuseSocket).AsTask(), callback, state);


I don't understand how does this work without TaskToApmBeginWithSyncExceptions now.

Ah ok, missed the discussion above.

antonfirsov · 2021-04-19T11:32:06Z

src/libraries/System.Net.Sockets/tests/FunctionalTests/Accept.cs

@@ -236,7 +236,7 @@ public void Accept_WithAlreadyBoundTargetSocket_Fails()

                server.BindToAnonymousPort(IPAddress.Loopback);

-                Assert.Throws<InvalidOperationException>(() => { AcceptAsync(listener, server); });
+                await Assert.ThrowsAsync<InvalidOperationException>(() => AcceptAsync(listener, server));


In case of APM, shouldn't this throw without awaiting?

#51212 (comment)

dotnet-issue-labeler bot added the area-System.Net.Sockets label Apr 14, 2021

stephentoub reviewed Apr 14, 2021

View reviewed changes

src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs Outdated Show resolved Hide resolved

stephentoub reviewed Apr 14, 2021

View reviewed changes

src/libraries/System.Net.Sockets/tests/FunctionalTests/SocketTestHelper.cs Show resolved Hide resolved

antonfirsov reviewed Apr 14, 2021

View reviewed changes

src/libraries/System.Net.Sockets/src/System/Net/Sockets/Socket.cs Show resolved Hide resolved

antonfirsov mentioned this pull request Apr 14, 2021

add Task-based DisconnectAsync and refactor APM methods on top of it #51213

Merged

geoffkizer force-pushed the acceptapm branch from 61cd98e to b31f555 Compare April 18, 2021 01:01

Geoffrey Kizer added 5 commits April 17, 2021 19:43

refactor old APM [Begin/End]Accept methods on top of Task APIs, and e…

dfdb557

…nable for Unix as well

fix BeginAccept to throw synchronously, and relevant test changes

72a3651

fix AcceptReceive test to run on unix too

e53e896

dispose accepted socket if receive throws

3a1b865

remove CallbackClosure cache

27305b2

geoffkizer force-pushed the acceptapm branch from 050d65e to 27305b2 Compare April 18, 2021 02:44

fix test helper to indicate AcceptReceive with EAP is still only supp…

9378d1d

…orted on Windows

stephentoub reviewed Apr 18, 2021

View reviewed changes

rip out TaskToApmBeginWithSyncExceptions and adjust tests

c9b31af

antonfirsov reviewed Apr 19, 2021

View reviewed changes

runfoapp bot mentioned this pull request Apr 19, 2021

Symlinks left on disk by Installer runs breaks job log upload on Linux #51502

Closed

This was referenced Apr 22, 2021

[Draft] Idea for detection of synchronous completions for APM over Task in sockets #51693

Closed

Socket.Begin*** methods wrapping Task variants do not throw SocketExceptions synchronously #47905

Closed

antonfirsov approved these changes Apr 22, 2021

View reviewed changes

geoffkizer merged commit 5902515 into dotnet:main Apr 26, 2021

geoffkizer mentioned this pull request Apr 26, 2021

Change IAsyncResult APIs in System.Net.Sockets to wrap Task APIs #43845

Closed

9 tasks

karelz added this to the 6.0.0 milestone May 20, 2021

ghost locked as resolved and limited conversation to collaborators Jun 19, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor old APM [Begin/End]Accept methods on top of Task APIs, and enable for Unix as well #51212

refactor old APM [Begin/End]Accept methods on top of Task APIs, and enable for Unix as well #51212

geoffkizer commented Apr 14, 2021

ghost commented Apr 14, 2021

geoffkizer commented Apr 14, 2021

azure-pipelines bot commented Apr 14, 2021

stephentoub commented Apr 14, 2021

geoffkizer commented Apr 18, 2021

geoffkizer commented Apr 18, 2021

azure-pipelines bot commented Apr 18, 2021

geoffkizer commented Apr 18, 2021

azure-pipelines bot commented Apr 18, 2021

geoffkizer commented Apr 18, 2021

azure-pipelines bot commented Apr 18, 2021

stephentoub Apr 18, 2021 •

edited

Loading

geoffkizer Apr 18, 2021

antonfirsov Apr 19, 2021

stephentoub Apr 19, 2021

antonfirsov Apr 19, 2021

stephentoub Apr 21, 2021

antonfirsov Apr 22, 2021

stephentoub Apr 22, 2021

antonfirsov Apr 22, 2021

geoffkizer Apr 26, 2021

geoffkizer commented Apr 18, 2021

azure-pipelines bot commented Apr 18, 2021

antonfirsov Apr 19, 2021

antonfirsov Apr 19, 2021

antonfirsov Apr 19, 2021

antonfirsov Apr 19, 2021

refactor old APM [Begin/End]Accept methods on top of Task APIs, and enable for Unix as well #51212

refactor old APM [Begin/End]Accept methods on top of Task APIs, and enable for Unix as well #51212

Conversation

geoffkizer commented Apr 14, 2021

ghost commented Apr 14, 2021

geoffkizer commented Apr 14, 2021

azure-pipelines bot commented Apr 14, 2021

stephentoub commented Apr 14, 2021

geoffkizer commented Apr 18, 2021

geoffkizer commented Apr 18, 2021

azure-pipelines bot commented Apr 18, 2021

geoffkizer commented Apr 18, 2021

azure-pipelines bot commented Apr 18, 2021

geoffkizer commented Apr 18, 2021

azure-pipelines bot commented Apr 18, 2021

stephentoub Apr 18, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

geoffkizer commented Apr 18, 2021

azure-pipelines bot commented Apr 18, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stephentoub Apr 18, 2021 •

edited

Loading