Move the IOCP thread pool on windows to managed code

[davidfowl]

Today the managed thread pool implementation moved worker threads and the gate thread but doesn't include IO threads to handle IOCP completions. Lots of complex logic (like handling multiple app domains) and overhead could be removed (and it would be easier to experiment with) by moving the logic into managed code:

• win32threadpool.cpp should use GetQueuedCompletionStatusEx rather than GetQueuedCompletionStatus #11314
• Optimize threadpool paths in CoreCLR used by ASP.NET #8299

Other benefits include having this pool be usable on the NativeAOT branch (which currently uses the windows thread pool for IOCP).

The other follow on and possibly unrelated item that might be interesting here is looking at unifying/centralizing how we poll for IO similar to how other platforms do (assuming it doesn't regress performance).

cc @kouvel @geoffkizer @stephentoub @jkotas

[davidfowl]

@kouvel I wonder if it makes sense to use worker threads to dequeue work from the IO Completion Port instead of having different IO threads and worker threads (similar to how we have these time sensitive work items that run after the normal work items).

[kouvel]

Yea I think using thread pool threads for polling would be better, maybe not with the time-sensitive queue though, as its latency could be too high. A question may be about cases where having multiple epoll threads is currently better (larger machines), I think the better batching and timing of polls may compensate for it some, depends on the delay/work involved in polling and queuing the completions, and differences in timing with larger batches. On Windows IO completions also run out-of-order and in parallel with normal work items, there would be behavioral differences there that would need to be accounted for / experimented with.

[geoffkizer]

I agree that it would be great to get rid of separate IO threads. This causes some weird issues, e.g. there are no work sharing queues on IO threads (at least last time I checked -- maybe this has changed)?

@kouvel I wonder if it makes sense to use worker threads to dequeue work from the IO Completion Port instead of having different IO threads and worker threads (similar to how we have these time sensitive work items that run after the normal work items).

Trying this out seems pretty straightforward to do, and we can run lots of perf tests like TechEmpower to analyze the impact.

[GSPP]

IO threads are a concept that is not widely understood in the .NET community. It would be a useful simplification if this went away.

[davidfowl]

@kouvel this should be closed right?

[kouvel]

Yea I think this can be closed, IO completion handling was moved to the portable thread pool implementation and IO completions run on worker threads after #64834. Polling for IO on worker threads needs some experimentation.