Need a solution for select / async events

[brson]

Pipes supports 'select', which allows one to receive on multiple ports at the same time. The new scheduler doesn't have a solution for this, but it needs to be more general than the old one.

Some types of events we might want to wait on:

• Port receives, both one-shot, stream and other receives
• I/O reads of all kinds
• Timers
• Socket accepts

Miscellaneous notes:

The biggest requirement is that we need one abstraction that allows for waiting on both I/O events (single-threaded) and comm events (multithreaded). uv handles and comm types have very different implementations.

select may not be the right abstraction. Maybe there are more expressive ways to do this. We could for instance have an event loop with callbacks. Also look an .NET's async I/O which I know nothing about. Needs more research.

Asynchronous I/O events should hopefully allow for all the various extension methods and decorator types that synchronous I/O allows.

The solution should additionally be workable for tasks that are not coroutines scheduled by an I/O event loop. We would like to have tasks that are backed simply by threads and don't use the I/O event loop and that makes the design of various runtime components more difficult. It's potentially ok not to solve this yet.

Some types of events need to be cancellable as well, e.g. if you wait on both a timer and an accept, if the timer goes off first you may want to cancel the accept.

[brson]

This will be greatly complicated by the fact that I/O types are not bound to their schedulers - so it won't be as simple as registering a uv callback.

[mneumann]

Wouldn't it be better if tasks would always stay in the Scheduler they are created in? Or if you migrate them, also somehow migrate the task's libuv context?

[thadguidry]

This idea comes out of the Data Architect world that I live in. It's more along the lines of that basic idea that your pitching which I call "receivership".

In Pentaho Data Integration, there is a process step called "Block this step until steps finish". What it does is Block a single step in an execution flow until a selected step or even multiple steps somewhere in the chain or process has completed it's execution and returned a True for it's "execution complete flag" where a launched listener is waiting and watching.

It's a generic wrapper if you will, for additional next steps to perform, but those next steps do not get called, do not get any input, and no variables are passed to them until the the child Blocking step has seen from (there's a listener process launched for each step or steps choosen to watch) all its parent or parents steps somewhere say that it's own execution has finished and lets the listener know. I think the "Block this step until steps finish" just waits for all those listeners it is watching to say it's OK to proceed with it's own set of next steps.

Anyways, just wanted to throw this out there... and maybe this idea should be focused on another issue, feel free to move it there... here's their Javadoc http://javadoc.pentaho.com/kettle/org/pentaho/di/trans/steps/blockuntilstepsfinish/BlockUntilStepsFinish.html

[brson]

@mneumann I believe that it is not better for tasks to stay in the scheduler that created them. Here scheduler == thread, and it's easy under such a strategy not to distribute load effectively (fwiw our current scheduler never migrates tasks and it's easy to underutilize cores. This is easy to see in long running test cases that block other test cases while leaving other cores unoccupied).

The libuv context is essentially the libuv event loop, which is deeply integrated with the scheduler - it's what drives the scheduler and allows synchronous I/O to yield to other tasks - so the event loop itself cannot change threads. It may be possible in some circumstances to transplant file handles from one libuv event loop to another, but I don't think libuv currently has such functionality, and it isn't possible in all cases.

We could make a limitation that I/O handles are not sendable, but that is fairly restrictive.

[brson]

@thadguidry I responded on the list.

Putting the list link here for posterity: https://mail.mozilla.org/pipermail/rust-dev/2013-May/004305.html

[mikedilger]

I haven't used libuv, but I understand it is for platform independence. My comments will be only about linux, as that is the only knowledge base I have worth contributing from.

In newer kernels, the different types of events have all been unified to use file descriptors and thus be "select"-able with epoll_wait(). Timers can be detected with timerfd, signals with signalfd, and network and file activity with the file descriptors you are already familiar with. Thus all these events can be handled asynchronously non-blocking in edge-triggered mode. epoll() is thread-aware in a way that multiple threads can wait on the same epoll, and the event will go to whichever one the kernel is most enamoured with ;-)

I have some very basic code (560 lines) that touches on all this if you want to see an example.

[mikedilger]

https://github.com/mikedilger/epoll_demo

[mneumann]

I think the "problem" is that libuv/libevent can only select on kernel events while we also want to wait for port receives which are handled in pure user level. To solve it, you have to use a regular pipe (known to the kernel) which can awake the event loop from waiting. This is how libuv implements async callbacks. Async in this context means that you invoke a callback in the same thread as the event loop, but triggered from another thread.

I don't know if you will use a separate I/O task, or just one thread that performs all I/O and also runs the tasks. In the latter case you'd only have to do an async cross-thread call when:

1. The port receive is from a task on another scheduler/native thread and
2. this thread is currently blocked in the event loop waiting for I/O to happen.

You can special case 1. and directly schedule to the receiving task. And 2. you can handle with an async callback. Actually you can use an async callback whenever the receiving tasks lives in another scheduler. But I think you have to take care when you run the async callback (and for example schedule another thread from there) and then another async callback is triggered, you might loose it or not.

[mneumann]

@brson: I think thread migration is a good general solution as long as you are (optionally) able to pin a task to it's originating scheduler to avoid trashing. And maybe it's good when threads are randomly distributed at creation time to limit the work stealing.

[mneumann]

Timers, socket accepts and I/O of all kind would be handled by libuv. The callbacks would just set a bit in the corresponding Rust I/O object, and determine if the corresponding task would become runnable as a result. After all callbacks have been run by libev, the scheduler would then "simply" run all runnable threads. Port receives would be triggered (as I described above) via async callbacks. An async callback is "global" (only one per event loop), i.e. you need to keep information about the ports which contain messages in another data structure, which then the async callback can inspect and perform the same action as for other callbacks (determine if corresponding task would become runnable).

[brson]

After further consideration with @bblum on IRC I think it may not make sense to conflate waiting on pipes with waiting on I/O - especially if it's going to cause complexity or extra overhead for either. (Rust) pipes and sockets are pretty easy to adapt to each other. In particular, for the important use case of waiting on I/O while also waiting on some other signal, I think we can use an I/O-based message abstraction, i.e. something that behaves like pipes but is backed by unix domain sockets.

[pnkfelix]

@brson Would signal handling support be considered part of the more general problem of asynchronous events described here? (I am wondering whether bugs that could be resolved via a program-specified signal handler should be considered blocked by this issue.)

[graydon]

Signals are i/o in uv. So yes.

[pnkfelix]

a solution for this seems like it ties into our story for event handling, such as in the concrete example in #2873.

Nominating.

[pnkfelix]

Accepted for P-backcompat-libs.

[derekchiang]

I'm not sure if anyone is still working on resolving this, but I just want to point out that currently there doesn't seem to be a way to 1. have a TcpSocket block on read, and 2. close the socket after a timeout. Ideally, you would be able to select from network events and timer pipe events, but this can't be done atm. You would think you could run the blocking read in a separate task and have it send data across a pipe, but then while you can indeed select from pipes, you can't terminate the task with the blocking read, because you can't close a task from outside. Anyone has any thoughts how to get around this?

[rapha]

Does #12855 provide a workaround here?

[ghost]

Apologies if this is a dumb question, but is there any particular reason why the Rust pipe cannot be a bog-standard Unix pipe with dummy writes? It seems to me that the only mechanism that could possibly satisfy all these requirements is the file descriptor.

[thestinger]

Unix pipes are very slow relative to efficient concurrent queues. Every write or read requires a system call. If you mean for signalling events, there are better ways to do that in combination with a select-like API (like epoll) on a per-platform basis like eventfd, but it's still significantly slower than a condition variable.

[brson]

Nominating for removal from milestone. Our strategy of keeping I/O and comm select separate, and satisfying use cases as they arise is I think passable for 1.0, and we can take another look at async I/O later.

[pnkfelix]

Removing from milestone and switching to P-high, to reflect demontion in importance as outlined by @brson above (i.e. it will not be end of the world if we have to solve this post 1.0).

[Ericson2314]

With #17325 could simple epoll/kqueue/select/poll wrappers be added to (the new) libnative without much difficulty? That seems like a harmless quick fix, leaving a portable interface that ideally works with rust channels too as a problem for later.

[dtantsur]

I would also vote for reevaluating this. Providing poll-like feature is essential for network programming, e.g. because tasks 1. presumably do not scale that well, 2. do not cover some cases like timeouts

[mhart]

Aw, this bummed me out 😿

Definitely feel like async I/O should be a core concern for a modern systems programming language. ie, important enough to have a standard API in place for 1.0

Looking at https://github.com/carllerche/mio for now (thanks @steveklabnik for the heads up), but would've loved for this to be in core

[mikedilger]

I think 1.0 just means no more code-breaking language-level changes, not necessarily "ready for mainstream usage" or "feature complete." Perhaps they could have chosen some version number less than 1.0 for such a milestone, but what number would you pick? 0.99? 0.100?

I agree on the high importance of async I/O to a systems level language.

[mhart]

I guess I just feel that async code tends to go beyond just the APIs, but actually affects programming idioms themselves – and that you'll avoid a number of inevitable religious wars over async I/O coding paradigms by bedding down a really well-sanctioned and performant standard before "releasing it to the wild" – which essentially is how everyone will see 1.0, regardless of how many caveats are added that it's "only" the language and not the standard library or modules or what have you.

Also, the comparison to other languages will almost certainly follow when 1.0 is released, and if Rust can't hold its weight in developing modern network applications, then I fear it will be prematurely dismissed by many potential advocates.

I'm all for a small well-contained core set of modules a la Node.js, but to add async I/O as an afterthought in a new language just seems a bit... backwards...?

[carllerche]

I disagree about including something like mio in rust std for a few reasons.

1. Leaving the library separate allows it to evolve on its own timeline. Rust, the language, is going to change on a timeline different from that of libraries (like mio). By keeping it separate, users can mix and match the various versions that suit their needs (rust 2.0 & mio 1.0, rust 1.0 & mio 2.0, etc..). Using Cargo makes including & managing dependencies very low overhead.

2. There is no "one size fits all" concurrency paradigm (async is just one concurrency paradigm). Rust supports extremely low level use cases (down to the kernel) and is also ergonomic enough to be used for higher level use cases as well. Paradigms don't match up at these various levels. I would not implement something like haproxy the same way that I would handle processing large data sets.

Anyway, I say let libs compete, and once patterns emerge (post 1.0) then consider standardizing if there is a strong win to do so.

[mhart]

You can apply that argument (make it external) much more so to other modules that have made it into the core set of crates though. graphviz!? getopts? Even in std you've got LruCache, PriorityQueue, half of the std::io module could be made external – Readers/Writers/Streams already represent concurrency paradigms that have been chosen to be included.

I don't think epoll/kqueue/select/poll are too "external" – they make up the backbone of the fastest web servers and databases in use today. Java's had non-blocking I/O for 12 years now!

I'm just gonna be disappointed if/when Rust 1.0 comes out and you won't be able to write a server out of the box that can beat the pants off Java or Go. IMO the announcement of 1.0 should make it compelling for people to write systems software in, not just desktop browsers...

[dtantsur]

While argument about not moving too much to the stdlib is completely valid, I would also argue that select/epoll and non-blocking sockets should be part of stdlib asap. It would be fair to say: we don't include socket library at all. But as stdlib does contain e.g. UdpSocket, I don't see the compelling reason not to include other parts of socket library.

One more thing that async IO is something that you usually expect to use in a system-level language. It's a good selling point (or better it's absence is not really good selling point for a system-level language).

[SirVer]

+1. I was playing around trying to implement a simple chat server and was confused that I could not select! over socket types or streams. Having channels is only half good if you cannot have poll on your sockets.

[lilith]

One of the earliest comments asked about .NET's async system. It's a callback system, like most, but there is a lot of syntactic sugar, and the 3 different systems that are exposed have different levels of overhead.

The most popular is C# async/await, which generates a closure and state machine to provide a continuation callback. Stack and thread context restoration is expensive, but you can opt out per-task.

F# has a simpler implementation, due to its functional nature and the ease of continuation passing.

The original used begin_[task] end_[task], i.e, https://msdn.microsoft.com/en-us/library/system.io.stream.beginread(v=vs.80).aspx and https://msdn.microsoft.com/en-us/library/system.io.stream.endread(v=vs.80).aspx

I am very interested in interop between Rust and other languages, particularly F#, C#, Ruby, and Lua, and making streams interoperable between them.

Are there any traits in the standard library which will describe a (callback+token/state object)-driven stream interface?

[steveklabnik]

Moving to the RFCs repo, like other wishlist items: rust-lang/rfcs#1081