tornado 5 fixes in ThreadedClient #352
Conversation
calling stop doesn’t wake the IOLoop with asyncio (tornado 5)
- avoid instantiating an IOLoop outside the thread in which it will be used, which sometimes causes problems. - ensure asyncio eventloop is defined in the thread, if asyncio might be in use
| # tornado may be using asyncio, | ||
| # ensure an eventloop exists for this thread | ||
| import asyncio | ||
| asyncio.set_event_loop(asyncio.new_event_loop()) |
There was a problem hiding this comment.
It seems odd that we have to do this. Is there a neater way that we can use in the future when we can assume that tornado is using asyncio?
There was a problem hiding this comment.
The situation: asyncio will not create an eventloop in a thread, you have to tell it to, at least in certain circumstances. Tornado is running on top of asyncio, but won't create the required asyncio eventloop if there isn't one (asyncio.get_event_loop will fail in threads that haven't initialized eventloops explicitly). I'm not sure if this is a tornado bug or not.
We can ask if the tornado configured IOLoop class is a subclass of AsyncIOLoop, which is the tornado asyncio wrapper implementation. If we assume that's going to be the case and not some other weird requires-asyncio implementation that doesn't subclass it.
Note that this is just instantiating an object, not starting it or anything, so it's a pretty minimal operation and harmless if the thread-local eventloop goes unused. I'm not quite sure what the cleanest solution is for "create the asyncio eventloop that tornado may need in this thread only if tornado is actually going to need it", because we would need more detailed try-except for importing tornado.platform.asyncio.AsyncIOLoop which may not be defined (e.g. python 2 or older tornado) in order to call issubclass().
There was a problem hiding this comment.
You could consider setting our own policy, so that we handle the calls to get_event_loop.
There was a problem hiding this comment.
We shouldn't set our own policy in jupyter_client because that would preclude applications setting their own policies. We could set our own in qtconsoleapp, but can't rely on that in the qtconsole kernelmanager, which can be used in other applications like Spyder.
|
I tested this against qtconsole and Spyder and it works as expected. Thanks @minrk! |
|
will there be a patch release soon so everybody can enjoy Tornado5 ? |
The main issue was failing to schedule `ioloop.stop` in the ioloop thread, which is required when tornado is running on asyncio in order to wake the thread. The result of not doing this is hanging forever when trying to exit, e.g. in QtConsole. There is further cleanup of threadsafety issues with respect to asyncio and tornado objects. closes jupyter/qtconsole#275 cc ccordoba12 Signed-off-by: Min RK <benjaminrk@gmail.com>
|
@stonebig yes, hopefully tomorrow. I'll open a PR with a changelog for 5.2.3 and it should be out shortly. |
The main issue was failing to schedule
ioloop.stopin the ioloop thread, which is required when tornado is running on asyncio in order to wake the thread. The result of not doing this is hanging forever when trying to exit, e.g. in QtConsole.There is further cleanup of threadsafety issues with respect to asyncio and tornado objects.
closes jupyter/qtconsole#275
cc @ccordoba12