Can't invoke asyncio event_loop after tornado 5.0 update

[betafcc]

On fresh python3.6 venv, after pip install jupyter && jupyter notebook and starting a new python3.6 notebook:

import asyncio

async def foo():
    return 42

asyncio.get_event_loop().run_until_complete(foo())

throws:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-5-3ad9bf216544> in <module>()
----> 1 asyncio.get_event_loop().run_until_complete(foo())

/usr/local/lib/python3.6/asyncio/base_events.py in run_until_complete(self, future)
    452         future.add_done_callback(_run_until_complete_cb)
    453         try:
--> 454             self.run_forever()
    455         except:
    456             if new_task and future.done() and not future.cancelled():

/usr/local/lib/python3.6/asyncio/base_events.py in run_forever(self)
    406         self._check_closed()
    407         if self.is_running():
--> 408             raise RuntimeError('This event loop is already running')
    409         if events._get_running_loop() is not None:
    410             raise RuntimeError(

RuntimeError: This event loop is already running

If I specify tornado==4.5.3 before pip install jupyter, it works fine

[lschoe]

Same for me (fresh Python 3.6.4 plus Jupyter, on Windows 7 32-bit).

[betafcc]

Probably related to the fact that

"On Python 3, IOLoop is always a wrapper around the asyncio event loop."

as listed in "Backwards-compatibility notes" of tornado 5.0: http://www.tornadoweb.org/en/stable/releases/v5.0.0.html#backwards-compatibility-notes

[takluyver]

I'm not sure at the moment what we can do about this; as the error message says, the event loop is already running, because tornado now runs on top of asyncio. What you want is essentially await, but you can't use that outside a function.

@Carreau I think you were working on something related to this. Any ideas?

[lschoe]

Guess Tornado needs to use its own event loop coupled to a different thread than the one(s) used for the Jupyter notebook cells. Python 3.6 documentation mentions in Section 18.5.2.4 that "The default policy defines context as the current thread, and manages an event loop per thread that interacts with asyncio." Maybe people have already experimented with this?

[takluyver]

I'd be inclined to figure out a way to run the user's coroutines on the existing event loop rather than starting a new thread. Threads cause all sorts of problems.

[Carreau]

Yep, ipython/ipython#10390 should help, but I din't had much chance to work on it. One of the TODO items is to make it work with ipykernel and run things on the current eventloop.

The framework is there it probably does not need much changes to work.

[mingwandroid]

Hey all, just wondering if there is there any update on this issue?

[parmentelat]

I'd like to second this request for an update

As far as I am concerned, this is a major hindrance, as anything remotely useful tends to have some dosage of asyncio these days

[parmentelat]

At the risk of stating the obvious, I'd just like to outline the following discrepency between what I get in a terminal - be it python or ipython - and in a notebook

I am not sure that I quite understand the other discussion there ipython/ipython#10390 but regardless, it is my feeling that something is wrong at this very early point already.

[takluyver]

Yep, that's expected. The kernel itself runs on an event loop, and as of Tornado 5.0, it's using the asyncio event loop. So the asyncio event loop is always running in the kernel. As far as I know, we haven't figured out a way to deal with this yet.

[parmentelat]

but wait, as far as I am concerned this means I can't run anything tainted with asyncio in a notebook

which will asymptotically amount to saying, I can't run anything in a notebook ;)

it that right, or am I missing something obvious ?

[takluyver]

For the moment, that's about right, though I don't think it's ever going to be the case that everything is async. We need to work out a way around it, but so far we haven't got there.

[minrk]

Right, you can't instantiate and run an asyncio eventloop in a thread with an asyncio loop already running. You can run asyncio in a thread, as seen in #11030, which is tedious and we would like it to be unnecessary. That's going to require that we finish #10390 or similar.

You can pin tornado to 4.x while we figure this out.

[btoueg]

I'm facing the same issue while I'm trying to run some code that uses internally:

with closing(asyncio.new_event_loop()) as loop:
    asyncio.set_event_loop(loop)
    return loop.run_until_complete(get_converted_form(msg))

Jupyter gives: Cannot run the event loop while another loop is running

[parmentelat]

to make min's suggestion more concrete, I could work around this issue by just issuing

pip3 install tornado==4.5.3

edit: I expect the jupyter server needs to be restarted as well

[takluyver]

You'll need to restart the kernel - you don't have to restart the whole notebook server.

[andyljones]

I also discovered this because all my asyncio code stopped working after an otherwise innocuous update. For anyone else who comes across this while not quite understanding how event loops work: creating a new event loop won't help, as only one can run at any one time.

Some Google bait:

Spyder IPython Jupyter tornado
RuntimeError: Cannot run the event loop while another loop is running
RuntimeError: This event loop is already running
<_UnixSelectorEventLoop running=True closed=False debug=False>
Can't kill event loop

[simsicon]

I think a possible solution/workaround is to to use the ioloop of tornado, instead of running a new ioloop, but I did not get it right, any ideas?

[Skorpeo]

I have the same issue...the only way I was able to make it work was:

import asyncio
loop = asyncio.get_event_loop()
loop.create_task(some_async_func())

heart = broken

[AIGyan]

@parmentelat
pip3 install tornado==4.5.3 solved the issue

[satra]

@minrk - are there any updates here? it's been a while since this issue was closed, but this should really be open, as the workaround is very specific and is affecting us anytime we install a jupyterhub/lab server, we need to pin tornado to that specific version whenever another package that uses asyncio needs to be present, and there are many more of those today.

is there a separate discussion channel that we should be using to discuss this/figure out a path?

[kevin-bates]

Hi @satra - I believe you meant to reference @minrk. However, I'm not aware of outstanding issues remaining. While there may have been some initial instabilities in this area at the immediate 6.0 timeframe, my understanding (and experience) is that the tornado/asyncio issues are essentially gone.

I would recommend you try upgrading jupyter notebook to 6.0.3 or, better yet, take 6.1.0rc1 for a spin via pip install --upgrade --pre notebook.

[satra]

thank you @kevin-bates - we'll try it out. ping @djarecka and @nicolocin

[djarecka]

@kevin-bates - thank you for your answer, but unfortunately I'm still getting the same error (This event loop is already running) with notebook 6.0.3 or 6.1.0rc1 (tornado==6.0.4). Everything works when using notebook==5.7.8 and tornado==4.5.3
checked for py37 and py38

[kevin-bates]

Where are you seeing this? In the notebook server console or from within a notebook executing a cell?

Please provide the output of jupyter --version.

[satra]

@kevin-bates - here is the response from my environment:

$ jupyter --version
jupyter core     : 4.6.1
jupyter-notebook : 6.0.3
qtconsole        : 4.6.0
ipython          : 7.12.0
ipykernel        : 5.1.4
jupyter client   : 5.3.4
jupyter lab      : 1.2.6
nbconvert        : 5.6.1
ipywidgets       : 7.5.1
nbformat         : 5.0.4
traitlets        : 4.3.3

and the results in jupyter-lab are the same error listed here: #3397 (comment)

[djarecka]

@kevin-bates : we are executing the cell with the jupyter notebook

One of the version I tested is:

jupyter core     : 4.6.3
jupyter-notebook : 6.1.0rc1
qtconsole        : 4.7.5
ipython          : 7.16.1
ipykernel        : 5.3.4
jupyter client   : 6.1.6
jupyter lab      : not installed
nbconvert        : 5.6.1
ipywidgets       : 7.5.1
nbformat         : 5.0.7
traitlets        : 4.3.3

[kevin-bates]

Your environments look fine (@satra, yours is slightly out of date).

Have you tried adjusting your cell's code to use await instead - as @bollwyvl suggests? I would also suggest revisiting these two (older) comments: #3397 (comment) and #3397 (comment)

If you can't make any progress and given this is directly related to IPython and/or ipykernel, I would suggest opening/searching issues in either of those repositories - where more specialized expertise resides.

I had originally thought your issue was more within notebook and/or kernel startup where tornado/async related issues have essentially died down and apologize for not being more aware of your context. Given this is coming directly from a cell, you may need to make adjustments due to tornado's use of asyncio.

[Mirk]

Hi guys, As Kevin said, you contacted the wrong user. I think you confused "@minrk" with "@Mirk".Anyway, good luck with the project. Best,Mirko D'Angelo Il mercoledì 22 luglio 2020, 15:00:00 CEST, Satrajit Ghosh <notifications@github.com> ha scritto: @Mirk - are there any updates here? it's been a while since this issue was closed, but this should really be open, as the workaround is very specific and is affecting us anytime we install a jupyterhub/lab server, we need to pin tornado to that specific version whenever another package that uses asyncio needs to be present, and there are many more of those today. is there a separate discussion channel that we should be using to discuss this/figure out a path? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[satra]

@kevin-bates - we were able to update our notebooks with the nest_asyncio solution and that works now on binder

[ahartikainen]

Hi, running a function in a thread circumvents the problem as shown in here stan-dev/pystan#80

In this case async functions are inside the function func

import concurrent.futures
def exec_async(func, *args, **kwargs):
    with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:
        future = executor.submit(func, *args, **kwargs)
    return future.result()

then calling a function as

result = exec_async(my_fun, arg_1, kwarg_1="hello")

idea was taken from here ipython/ipython#11030

[lmeyerov]

For gremlin users, gremlinpython has been fine w/ tornado 5.x for awhile now. For 6.x (standard jupyter), the main technical requirement is py3.6+, but may require install gremlinpython --no-deps , and TBD, patching: apache/tinkerpop#1212

[riddell-stan]

I agree with @satra, this issue seems like it should be open, not closed. The Python REPL and the IPython REPL work fine, jupyter notebook does not.

The problem still occurs with notebook 6.1.3.

[satra]

@riddell-stan - we were able to make our notebooks work by installing and inserting nest_asyncio at the beginning of the notebook.

import nest_asyncio
nest_asyncio.apply()

(see https://github.com/nipype/pydra-tutorial/blob/master/notebooks/4_intro_workflow.ipynb)

it would be helpful to include this somewhere in the docs if it is indeed a universal solution. if it's not, i agree, this issue should be reopened.

[kevin-bates]

I understand the frustration, but this issue is not happening in the Notebook server process itself - it's happening in the kernel process. Repeating my comment above:

If you can't make any progress and given this is directly related to IPython and/or ipykernel, I would suggest opening/searching issues in either of those repositories - where more specialized expertise resides.

There are no code changes necessary in this repository relative to this particular issue (running async code within a notebook [kernel]). I understand such issues have been handled by this repository in the past. That was back when the active maintainers were also the maintainers in IPython and ipykernel. That's no longer the case and we need to be more diligent about redirecting issues to their appropriate locations. I apologize for this, and a one-stop-shop is always preferred, but that's the only sustainable path forward.

it would be helpful to include this somewhere in the docs if it is indeed a universal solution.

Documentation changes, especially those that help troubleshoot issues, are always welcome. Even if this solution is not universal, there have been enough successes that make a doc update something of value.

[satra]

@kevin-bates - indeed this should go over to ipykernel as the place to troubleshoot/fix. it's just that for most users, they don't understand the distinctions when they use the jupyter notebook/lab

[kevin-bates]

Thank you @satra. I understand and completely agree. It's a very complex stack happening here, coupled with extremely varied technical levels. I have no intention of expecting folks to change how issues are reported. I just believe that redirection should happen as early in the analysis as possible and, over time, users will understand. That said, users are always welcome to start that process here (and rightly so).

[riddell-stan]

@kevin-bates Thanks for the response. IPython works fine for me. The error only occurs in notebook and jupyterlab. So I gather that means it is a bug with ipykernel? Confirming that would help identify where the problem is.

Also, if folks really should direct their inquires elsewhere, perhaps it would be useful to lock this and mark it as wontfix?

[kevin-bates]

IPython works fine for me. The error only occurs in notebook and jupyterlab. So I gather that means it is a bug with ipykernel?

Not necessarily - the actual change could wind up being in IPython, but, yes, a deeper investigation should occur from ipykernel.

perhaps it would be useful to lock this and mark it as wontfix

I hesitate to lock the conversation because I feel that part of things can be useful to people. I have applied the 'wontfix' label however (and anxiously waiting for karma come back at me on this one 😄 ).