asyncio.exceptions.InvalidStateError: invalid state when restarting external kernel

[vindi97]

Description

What steps will reproduce the problem?

Wehn i change the interpreter and try to restart the kernel

Traceback

Traceback (most recent call last):
  File "C:\Program Files\Spyder\pkgs\jupyter_client\manager.py", line 89, in wrapper
    self._ready.set_result(None)
asyncio.exceptions.InvalidStateError: invalid state

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Spyder\pkgs\qtconsole\manager.py", line 27, in poll
    super().poll()
  File "C:\Program Files\Spyder\pkgs\jupyter_client\restarter.py", line 140, in poll
    self.kernel_manager.restart_kernel(now=True, newports=newports)
  File "C:\Program Files\Spyder\pkgs\jupyter_core\utils\__init__.py", line 166, in wrapped
    return loop.run_until_complete(inner)
  File "asyncio\base_events.py", line 616, in run_until_complete
  File "C:\Program Files\Spyder\pkgs\jupyter_client\manager.py", line 562, in _async_restart_kernel
    await self._async_start_kernel(**self._launch_args)
  File "C:\Program Files\Spyder\pkgs\jupyter_client\manager.py", line 92, in wrapper
    self._ready.set_exception(e)
asyncio.exceptions.InvalidStateError: invalid state

Versions

• Spyder version: 5.4.5 (standalone)
• Python version: 3.8.10 64-bit
• Qt version: 5.15.2
• PyQt5 version: 5.15.9
• Operating System: Windows-10-10.0.19045-SP0

Dependencies

# Mandatory:
atomicwrites >=1.2.0                                                        :  1.4.1 (OK)
chardet >=2.0.0                                                             :  5.2.0 (OK)
cloudpickle >=0.5.0                                                         :  2.2.1 (OK)
cookiecutter >=1.6.0                                                        :  2.3.0 (OK)
diff_match_patch >=20181111                                                 :  20230430 (OK)
intervaltree                                                                :  None (OK)
IPython >=7.31.1,<9.0.0,!=8.8.0,!=8.9.0,!=8.10.0,!=8.11.0,!=8.12.0,!=8.12.1 :  8.12.2 (OK)
jedi >=0.17.2,<0.19.0                                                       :  0.18.2 (OK)
jellyfish >=0.7                                                             :  1.0.0 (OK)
jsonschema >=3.2.0                                                          :  4.19.0 (OK)
keyring >=17.0.0                                                            :  24.2.0 (OK)
nbconvert >=4.0                                                             :  7.8.0 (OK)
numpydoc >=0.6.0                                                            :  1.5.0 (OK)
paramiko >=2.4.0                                                            :  3.3.1 (OK)
parso >=0.7.0,<0.9.0                                                        :  0.8.3 (OK)
pexpect >=4.4.0                                                             :  4.8.0 (OK)
pickleshare >=0.4                                                           :  0.7.5 (OK)
psutil >=5.3                                                                :  5.9.5 (OK)
pygments >=2.0                                                              :  2.16.1 (OK)
pylint >=2.5.0,<3.0                                                         :  2.17.5 (OK)
pylint_venv >=3.0.2                                                         :  3.0.2 (OK)
pyls_spyder >=0.4.0                                                         :  0.4.0 (OK)
pylsp >=1.7.4,<1.8.0                                                        :  1.7.4 (OK)
pylsp_black >=1.2.0,<3.0.0                                                  :  1.3.0 (OK)
qdarkstyle >=3.0.2,<3.2.0                                                   :  3.1 (OK)
qstylizer >=0.2.2                                                           :  0.2.2 (OK)
qtawesome >=1.2.1                                                           :  1.2.3 (OK)
qtconsole >=5.4.2,<5.5.0                                                    :  5.4.3 (OK)
qtpy >=2.1.0                                                                :  2.4.0 (OK)
rtree >=0.9.7                                                               :  1.0.1 (OK)
setuptools >=49.6.0                                                         :  68.1.2 (OK)
sphinx >=0.6.6                                                              :  7.1.2 (OK)
spyder_kernels >=2.4.4,<2.5.0                                               :  2.4.4 (OK)
textdistance >=4.2.0                                                        :  4.5.0 (OK)
three_merge >=0.1.1                                                         :  0.1.1 (OK)
watchdog                                                                    :  3.0.0 (OK)
zmq >=22.1.0                                                                :  25.1.1 (OK)

# Optional:
cython >=0.21                                                               :  3.0.2 (OK)
matplotlib >=3.0.0                                                          :  3.7.2 (OK)
numpy >=1.7                                                                 :  1.24.4 (OK)
pandas >=1.1.1                                                              :  2.0.3 (OK)
scipy >=0.17.0                                                              :  1.10.1 (OK)
sympy >=0.7.3                                                               :  1.12 (OK)

# Spyder plugins:
spyder_terminal.terminalplugin 1.2.2                                        :  1.2.2 (OK)

Environment

Environment

# packages in environment at C:\ProgramData\miniconda3\envs\spyder_env:
#
# Name                    Version                   Build  Channel
asttokens                 2.0.5              pyhd3eb1b0_0  
backcall                  0.2.0              pyhd3eb1b0_0  
bzip2                     1.0.8                he774522_0  
ca-certificates           2023.08.22           haa95532_0  
cloudpickle               2.2.1           py311haa95532_0  
colorama                  0.4.6           py311haa95532_0  
comm                      0.1.2           py311haa95532_0  
debugpy                   1.6.7           py311hd77b12b_0  
decorator                 5.1.1              pyhd3eb1b0_0  
executing                 0.8.3              pyhd3eb1b0_0  
ipykernel                 6.25.0          py311h746a85d_0  
ipython                   8.15.0          py311haa95532_0  
jedi                      0.18.1          py311haa95532_1  
jupyter_client            8.1.0           py311haa95532_0  
jupyter_core              5.3.0           py311haa95532_0  
libffi                    3.4.4                hd77b12b_0  
libsodium                 1.0.18               h62dcd97_0  
matplotlib-inline         0.1.6           py311haa95532_0  
nest-asyncio              1.5.6           py311haa95532_0  
openssl                   3.0.10               h2bbff1b_2  
packaging                 23.1            py311haa95532_0  
parso                     0.8.3              pyhd3eb1b0_0  
pickleshare               0.7.5           pyhd3eb1b0_1003  
pip                       23.2.1          py311haa95532_0  
platformdirs              3.10.0          py311haa95532_0  
prompt-toolkit            3.0.36          py311haa95532_0  
psutil                    5.9.0           py311h2bbff1b_0  
pure_eval                 0.2.2              pyhd3eb1b0_0  
pygments                  2.15.1          py311haa95532_1  
python                    3.11.5               he1021f5_0  
python-dateutil           2.8.2              pyhd3eb1b0_0  
pywin32                   305             py311h2bbff1b_0  
pyzmq                     25.1.0          py311hd77b12b_0  
setuptools                68.0.0          py311haa95532_0  
six                       1.16.0             pyhd3eb1b0_1  
spyder-kernels            2.4.4           py311haa95532_0  
sqlite                    3.41.2               h2bbff1b_0  
stack_data                0.2.0              pyhd3eb1b0_0  
tk                        8.6.12               h2bbff1b_0  
tornado                   6.3.2           py311h2bbff1b_0  
traitlets                 5.7.1           py311haa95532_0  
tzdata                    2023c                h04d1e81_0  
vc                        14.2                 h21ff451_1  
vs2015_runtime            14.27.29016          h5e58377_2  
wcwidth                   0.2.5              pyhd3eb1b0_0  
wheel                     0.38.4          py311haa95532_0  
xz                        5.4.2                h8cc25b3_0  
zeromq                    4.3.4                hd77b12b_0  
zlib                      1.2.13               h8cc25b3_0  

[ccordoba12]

Hey @vindi97, thanks for reporting. This seems like a legit error but I don't understand what's causing it.

@blink1073, could you take a look at this one? Perhaps this exception needs to be caught in Jupyter-client.

[blink1073]

I think this is the same root cause as jupyter/jupyter_client#959. I proposed a fix here, we're waiting on someone to be able to confirm, since I haven't been able to repro.

[ccordoba12]

we're waiting on someone to be able to confirm, since I haven't been able to repro.

@vindi97, could you help us to check that? We can guide you on how to apply @blink1073's patch.

[vindi97]

@

we're waiting on someone to be able to confirm, since I haven't been able to repro.

@vindi97, could you help us to check that? We can guide you on how to apply @blink1073's patch.

Please guide me

[ccordoba12]

Ok, thanks for your help @vindi97! First, you need to navigate to this directory:

C:\Program Files\Spyder\pkgs\jupyter_core\utils

look for the file called __init__.py there and open it in a text editor (e.g. Notepad++ or Spyder).

After that, please search for the following text:

# Run the loop for this thread.

That should point to a single line (line 163 or close that one).

Finally, please replace the three lines below that one

        # Run the loop for this thread.
        if name not in _loop_map:
            _loop_map[name] = asyncio.new_event_loop()
        loop = _loop_map[name]

with these lines instead

        # Run the loop for this thread.
        try:
            loop = asyncio.get_event_loop()
        except RuntimeError:
            loop = asyncio.new_event_loop()
            asyncio.set_event_loop(loop)

If you're using Spyder to edit the file, close it and start it again. If everything goes well, you shouldn't see the error you reported above.

[ccordoba12]

@blink1073, on issue #21371 @vindi97 confirmed that your patch solves the problem reported here (now @vindi97 is having a different one).

Thanks for your input!

[blink1073]

Great! I'll apply the fix and make a release in jupyter_core.

[ccordoba12]

Closing because this was fixed in Jupyter-core 5.3.2.