OutStream.close raises if watchfd=False

[mdickinson]

Closing an OutStream instance via its close method fails if that OutStream instance is created with watchfd=False, with an exception AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'.

Here's a full traceback from some Envisage tests. We're creating a kernel app with kernel = IPKernelApp.instance(capture_fd_output=False), and then later shutting that kernel down. Only the last two lines are directly relevant for ipykernel. I'll see if I can come up with a more self-contained reproducer.

======================================================================
ERROR: test_ipython_util_io_globals_restored (envisage.plugins.ipython_kernel.tests.test_internal_ipkernel.TestInternalIPKernel)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/mdickinson/Enthought/ETS/envisage/envisage/plugins/ipython_kernel/tests/test_internal_ipkernel.py", line 162, in test_ipython_util_io_globals_restored
    self.create_and_destroy_kernel()
  File "/Users/mdickinson/Enthought/ETS/envisage/envisage/plugins/ipython_kernel/tests/test_internal_ipkernel.py", line 319, in create_and_destroy_kernel
    kernel.shutdown()
  File "/Users/mdickinson/Enthought/ETS/envisage/envisage/plugins/ipython_kernel/internal_ipkernel.py", line 133, in shutdown
    self.ipkernel.close()
  File "/Users/mdickinson/Enthought/ETS/envisage/envisage/plugins/ipython_kernel/kernelapp.py", line 202, in close
    self.close_io()
  File "/Users/mdickinson/Enthought/ETS/envisage/envisage/plugins/ipython_kernel/kernelapp.py", line 296, in close_io
    sys.stderr.close()
  File "/Users/mdickinson/.venvs/envisage/lib/python3.10/site-packages/ipykernel/iostream.py", line 429, in close
    self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'

----------------------------------------------------------------------
Ran 1 test in 0.161s

FAILED (errors=1)

From the source, the cause looks fairly clear: at OutStream creation time, the _should_watch and watch_fd_thread attributes are only created if watchfd=True, but the close method assumes that both attributes exist unconditionally. (Well, at least on Linux and macOS, anyway.)

ipykernel version: 6.9.1
Python version: 3.10.2
OS + hardware: macOS 11.6.3 (Big Sur), Intel MacBook Pro

[mdickinson]

Here's a minimal reproducer on my machine:

import zmq

from jupyter_client.session import Session
from ipykernel.iostream import OutStream, IOPubThread


def test():
    context = zmq.Context()
    pub_thread = IOPubThread(socket=context.socket(zmq.PUB))
    pub_thread.start()

    stream = OutStream(
        session=Session(),
        pub_thread=pub_thread,
        name="stderr",
        watchfd=False,
    )
    stream.close()

    pub_thread.stop()
    context.destroy()

test()

Here's a complete session: where I:

• set up a new Python 3.10 venv
• install the latest ipykernel from PyPI
• run the script above

mdickinson@mirzakhani Desktop % python -VV
Python 3.10.2 (main, Jan 15 2022, 10:49:49) [Clang 12.0.5 (clang-1205.0.22.11)]
mdickinson@mirzakhani Desktop % python -m venv --clear ~/.venvs/testing && source ~/.venvs/testing/bin/activate
(testing) mdickinson@mirzakhani Desktop % python -m pip install -q --upgrade pip setuptools wheel
(testing) mdickinson@mirzakhani Desktop % python -m pip install -q --upgrade ipykernel           
(testing) mdickinson@mirzakhani Desktop % pip list
Package           Version
----------------- -------
appnope           0.1.2
asttokens         2.0.5
backcall          0.2.0
black             22.1.0
click             8.0.3
debugpy           1.5.1
decorator         5.1.1
entrypoints       0.4
executing         0.8.2
ipykernel         6.9.1
ipython           8.0.1
jedi              0.18.1
jupyter-client    7.1.2
jupyter-core      4.9.2
matplotlib-inline 0.1.3
mypy-extensions   0.4.3
nest-asyncio      1.5.4
parso             0.8.3
pathspec          0.9.0
pexpect           4.8.0
pickleshare       0.7.5
pip               22.0.3
platformdirs      2.5.0
prompt-toolkit    3.0.28
ptyprocess        0.7.0
pure-eval         0.2.2
Pygments          2.11.2
python-dateutil   2.8.2
pyzmq             22.3.0
setuptools        60.9.1
six               1.16.0
stack-data        0.2.0
tomli             2.0.1
tornado           6.1
traitlets         5.1.1
wcwidth           0.2.5
wheel             0.37.1
(testing) mdickinson@mirzakhani Desktop % cat test.py
import zmq

from jupyter_client.session import Session
from ipykernel.iostream import OutStream, IOPubThread


def test():
    context = zmq.Context()
    pub_thread = IOPubThread(socket=context.socket(zmq.PUB))
    pub_thread.start()

    stream = OutStream(
        session=Session(),
        pub_thread=pub_thread,
        name="stderr",
        watchfd=False,
    )
    stream.close()

    pub_thread.stop()
    context.destroy()

test()
(testing) mdickinson@mirzakhani Desktop % python test.py 
Traceback (most recent call last):
  File "/Users/mdickinson/Desktop/test.py", line 23, in <module>
    test()
  File "/Users/mdickinson/Desktop/test.py", line 18, in test
    stream.close()
  File "/Users/mdickinson/.venvs/testing/lib/python3.10/site-packages/ipykernel/iostream.py", line 429, in close
    self.watch_fd_thread.join()
AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'