pip does not know of required pyzmq

[solstag]

On an up to date Fedora, using python3, pip install does not grab the required version of pyzmq, leading notebook to try the system version and fail.

$ pip install notebook
(...works fine...)

$ jupyter-notebook 
Traceback (most recent call last):
  File "/home/narf/.local/bin/jupyter-notebook", line 7, in <module>
    from notebook.notebookapp import main
  File "/home/narf/.local/lib/python3.6/site-packages/notebook/notebookapp.py", line 45, in <module>
    ioloop.install()
  File "/usr/lib64/python3.6/site-packages/zmq/eventloop/ioloop.py", line 210, in install
    assert (not ioloop.IOLoop.initialized()) or \
AttributeError: type object 'IOLoop' has no attribute 'initialized'

$ pip install --user --upgrade pyzmq

$ jupyter-notebook 
(...works fine...)

[takluyver]

This is a fiddly one. The notebook can run fine on older versions of pyzmq, but in that case you need an older version of tornado as well. Tornado 5 is not compatible with pyzmq < 17, but neither of those packages depend on the other.

[solstag]

In that case, this is a bug in Tornado?
Pip was happy to install Tornado 5 without requiring an updated pyzmq.
If Tornado 5 does not require pyzmq, but if using it then requires that it be not < 17, perhaps it should check and give a more informative error at loading time?
Or perhaps, since it's Notebook that requires both, it could check the versions when started and tell the user what are his options (upgrade pyzmq/downgrade tornado)?
Thanks!

[takluyver]

It's definitely not a bug in tornado. I'm coming round to the idea that we should do something about this, though, because I think one of the changes in pip 10 makes it more likely that people will hit it. Specifically, pip install --upgrade now only upgrades dependencies if it needs to satisfy a version requirement, whereas previously it upgraded all dependencies.

@minrk would you be OK with notebook 5.5 requiring pyzmq >= 17? I know that it doesn't really, but it makes it less likely that people will hit this?

[minrk]

I'm not a big fan, but I think it's an okay solution if this is going to keep coming up.

[takluyver]

OK, #3586 does that.

[solstag]

Just a final note, in case it is relevant. I see now that when installing notebook with a clean account, it just works because tornado does not get updated. My tornado must have gotten updated with older pip versions that were updating all dependencies. So in a way this is a symptom of the change in behavior of pip. If you consistently used pip<10, things are going to be fine 'cause you'll get all the new dependencies, and if you use new pip>10 from the start things will be fine because these dependencies will all remain at their older versions. In any case, given that this pip transition is ongoing, it does seem reasonable to bump the requirements and make it works regardless of pip version changes.

[solstag]

And of course, thanks!