Kernel crashes due to upstream problems with zmq package (seen with usage of bokeh and similar packages) #10386

uwezi · 2022-06-09T12:00:53Z

Environment data

VS Code version: 1.67.2
Jupyter Extension version (available under the Extensions sidebar): v2022.4.1021342353
Python Extension version (available under the Extensions sidebar): v2022.6.3
OS (Windows | Mac | Linux distro) and version: Windows 10 64 bit
Python and/or Anaconda version: winpython 3.9.8
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A
Jupyter server running: Local

Expected behaviour

flawless execution of example notebook from bqplot

Actual behaviour

crashing of the IDE

Steps to reproduce:

download "interaction layer.ipynb" from https://github.com/bqplot/bqplot/tree/master/examples/Interactions

Logs

Issue Type: Bug

Opening an example script file for the bqplot library, such as "Interaction Layer.ipynb" from https://github.com/bqplot/bqplot/tree/master/examples/Interactions
worked fine in the previous release of VSCode. Since upgrading on both my computer to the latest VSCode-release (April 2022) the script somehow crashes the IDE. When running the script, operation suddenly ceases and a prompt asks to save changes in the notebook file. Afterwards cell contents has been disappearing.

No change in the script files, nor in the python installation has been done simultaneously. Only change was the update of VSCode.

I observed this first yesterday on my stationary computer, today also my laptop updated itself to the new version of VSCode and now this behavior is persistent on both systems. Reinstallation of the jupyter-extension didn't help.

Extension version: 2022.4.1021342353
VS Code version: Code 1.67.2 (c3511e6c69bb39013c4a4b7b9566ec1ca73fc4d5, 2022-05-17T18:15:52.058Z)
OS version: Windows_NT x64 10.0.19044
Restricted Mode: No

System Info

Item	Value
CPUs	AMD Ryzen 5 3500U with Radeon Vega Mobile Gfx (8 x 2096)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on oop_rasterization: enabled opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_renderer: enabled_on video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled
Load (avg)	undefined
Memory (System)	17.94GB (7.71GB free)
Process Argv	--crash-reporter-id dfd4ad75-9b10-405c-836f-ded4c33df111
Screen Reader	no
VM	0%

A/B Experiments

vsliv368:30146709
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
pythonvspyl392:30443607
pythontb:30283811
pythonvspyt551cf:30345471
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
vscoreces:30445986
pythondataviewer:30285071
vscod805cf:30301675
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593:30376534
vsc1dst:30438360
pythonvs932:30410667
wslgetstarted:30449410
vscscmwlcmt:30465135
cppdebug:30492333


vscaac:30438847
vsclangdf:30486550

The text was updated successfully, but these errors were encountered:

IanMatthewHuff · 2022-06-09T19:10:48Z

@uwezi Thanks for the report here. I actually repro the same thing. Checking into it now. I see this as the possible first error in the developer tools.

FATAL ERROR: Error::New napi_get_last_error_info 1: 0x118d09f85 node::FatalException(v8::Isolate*, v8::TryCatch const&) [/Applications/Visual Studio Code - Insiders.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework] 2: 0x118d0a12f node::OnFatalError(char const*, char const*) [/Applications/Visual Studio Code - Insiders.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework] 3: 0x118d09fa9 node::FatalException(v8::Isolate*, v8::TryCatch const&) [/Applications/Visual Studio Code - Insiders.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework] 4: 0x118cbb2ea napi_fatal_error [/Applications/Visual Studio Code - Insiders.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework] 5: 0x10d14f46d zmq::stream_listener_base_t::create_engine(int) [/Users/ianhuff/.vscode-insiders/extensions/ms-toolsai.jupyter-2022.6.1001611018/out/node_modules/zeromq/prebuilds/darwin-x64/electron.napi.node] 6: 0x10d14f20e zmq::stream_listener_base_t::create_engine(int) [/Users/ianhuff/.vscode-insiders/extensions/ms-toolsai.jupyter-2022.6.1001611018/out/node_modules/zeromq/prebuilds/darwin-x64/electron.napi.node] 7: 0x10d15fd9e napi_register_module_v1 [/Users/ianhuff/.vscode-insiders/extensions/ms-toolsai.jupyter-2022.6.1001611018/out/node_modules/zeromq/prebuilds/darwin-x64/electron.napi.node] 8: 0x10d168fb4 napi_register_module_v1 [/Users/ianhuff/.vscode-insiders/extensions/ms-toolsai.jupyter-2022.6.1001611018/out/node_modules/zeromq/prebuilds/darwin-x64/electron.napi.node] 9: 0x118cba4bb napi_is_detached_arraybuffer [/Applications/Visual Studio Code - Insiders.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]10: 0x11433ea2b v8::internal::Accessors::MakeAccessor(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Name>, void (*)(v8::Local<v8::Name>, v8::PropertyCallbackInfo<v8::Value> const&), void (*)(v8::Local<v8::Name>, v8::Local<v8::Value>, v8::PropertyCallbackInfo<v8::Boolean> const&)) [/Applications/Visual Studio Code - Insiders.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Electron Framework]11: 0x1207f0aa38 12: 0x12003bb998

Not sure on the cause yet, but does seem to be coming from the Jupyter extension.

rchiodo · 2022-06-09T19:54:25Z

Looks like an out of memory error. I wonder if we're keeping messages for ipywidgets in memory by mistake. Might this be related to the work that Don was doing?

DonJayamanne · 2022-06-09T21:36:20Z

@IanMatthewHuff that looks like something that'll crash the node runtime (native node modules can bring down the extension host, in this case we're bringing down the entire VS Code UI) - think somehow our native modules are getting loaded in a context that it shouldn't.

DonJayamanne · 2022-06-09T23:00:31Z

Should test an older version of VS Code, could be due to a change in vscode (version of node shipped with VS Code).

DonJayamanne · 2022-06-09T23:49:38Z

@rchiodo

Might this be related to the work that Don was doing?

Which changes, I don't think I'm caching anything, can't remember, would like to look into it.

rchiodo · 2022-06-09T23:54:43Z

The code were we were not disposing of widgets correctly. Maybe that was in the renderer though.

rchiodo · 2022-06-16T17:05:40Z

We should try this again with Don's new changes.

rchiodo · 2022-06-17T17:57:37Z

This problem only occurs in the desktop ZMQ version. Works fine in web and remote and with ZMQ disabled.

Workaround then is to set this hidden setting (or use remote):

"jupyter.disableZMQSupport": true

uwezi · 2022-06-19T14:07:00Z

This problem only occurs in the desktop ZMQ version. Works fine in web and remote and with ZMQ disabled.

Workaround then is to set this hidden setting (or use remote):
"jupyter.disableZMQSupport": true

As an un-initiated user: where would I find this hidden setting?

rchiodo · 2022-06-20T16:04:25Z

As an un-initiated user: where would I find this hidden setting?

You can just type it into your global settings.json. The schema validation won't recognize it, but it will still allow you to type it.

rchiodo · 2022-06-21T17:04:25Z

Did some more investigation. This is a bug in ZMQ 6. If I switch our code to ZMQ 5, the problem goes away.

I think we have two choices:

Fix the bug in ZMQ 6. This probably requires us building new node modules for ZMQ 6 and maybe shipping from a fork if the owners of ZMQ don't take our change.
Move back to ZMQ 5. This has the problem of the node files in ZMQ 5 don't work with the latest version of electron. I had to run electron-rebuild to get them to work. We'd have to ship new node modules here too.

rchiodo · 2022-06-21T17:06:01Z

ZMQ 5 of our source is here:
https://github.com/microsoft/vscode-jupyter/tree/rchiodo/try_zmq5

IanMatthewHuff · 2022-06-21T17:10:55Z

Ack, that's sticky...

NAPI support was our big reason for moving to ZMQ6. Electron-rebuild works for dev work, but I'm not even sure how we'd ship ZMQ5. VS Code versions update node as needed, extension can't just support a specific single version. I should look and see how others are dealing with this.

rchiodo · 2022-06-21T17:13:22Z

Perhaps we can rebuild the ZMQ5 code to use NAPI. Although seems easier to just fix the bug in 6 version.

IanMatthewHuff · 2022-06-21T17:16:52Z

Per the versions it honestly looks like ZMQ6 might be abandoned? Looks like it stalled out a years ago.
https://www.npmjs.com/package/zeromq

Readme and guide pushed 6.0 for new projects, but it last released in 2019.

rchiodo · 2022-06-21T17:23:35Z

I think we might become the owners or next to update it then. @DonJayamanne is actually going to be building new bits for M1 support for ZMQ 6 based on the latest feedback on this:
#5206

rchiodo · 2022-06-23T20:32:54Z

Here's the callstack of the exit process:

kernel32.dll!ExitProcessImplementation()	Unknown
 	Code - Insiders.exe!exit_or_terminate_process(const unsigned int return_code) Line 143	C++
 	Code - Insiders.exe!common_exit(const int return_code, const _crt_exit_cleanup_mode cleanup_mode, const _crt_exit_return_mode return_mode) Line 280	C++
 	Code - Insiders.exe!node::Abort() Line 260	C++
 	Code - Insiders.exe!node::OnFatalError(const char * location, const char * message) Line 450	C++
 	Code - Insiders.exe!node::FatalError(const char * location, const char * message) Line 421	C++
 	Code - Insiders.exe!napi_fatal_error(const char * location, unsigned __int64 location_len, const char * message, unsigned __int64 message_len) Line 763	C++
 	electron.napi.node!00007ff94b9d2b09()	Unknown
 	electron.napi.node!00007ff94b9e80f4()	Unknown
 	electron.napi.node!00007ff94b9e8725()	Unknown
 	electron.napi.node!00007ff94b9fd963()	Unknown
 	electron.napi.node!00007ff94b9f48f3()	Unknown
 	[Inline Frame] Code - Insiders.exe!v8impl::`anonymous namespace'::CallbackWrapperBase::InvokeCallback::<lambda_1>::operator()(napi_env__ * env) Line 312	C++

rchiodo · 2022-06-23T20:34:35Z

Ipyvolume and fileupload also crash this way.

uwezi · 2022-07-01T11:41:16Z

The problem persists in VS Code 1.68.1

Bartel-C8 · 2022-07-04T10:51:36Z

Related to this (closed) issue, which was not reproducible:

#4822

corbt · 2022-11-04T05:52:28Z

Problem still exists in VS Code 1.72.2. Trying to upload a file using ipywidgets.FileUpload crashes VS Code.

corbt · 2022-11-04T05:54:42Z

This problem only occurs in the desktop ZMQ version. Works fine in web and remote and with ZMQ disabled.

Workaround then is to set this hidden setting (or use remote):
"jupyter.disableZMQSupport": true

This workaround appears to no longer work? When I enable this setting and run my notebook, I get the following error:

originalException: a [Error]: '_xsrf' argument missing from POST
      at Function.create (/home/ubuntu/.vscode-server/extensions/ms-toolsai.jupyter-2022.9.1202862440/out/node_modules/@jupyterlab/services.js:2:53387)

aminya · 2022-11-15T09:37:17Z

Could you try the latest beta version of zeromq to see if the issue is resolved?

https://github.com/zeromq/zeromq.js/releases

brentkendrick · 2023-04-21T13:50:14Z

Disabling ZMQ support fix works for me with VS Code version 1.77.3 using Windows.

DonJayamanne · 2023-12-01T01:12:56Z

Closing this issue as this hasn't been reported for a while.

uwezi added the bug Issue identified by VS Code Team member as probable bug label Jun 9, 2022

github-actions bot added the triage-needed Issue needs to be triaged label Jun 9, 2022

github-actions bot assigned IanMatthewHuff Jun 9, 2022

IanMatthewHuff added the ipywidgets Rendering, loading, saving, anything to do with IPyWidgets label Jun 9, 2022

rchiodo added notebook-regression and removed triage-needed Issue needs to be triaged labels Jun 9, 2022

rchiodo added this to the June 2022 milestone Jun 9, 2022

greazer added nb-golden and removed iteration-candidate labels Jun 10, 2022

rchiodo mentioned this issue Jun 17, 2022

Scrolling cells off screen prevents rendering widgets #10485

Closed

DonJayamanne added the zmq Issues related to zmq label Jun 17, 2022

rchiodo added triage-needed Issue needs to be triaged discuss-at-standup labels Jun 21, 2022

greazer added the upstream-zmq label Jun 21, 2022

rchiodo mentioned this issue Jun 23, 2022

File upload widget crashes extension host when using non-jupyter (zmq) #10525

Closed

rchiodo unassigned IanMatthewHuff Jun 23, 2022

rchiodo added iteration-candidate and removed triage-needed Issue needs to be triaged labels Jun 23, 2022

rchiodo removed this from the June 2022 milestone Jun 23, 2022

rchiodo removed the discuss-at-standup label Jun 23, 2022

greazer removed the nb-golden label Jun 27, 2022

rebornix mentioned this issue Jul 1, 2022

VS Code extension host napi crash microsoft/vscode#153878

Closed

DonJayamanne changed the title ~~VScode 1.67.2 - jupyter issue with interactive plots using bqplot~~ Interactive plots using bqplot due to an error with zmq Jul 4, 2022

Bartel-C8 mentioned this issue Jul 5, 2022

[6.x] Troubleshooting memory leaks zeromq/zeromq.js#466

Closed

amunger mentioned this issue Jul 25, 2022

Jupyter Notebook will not render ipyaggrid. #10905

Closed

2 tasks

rebornix assigned DonJayamanne Aug 12, 2022

DonJayamanne mentioned this issue Nov 25, 2022

Releasing v6.0.0 zeromq/zeromq.js#529

Closed

DonJayamanne removed the upstream-zmq label Dec 4, 2022

DonJayamanne changed the title ~~Interactive plots using bqplot due to an error with zmq~~ Kernel crashes due to upstream problems with zmq package (seen with usage of bokeh and similar packages) Dec 5, 2022

DonJayamanne mentioned this issue Dec 5, 2022

"Socket is busy writing" error when interacting with large dataset using ipyleaflet in VS code #10188

Closed

2 tasks

DonJayamanne removed the notebook-regression label Dec 8, 2022

DonJayamanne removed the iteration-candidate label Dec 27, 2022

DonJayamanne closed this as completed Dec 1, 2023

github-actions bot locked as resolved and limited conversation to collaborators Jan 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Kernel crashes due to upstream problems with zmq package (seen with usage of bokeh and similar packages) #10386

Kernel crashes due to upstream problems with zmq package (seen with usage of bokeh and similar packages) #10386

uwezi commented Jun 9, 2022

IanMatthewHuff commented Jun 9, 2022

rchiodo commented Jun 9, 2022

DonJayamanne commented Jun 9, 2022

DonJayamanne commented Jun 9, 2022

DonJayamanne commented Jun 9, 2022

rchiodo commented Jun 9, 2022

rchiodo commented Jun 16, 2022

rchiodo commented Jun 17, 2022 •

edited

Loading

uwezi commented Jun 19, 2022 •

edited

Loading

rchiodo commented Jun 20, 2022

rchiodo commented Jun 21, 2022

rchiodo commented Jun 21, 2022

IanMatthewHuff commented Jun 21, 2022

rchiodo commented Jun 21, 2022

IanMatthewHuff commented Jun 21, 2022

rchiodo commented Jun 21, 2022

rchiodo commented Jun 23, 2022

rchiodo commented Jun 23, 2022

uwezi commented Jul 1, 2022 •

edited

Loading

Bartel-C8 commented Jul 4, 2022

corbt commented Nov 4, 2022

corbt commented Nov 4, 2022

aminya commented Nov 15, 2022 •

edited

Loading

brentkendrick commented Apr 21, 2023

DonJayamanne commented Dec 1, 2023

Kernel crashes due to upstream problems with zmq package (seen with usage of bokeh and similar packages) #10386

Kernel crashes due to upstream problems with zmq package (seen with usage of bokeh and similar packages) #10386

Comments

uwezi commented Jun 9, 2022

Environment data

Expected behaviour

Actual behaviour

Steps to reproduce:

Logs

IanMatthewHuff commented Jun 9, 2022

rchiodo commented Jun 9, 2022

DonJayamanne commented Jun 9, 2022

DonJayamanne commented Jun 9, 2022

DonJayamanne commented Jun 9, 2022

rchiodo commented Jun 9, 2022

rchiodo commented Jun 16, 2022

rchiodo commented Jun 17, 2022 • edited Loading

uwezi commented Jun 19, 2022 • edited Loading

rchiodo commented Jun 20, 2022

rchiodo commented Jun 21, 2022

rchiodo commented Jun 21, 2022

IanMatthewHuff commented Jun 21, 2022

rchiodo commented Jun 21, 2022

IanMatthewHuff commented Jun 21, 2022

rchiodo commented Jun 21, 2022

rchiodo commented Jun 23, 2022

rchiodo commented Jun 23, 2022

uwezi commented Jul 1, 2022 • edited Loading

Bartel-C8 commented Jul 4, 2022

corbt commented Nov 4, 2022

corbt commented Nov 4, 2022

aminya commented Nov 15, 2022 • edited Loading

brentkendrick commented Apr 21, 2023

DonJayamanne commented Dec 1, 2023

rchiodo commented Jun 17, 2022 •

edited

Loading

uwezi commented Jun 19, 2022 •

edited

Loading

uwezi commented Jul 1, 2022 •

edited

Loading

aminya commented Nov 15, 2022 •

edited

Loading