-
-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Keras progress bars cause jupyterlab to freeze firefox and chrome #4202
Comments
Actually, I also observed the |
Maybe it's not that a big issue anymore in the recent versions. There might still be opportunities for further optimization of transfer of text streams with frequent calls to flush that also have many |
Thanks. So are you saying JupyterLab behaves like the classic notebook right now? The server-side buffer idea would be an issue for the ipython kernel or the jupyter notebook server. |
Yes I believe so but both seem to slow down when running the above snippet. |
This is not too surprising. The updates are likely causing a layout reflow of the notebook, which we know is slow on FF and will be fixed for 1.0 |
For what it's worth, I tried the example above in current jlab master and didn't see any slowdown across the 30 progress bars. FF 59 on macOS 10.13.3. |
cf also #1639 for firefox rendering/layout slowdown issues. |
I am also experiencing freezes using chrome/windows. Does this have to do with the update rate or the absolute number of updates of the bar? - I have also experienced that the bar itself just stops progressing, might be related. |
I am facing same issue.. Running a keras script on Chrome browser v. 67.0.3396.99, After some epochs, the browser hangs and asks for killing the page. I tried also with incognite and still same problem Jupyterlab version: 0.32.1 |
Could have a bit less output, but not nothing like verbose=0. This may "solve" it for most cases. |
Same issue here. Ubuntu 16.04, Chromium 70.0.3, Jupyterlab 0.35.4, Keras 2.2, python 3.6 |
Same issue here. I can not train a Keras model for more than 5 epochs... IMHO this is a blocking issue for using JupyterLab with Keras. |
I've had this same problem. My workaround was to set def _progress_cb(progress, html):
def on_epoch_end(batch, logs):
progress.value += 1
html.value = f"<div><div>Epoch: {batch}</div><div>{logs}</div></div>"
return callbacks.LambdaCallback(
on_epoch_end=on_epoch_end
) Then use it with something like: html = widgets.HTML(value="")
progress = widgets.IntProgress(min=0, max=num_total_epochs, value=0)
display(progress)
display(html)
model.fit(x=x,
y=y,
verbose=0,
callbacks=[_progress_cb(progress, html)]) You obviously loose history in your cell output div but I find TensorBoard more useful for that anyway. |
Nice! Thanks for sharing! |
Same issue here, though it works perfectly with the classical jupyter notebook. |
Hi, any updates regarding this issue? |
Is this still an issue with the 1.0a3 released last week? There has been some recent work on handling |
Yes, this is still an issue in 1.0a3. The code in the OP works great for me in classic notebook, ff 66, macOS. Stops dead in the first progress bar in JupyterLab 1.0a3, ff 66 or chrome 74. I'm going to tag it for 1.0 and try to spend at least some time looking at it, to see if there is an obvious optimization we're missing somewhere. |
@ian-r-rose has some thoughts on this, like that the classic notebook does some update throttling. |
Specifically here |
Thanks to @ian-r-rose for the fix! I just released the fix in 0.35.6. I verified by running @ogrisel's original example. |
I noticed the builtin keras progress bar often causes the jupyterlab UI to freeze after a while (both under firefox and chrome, under Linux at least). I do not observe this issue under the traditional jupyter notebook app.
Here is a minimal reproduction script that does not depend on Keras. I am not sure Keras is implemented that way but might script seem to be able able to reproduce the same behavior as Keras is causing.
Steps to reproduce:
Observed:
Expected:
I suspect that the
\r
not handled in the same way as in jupyter notebook and some datastructure in the browser is growing beyond acceptable.Note that this bug is not the same as #1104 that was already fixed in previous versions of jupyter lab.
This issue was observed using jupyter lab 0.31.10. I have just upgraded to 0.31.12 and it seems to behave a bit better: I now get the following warning after a while:
but firefox still gets much slower than under the original jupyter notebook app after a while.
The text was updated successfully, but these errors were encountered: