Add data assimilation notebook

[richardarsenault]

Overview

This PR adds the data assimilation notebook, the final one in the tutorial series. Also updates the doc index to add it to the list.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[Zeitsperre]

From ReadTheDocs:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/pavics-raven/conda/421/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/docs/checkouts/readthedocs.org/user_builds/pavics-raven/conda/421/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pavics-raven/conda/421/lib/python3.10/site-packages/sphinx/__main__.py", line 13, in <module>
    from sphinx.cmd.build import main
  File "/home/docs/checkouts/readthedocs.org/user_builds/pavics-raven/conda/421/lib/python3.10/site-packages/sphinx/cmd/build.py", line 25, in <module>
    from sphinx.application import Sphinx
  File "/home/docs/checkouts/readthedocs.org/user_builds/pavics-raven/conda/421/lib/python3.10/site-packages/sphinx/application.py", line 32, in <module>
    from sphinx.config import Config
  File "/home/docs/checkouts/readthedocs.org/user_builds/pavics-raven/conda/421/lib/python3.10/site-packages/sphinx/config.py", line 23, in <module>
    from sphinx.util import logging
  File "/home/docs/checkouts/readthedocs.org/user_builds/pavics-raven/conda/421/lib/python3.10/site-packages/sphinx/util/__init__.py", line 35, in <module>
    from sphinx.util import smartypants  # noqa
  File "/home/docs/checkouts/readthedocs.org/user_builds/pavics-raven/conda/421/lib/python3.10/site-packages/sphinx/util/smartypants.py", line 33, in <module>
    from sphinx.util.docutils import __version_info__ as docutils_version
  File "/home/docs/checkouts/readthedocs.org/user_builds/pavics-raven/conda/421/lib/python3.10/site-packages/sphinx/util/docutils.py", line 31, in <module>
    from sphinx.util.typing import RoleFunction
  File "/home/docs/checkouts/readthedocs.org/user_builds/pavics-raven/conda/421/lib/python3.10/site-packages/sphinx/util/typing.py", line 34, in <module>
    from types import Union as types_Union
ImportError: cannot import name 'Union' from 'types' (/home/docs/checkouts/readthedocs.org/user_builds/pavics-raven/conda/421/lib/python3.10/types.py)

Looks as though someone pushed a bug to sphinx (Union is in typing, not types).

[richardarsenault]

@Zeitsperre I'm pretty sure I didn't do it though... possible someone else pushed that somehow? Or else I am at a loss how I could have modified the sphinx package!

[Zeitsperre]

Definitely not you, @richardarsenault, but I can see that we're about a year behind on Sphinx versions and RtD is now using Python3.10. Still, that import is verifiably wrong. If the next build fails, I'll see if there's something I can do about it.

[huard]

Edited, please review to make sure I did not screw anything.

Added an interactive graph at the end, keep only one of both.

[richardarsenault]

@huard It seems I can't even import the data assimilation package because it can't find perturb_full_series (and I imagine the others that I wrote). I tried forcing raven version 0.7.8 to no avail, same issue. I was able to run it before though... so could it be on my pavics server instance?

[richardarsenault]

Small update and more info :

This notebook worked on the server (pavics.ouranos.ca jputerlab instance) last week when I worked on and submitted it here for review, so I am pretty sure the server had ravenpy 0.7.8 installed -- either automatically or because I had forced it using:

!conda install -c conda-forge ravenpy>=0.7.8 -y

Today, I uploaded the new version after the commit from @huard (c2eeaaf) on the jupyterlab to test it out. But now at this line:

from ravenpy.utilities.data_assimilation import perturb_full_series, sequential_assimilation, assimilation_initialization

I get an error: cannot import name 'perturb_full_series' from 'ravenpy.utilities.data_assimilation', which I had in version 0.7.7 because the functions just didn't exist yet. But it should work in 0.7.8.

No big deal I thought, I'll just re-force the ravenpy 0.7.8 package using the command from above... but it still fails with the same error.

@tlvu Perhaps there is something I did to revert versions, but I doubt it. Any ideas? Has the server perhaps been reverted to 0.7.7 and blocking me from forcing 0.7.8? It is also probably a "user error - replace user" kind of thing so if you have any idea how this could be fixed I'd be glad to test them! Thanks!

[richardarsenault]

Just as I finished typing my previous comment I thought of the fact that I can test for the ravenpy version...

import ravenpy
ravenpy.__version__

'0.7.5'

So I guess this is the problem, but no idea why I can't supersede it with 0.7.8?

[tlvu]

@tlvu Perhaps there is something I did to revert versions, but I doubt it. Any ideas? Has the server perhaps been reverted to 0.7.7 and blocking me from forcing 0.7.8? It is also probably a "user error - replace user" kind of thing so if you have any idea how this could be fixed I'd be glad to test them! Thanks!

@richardarsenault

There has been a full server restart yesterday. You Jupyter server has reset itself to the original state with raven 0.7.5.

We sometime have to restart for various reasons so do not be surprised next time if your customization seems to vanish. We also have an "auto kill idle Jupyter server" to avoid resource exhaustion on our system so your Jupyter server could also vanish even if we did not intentionally restart the full system.

However, good news for you, there is currently 3 flavors of the Jupyter server to choose from, in production. If you select the "beta" flavor right now, you'll get ravenpy 0.7.8. This "beta" version will most likely become the "current" version next week.

If you do not get the prompt below, shutdown your currently Jupyter server and restart it again.

[richardarsenault]

Thanks @tlvu ! It worked like a charm 👍