Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add docs for DANDI hub #95

Merged
merged 13 commits into from
Jan 9, 2024
Merged

add docs for DANDI hub #95

merged 13 commits into from
Jan 9, 2024

Conversation

bendichter
Copy link
Member

@bendichter bendichter commented Sep 7, 2023

Add docs for accessing DANDI Hub and for example-notebook best practices

docs/50_hub.md Outdated Show resolved Hide resolved
@rly
Copy link

rly commented Sep 7, 2023

Would it be possible for DANDI to create environments and kernels for each DANDI example notebook such that when the notebook is opened, the notebook can be run from an existing kernel?

Currently, even if there is an environment.yml for each notebook, the notebook has to say something like the following to tell the user to set up a conda environment, create a kernel, and use the kernel:

If this is NOT the first time that you are running this notebook, make sure the kernel is set to "Python (demo_000004_upd)". If it is not, change your kernel to "Python (demo_000004_upd)".

If this is the first time that you are running this notebook, open a new Terminal tab in DANDI Hub, and run the following commands to create a new Python kernel called "Python (demo_000004_upd)":

PIP_NO_DEPS=1 mamba env create -f ~/dandi-notebooks/000004/RutishauserLab/environment_updated.yml

PYDEVD_DISABLE_FILE_VALIDATION=1 mamba run --name=demo_000004_upd python -m ipykernel install --user --name=demo_000004_upd --display-name "Python (demo_000004_upd)"

Then, restart your kernel so that the newly defined kernel "Python (demo_000004_upd)" is recognized. In the upper right corner of this notebook, click the current kernel, and change it from "Python 3 (ipykernel)" to "Python (demo_000004_upd)".

@rly
Copy link

rly commented Sep 7, 2023

Perhaps that request would be best implemented via Binder or BinderHub...

@satra
Copy link
Member

satra commented Sep 7, 2023

@rly - 2i2c is working on such a project: https://2i2c.org/blog/2022/gesis-2i2c-collaboration-update/

@bendichter
Copy link
Member Author

alternatively, we could put these instructions in the root README of the example-notebooks repo

@bendichter
Copy link
Member Author

I've opened a PR on the example-notebooks repo here: dandi/example-notebooks#56

@yarikoptic
Copy link
Member

FWIW, I am also checking out davos as a possible way to make those notebooks usable in dedicated environments "turnkey": ContextLab/davos#96

Copy link
Member

@yarikoptic yarikoptic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor changes. Let's finalize/merge this @bendichter !

docs/50_hub.md Outdated Show resolved Hide resolved
docs/50_hub.md Outdated Show resolved Hide resolved
docs/50_hub.md Outdated Show resolved Hide resolved
@yarikoptic yarikoptic self-assigned this Nov 17, 2023
bendichter and others added 2 commits December 18, 2023 11:04
Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
@bendichter bendichter mentioned this pull request Dec 18, 2023
Copy link
Member

@kabilar kabilar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @bendichter. These instructions are great.

(I have a couple of minor suggestions for the Example notebooks section which I will submit in #108 to facilitate the merging of this pull request.)

@kabilar
Copy link
Member

kabilar commented Dec 22, 2023

@bendichter Looks like there is a conflict with the mkdocs.yaml file that you will need to resolve before we can merge.

Copy link
Member

@kabilar kabilar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went ahead and provided my suggestions below since a conflict has to be resolved with this pull request before merging.

docs/50_hub.md Outdated Show resolved Hide resolved
docs/50_hub.md Outdated Show resolved Hide resolved
@yarikoptic yarikoptic removed their assignment Dec 22, 2023

The best way to share analyses on DANDI data is through the DANDI example notebooks.
These notebooks are maintained in https://github.com/dandi/example-notebooks repository which provides more information about their organization.
Dandiset contributors are encouraged to use these notebooks to demonstrate how to read, analyze, and visualize the data, and how to produce figures from associated scientific publications.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO here it would be very valuable to show how to reference those from within dandiset metadata/meditor

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @yarikoptic, would it be best to reference these notebooks in the Related resources section of the Dandiset metadata?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here we are talking about hub... hub is particularly "good" for notebooks. I think it was a right call to mention them here (and may be elsewhere too).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @yarikoptic, sorry, I am confused regarding this suggestion.

docs/50_hub.md Outdated Show resolved Hide resolved
bendichter and others added 2 commits December 23, 2023 15:23
Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
Co-authored-by: Kabilar Gunalan <kabi@mit.edu>
bendichter and others added 2 commits December 23, 2023 15:24
Co-authored-by: Kabilar Gunalan <kabi@mit.edu>
docs/50_hub.md Outdated Show resolved Hide resolved
docs/50_hub.md Outdated Show resolved Hide resolved
bendichter and others added 2 commits January 9, 2024 09:00
Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>
Co-authored-by: Kabilar Gunalan <kabi@mit.edu>
@bendichter
Copy link
Member Author

I've incorporated most of the suggestions. I'm going to merge now as I think this is a substantial value-add as-is. We can always improve it later.

@bendichter bendichter merged commit 9a7fa5c into master Jan 9, 2024
1 check passed
kabilar pushed a commit to kabilar/handbook that referenced this pull request Jan 18, 2024
@kabilar kabilar deleted the hub_docs branch January 18, 2024 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants