From 83fca385bf940485dd82946e376c411c6890b119 Mon Sep 17 00:00:00 2001 From: John Sundh Date: Thu, 21 Nov 2024 07:36:38 +0100 Subject: [PATCH 1/3] Update mermaid theme for conda lecture --- lectures/conda/conda.qmd | 70 ++++++++++++++++------------------------ 1 file changed, 28 insertions(+), 42 deletions(-) diff --git a/lectures/conda/conda.qmd b/lectures/conda/conda.qmd index 89302dd9..15210010 100644 --- a/lectures/conda/conda.qmd +++ b/lectures/conda/conda.qmd @@ -1,13 +1,11 @@ --- title: 'Managing software environments with
' -format: revealjs +format: + revealjs: + mermaid: + theme: neutral execute: echo: false -jupyter: - kernelspec: - display_name: Python 3 - language: python - name: python3 --- ## The problem @@ -19,38 +17,33 @@ _Full reproducibility requires the possibility to recreate the system that was originally used to generate the results._ ::: -## Conda is a package, dependency, and environment manager {auto-animate="true"} +## Conda is a package, dependency, and environment manager - [package]{.green}: any type of program (_e.g._ multiqc, snakemake etc.) -::: {data-id="flowchart1"} ```{mermaid} flowchart LR multiqc(multiqc) ``` -::: -## Conda is a package, dependency, and environment manager {auto-animate="true"} +## Conda is a package, dependency, and environment manager - [package]{.green}: any type of program (_e.g._ multiqc, snakemake etc.) - [dependency]{.green}: other software required by a package -::: {data-id="flowchart1"} ```{mermaid} flowchart LR multiqc(multiqc) -.-> numpy(numpy) multiqc -.-> matplotlib(matplotlib) multiqc -.-> python(python) ``` -::: -## Conda is a package, dependency, and environment manager {auto-animate="true"} +## Conda is a package, dependency, and environment manager - [package]{.green}: any type of program (_e.g._ multiqc, snakemake etc.) - [dependency]{.green}: other software required by a package - dependencies in turn have their own dependencies -::: {data-id="flowchart1"} ```{mermaid} flowchart LR multiqc(multiqc) -.-> numpy(numpy) @@ -62,16 +55,14 @@ flowchart LR numpy -.-> python numpy -.-> libcxx(libcxx) ``` -::: -## Conda is a package, dependency, and environment manager {auto-animate="true"} +## Conda is a package, dependency, and environment manager - [package]{.green}: any type of program (_e.g._ multiqc, snakemake etc.) - [dependency]{.green}: other software required by a package - dependencies in turn have their own dependencies - [environment]{.green}: a distinct collection of packages -::: {data-id="flowchart1"} ```{mermaid} flowchart LR subgraph environment @@ -87,13 +78,11 @@ flowchart LR numpy -.-> libcxx(libcxx) end ``` -::: - -## Conda channels {auto-animate=true} +## Conda channels + [Channels]{.green} are remote directories containing packages -::: {data-id="m1"} ```{mermaid} flowchart TD ch1[(channel1)] --- p1[package1] @@ -104,16 +93,15 @@ flowchart TD ch2[(channel2)] --- p5[package5] ch2[(channel2)] --- p6[package6] ``` -::: -## Conda channels {auto-animate=true} + +## Conda channels Two common examples are: - [bioconda]{.green} (a channel specializing in bioinformatics software) - [conda-forge]{.green} (a community-led channel made up of thousands of contributors) -::: {data-id="m1"} ```{mermaid} flowchart TD ch1[(bioconda)] --- p1[bowtie2] @@ -124,16 +112,14 @@ flowchart TD ch2[(conda-forge)] --- p5[jupyter] ch2[(conda-forge)] --- p6[wget] ``` -::: -## Conda channels {auto-animate=true} +## Conda channels Two common examples are: - [bioconda]{.green} (a channel specializing in bioinformatics software) - [conda-forge]{.green} (a community-led channel made up of thousands of contributors) -::: {data-id="m1"} ```{mermaid} flowchart TD ch1[(bioconda)] --- p1[bowtie2] @@ -147,13 +133,12 @@ flowchart TD p5 -.-> l1([conda install -c conda-forge -c bioconda snakemake jupyter]) p3 -.-> l1 ``` -::: ## Defining and sharing environments Define a Conda environment in an `environment.yml` file: -``` {.yaml code-line-numbers="|1-3|4-12"} +```{.yaml filename="environment.yml" code-line-numbers="|1-3|4-12"} channels: - conda-forge - bioconda @@ -168,47 +153,48 @@ dependencies: - graphviz=2.38.0 ``` -## Defining and sharing environments {auto-animate="true"} +## Conda command-line -- Update an existing environment: +::: {.fragment} +- Create a new environment: ```bash -conda env update -f environment.yml +conda env create -f environment.yml ``` +::: -## Defining and sharing environments {auto-animate="true"} - +::: {.fragment} - Update an existing environment: ```bash conda env update -f environment.yml ``` +::: -- Export environment (including all dependencies) to a file: - -```bash -conda env export > environment.yml -``` - -## Defining and sharing environments {auto-animate="true"} - +::: {.fragment} - Update an existing environment: ```bash conda env update -f environment.yml ``` +::: + +::: {.fragment} - Export environment (including all dependencies) to a file: ```bash conda env export > environment.yml ``` +::: +::: {.fragment} - Export historical environment (only packages explicitly installed): ```bash conda env export --from-history > environment.yml ``` +::: ## Conda, Anaconda, Miniconda, Miniforge... From 6af00016a358ca61913d9847b61cd9298b890016 Mon Sep 17 00:00:00 2001 From: John Sundh Date: Thu, 21 Nov 2024 07:53:46 +0100 Subject: [PATCH 2/3] Update formatting in jupyter lecture --- lectures/jupyter/jupyter.qmd | 68 +++++++++--------------------------- 1 file changed, 16 insertions(+), 52 deletions(-) diff --git a/lectures/jupyter/jupyter.qmd b/lectures/jupyter/jupyter.qmd index 12633880..9d1a7bf2 100644 --- a/lectures/jupyter/jupyter.qmd +++ b/lectures/jupyter/jupyter.qmd @@ -42,7 +42,7 @@ _The next-generation interface for Jupyter_ ![](https://jupyterlab.readthedocs.io/en/stable/_images/jupyterlab.png){height="100%" fig-align=center} :::: :::: {.column width="25%"} -::: {.incremental} +::: {.incremental .small} - Similar to an integrated development environments (IDE). - Tab views, Code consoles, Show output in a separate tab, Live rendering of edits _etc._ - The jupyter lab interface can run Jupyter notebooks in the **main** work area @@ -50,55 +50,39 @@ _The next-generation interface for Jupyter_ :::: ::::: -## Using Jupyter notebooks {auto-animate="true" transition="none" transition-speed="fast" } +## Using Jupyter notebooks -::: {data-id="bullets"} - Document your work in markdown... -::: -::: {data-id="markdown1"} ``` {.default} **Penguin data analysis** Here we will investigate the [Penguin dataset](https://github.com/allisonhorst/palmerpenguins). ---- - The species included in this set are: - - _Adelie_ - - _Chinstrap_ - - _Gentoo_ - ``` -::: +- _Adelie_ +- _Chinstrap_ +- _Gentoo_ +``` -## Using Jupyter notebooks {auto-animate="true" transition="none" transition-speed="fast" } +## Using Jupyter notebooks -::: {data-id="bullets"} - Document your work in markdown...and have it rendered automatically. -::: - -::: {data-id="markdown1"} **Penguin data analysis** Here we will investigate the [Penguin dataset](https://github.com/allisonhorst/palmerpenguins). ---- - The species included in this set are: - - _Adelie_ - - _Chinstrap_ - - _Gentoo_ -::: +- _Adelie_ +- _Chinstrap_ +- _Gentoo_ -## Using Jupyter notebooks {auto-animate="true" transition="none" transition-speed="fast" } +## Using Jupyter notebooks -::: {data-id="bullets"} -- Document your work in markdown...and have it rendered automatically. - Execute code directly from the browser, with results attached to the code which generated them. -::: ```{python} #| echo: true @@ -108,51 +92,31 @@ data = sns.load_dataset("penguins") data.groupby("species").mean(numeric_only=True) ``` -## Using Jupyter notebooks {auto-animate="true" transition="none" transition-speed="fast" } +## Using Jupyter notebooks -::: {data-id="bullets"} -- Document your work in markdown...and have it rendered automatically. -- Execute code directly from the browser, with results attached to the code which generated them. - Mix and match languages in addition to `python`. -::: +::: {.fragment} **R** ```{python} #| echo: true %%R print(paste(Sys.Date(), ": Hello World", sep="")) ``` - -## Using Jupyter notebooks {auto-animate="true" transition="none" transition-speed="fast" } - -::: {data-id="bullets"} -- Document your work in markdown...and have it rendered automatically. -- Execute code directly from the browser, with results attached to the code which generated them. -- Mix and match languages in addition to `python`. ::: -**R** -```{python} -#| echo: true -%%R -print(paste(Sys.Date(), ": Hello World", sep="")) -``` - +::: {.fragment} **bash** ```{python} #| echo: true %%bash echo "$(date): Hello World!" ``` +::: -## Using Jupyter notebooks {auto-animate="true" transition="none" transition-speed="fast" } +## Using Jupyter notebooks -::: {data-id="bullets"} -- Document your work in markdown...and have it rendered automatically -- Execute code directly from the browser, with results attached to the code which generated them -- Mix and match languages in addition to `python`. - Generate plots directly in the browser and/or save to file. -::: ```{python} #| echo: true From 70c57295ba871dfe070b57fe4609b730e3986d7f Mon Sep 17 00:00:00 2001 From: John Sundh Date: Thu, 21 Nov 2024 08:05:59 +0100 Subject: [PATCH 3/3] Update container lecture Mostly small updates to formatting, but also removed Singularity logo and slide --- lectures/containers/containers.qmd | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/lectures/containers/containers.qmd b/lectures/containers/containers.qmd index fc4ddb24..9a3a7ce3 100644 --- a/lectures/containers/containers.qmd +++ b/lectures/containers/containers.qmd @@ -1,8 +1,9 @@ --- -title: 'Working with containers +title: 'Working with containers' +subtitle: ' - ' + ' format: revealjs --- @@ -81,14 +82,20 @@ Status: Downloaded image for ubuntu:16.04 ``` ::: -::: {.fragment} +## An example: using a different OS + +Check OS on local machine +```{.bash} +$ uname -a +Darwin johnsmbp.local 19.6.0 Darwin Kernel Version 19.6.0: [...] x86_64 +``` + Run the container interactively and check OS version ```{.bash code-line-numbers="|1|2|3"} $ docker run -it ubuntu:16.04 root@407b0fd13fe5:/## uname -a Linux 407b0fd13fe5 4.9.60-linuxkit-aufs [...] x86_64 GNU/Linux ``` -::: ## Mounting volumes @@ -168,9 +175,9 @@ docker run \ ``` ::: -## What is Singularity? +## What is Apptainer? -![](https://sylabs.io/guides/3.0/user-guide/_static/logo.png){height=250 fig-align=center} +![](http://apptainer.org/static/hero-img-e614806cf573fd0cbbe09a25d9e69637.png){fig-align=center} ::: {.incremental} - Another software for working with containers, similar to Docker @@ -183,6 +190,7 @@ docker run \ ![](http://apptainer.org/static/hero-img-e614806cf573fd0cbbe09a25d9e69637.png){fig-align=center} ::: {.incremental} +- Apptainer used to be 'Singularity'. - In 2021 Singularity joined the Linux Foundation and became [Apptainer]{.green}. - The company [Sylabs]{.green} still maintains Singularity as a commercial