chore: Update development (#128)

* docs: enhancing documentation * docs: better quickstart * chore: ubdate github actions to setup-micromamba * docs: remove default channel from environment file * docs: improvements, like QC report (#125) * added .DS_Store to gitignore. * Fixed the overflow of the features section by using the table. * Fixed the broked report link. * fixed typo project * Typo fix controlled * Sample QC report HTML file * Added the link to the QC report in experiment. * Added the assignment QC report. * Add link to QC report in assignment documentation * Update documentation in quickstart.rst. Fixed typos and gramatical mistakes. * Update documentation in index.rst. Fix typos and grammatical mistakes. * Fix typo in installation documentation * Refactor documentation in config.rst --------- Co-authored-by: Max <visze@users.noreply.github.com> * docs: Fixed the link for the QC report in Experiment and Assignment (#126) * added .DS_Store to gitignore. * Fixed the overflow of the features section by using the table. * Fixed the broked report link. * fixed typo project * Typo fix controlled * Sample QC report HTML file * Added the link to the QC report in experiment. * Added the assignment QC report. * Add link to QC report in assignment documentation * Update documentation in quickstart.rst. Fixed typos and gramatical mistakes. * Update documentation in index.rst. Fix typos and grammatical mistakes. * Fix typo in installation documentation * Refactor documentation in config.rst * Update documentation links in assignment.rst and experiment.rst * Testing the iframe html file. * Update documentation links in assignment.rst and experiment.rst --------- Co-authored-by: Max <visze@users.noreply.github.com> * chore: delete not necessary files * docs: automatic versioning * style: automatic version printing of MPRAsnakeflow * fix: memory resources for bbmap (#123) * fix: add memory resources for bbmap * set lower memm in bbmap workflow profile * increasing memory for bmap --------- Co-authored-by: Max Schubach <max.schubach@bih-charite.de> Co-authored-by: Max Schubach <max.schubach@bihealth.de> * fix: Detach from anaconda (#122) * fix: detach from anaconda. Remove defaults conda channels * fixing linting errors * update hashes in dockerfile from lining errors --------- Co-authored-by: Max Schubach <max.schubach@bih-charite.de> * chore(master): release MPRAsnakeflow 0.1.1 (#124) * chore(master): release MPRAsnakeflow 0.1.1 * Update .release-please-manifest.json * Update version.txt * Update CHANGELOG.md --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Max <visze@users.noreply.github.com> * forgot to upgrade two envs * docs: correct link in docs badge --------- Co-authored-by: Max Schubach <max.schubach@bih-charite.de> Co-authored-by: Ali <69039717+bioinformaticsguy@users.noreply.github.com> Co-authored-by: Max Schubach <max.schubach@bihealth.de> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
kircherlab · Oct 17, 2024 · e647309 · e647309
1 parent e8d4007
commit e647309
Show file tree

Hide file tree

Showing 32 changed files with 186 additions and 130 deletions.
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
@@ -13,7 +13,7 @@ jobs:
     steps:
       - uses: actions/checkout@v4
       - name: install micromamba
-        uses: mamba-org/provision-with-micromamba@v16
+        uses: mamba-org/setup-micromamba@v1
         with:
           environment-file: docs/environment.yml
           environment-name: sphinx

diff --git a/.gitignore b/.gitignore
@@ -26,4 +26,5 @@ mix_data
 *data
 *report.html
 *.simg
-*results
+*results
+.DS_Store
diff --git a/.release-please-manifest.json b/.release-please-manifest.json
@@ -1,3 +1,3 @@
 {
-  ".": "0.1.0"
+  ".": "0.1.1"
 }
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,12 @@
 # Changelog
 
+## [0.1.1](https://github.com/kircherlab/MPRAsnakeflow/compare/MPRAsnakeflow-v0.1.0...MPRAsnakeflow-v0.1.1) (2024-09-30)
+
+### Bug Fixes
+
+* Detach from anaconda ([#122](https://github.com/kircherlab/MPRAsnakeflow/issues/122)) ([16bcea2](https://github.com/kircherlab/MPRAsnakeflow/commit/16bcea2f04190a5965ad1865cf30f6dd44f1b6a0))
+* memory resources for bbmap ([#123](https://github.com/kircherlab/MPRAsnakeflow/issues/123)) ([af93f58](https://github.com/kircherlab/MPRAsnakeflow/commit/af93f588e9387ddf91197f5587d36c3481499b38))
+
 ## [0.1.0](https://github.com/kircherlab/MPRAsnakeflow/compare/MPRAsnakeflow-v0.0.1...MPRAsnakeflow-v0.1.0) (2024-09-18)
 
 First release of MPRAsnakeflow! 

diff --git a/Dockerfile b/Dockerfile
@@ -1,90 +1,86 @@
-FROM condaforge/mambaforge:latest
+ARG VERSION=0.1.1
+
+FROM condaforge/miniforge3:latest
 LABEL io.github.snakemake.containerized="true"
 LABEL io.github.snakemake.conda_env_hash="7a57714fe74eb25255d53b45e2095cd8a4dd4fe73db79006353670c432af97b1"
 
 # Step 1: Retrieve conda environments
 
 # Conda environment:
 #   source: workflow/envs/NGmerge.yaml
-#   prefix: /conda-envs/c243bde7dc056785a077f6c33e56e8d6
+#   prefix: /conda-envs/8b5bbd33d7ccdbbe3a28773771abe2b3
 #   ---
 #   channels:
 #     - conda-forge
 #     - bioconda
-#     - defaults
 #   dependencies:
 #     - ngmerge=0.3
 #     - python
-#     - click  
+#     - click
 #     - htslib
-RUN mkdir -p /conda-envs/c243bde7dc056785a077f6c33e56e8d6
-COPY workflow/envs/NGmerge.yaml /conda-envs/c243bde7dc056785a077f6c33e56e8d6/environment.yaml
+RUN mkdir -p /conda-envs/8b5bbd33d7ccdbbe3a28773771abe2b3
+COPY workflow/envs/NGmerge.yaml /conda-envs/8b5bbd33d7ccdbbe3a28773771abe2b3/environment.yaml
 
 # Conda environment:
 #   source: workflow/envs/bbmap_samtools_htslib.yaml
-#   prefix: /conda-envs/575ebc82fb464fb2d0748323abbd3a13
+#   prefix: /conda-envs/f24b0ccfc23aadb93b466380cd592733
 # ---
 # channels:
 #   - bioconda
 #   - conda-forge
-#   - defaults
 # dependencies:
 #   - bbmap
 #   - samtools
 #   - htslib
-RUN mkdir -p /conda-envs/575ebc82fb464fb2d0748323abbd3a13
-COPY workflow/envs/bbmap_samtools_htslib.yaml /conda-envs/575ebc82fb464fb2d0748323abbd3a13/environment.yaml
+RUN mkdir -p /conda-envs/f24b0ccfc23aadb93b466380cd592733
+COPY workflow/envs/bbmap_samtools_htslib.yaml /conda-envs/f24b0ccfc23aadb93b466380cd592733/environment.yaml
 
 # Conda environment:
 #   source: workflow/envs/bwa_samtools_picard_htslib.yaml
-#   prefix: /conda-envs/f354d1f7a8fd64abb8ea8902ec91d399
+#   prefix: /conda-envs/a0a27bbdca540d02023c7fdcf819bfc1
 #   ---
 #   channels:
 #     - bioconda
 #     - conda-forge
-#     - defaults
 #   dependencies:
 #     - bwa
 #     - samtools
 #     - picard
 #     - htslib
-RUN mkdir -p /conda-envs/f354d1f7a8fd64abb8ea8902ec91d399
-COPY workflow/envs/bwa_samtools_picard_htslib.yaml /conda-envs/f354d1f7a8fd64abb8ea8902ec91d399/environment.yaml
+RUN mkdir -p /conda-envs/a0a27bbdca540d02023c7fdcf819bfc1
+COPY workflow/envs/bwa_samtools_picard_htslib.yaml /conda-envs/a0a27bbdca540d02023c7fdcf819bfc1/environment.yaml
 
 # Conda environment:
 #   source: workflow/envs/default.yaml
-#   prefix: /conda-envs/9444545a0ebc79ec516fa74514742720
+#   prefix: /conda-envs/899c0a8c04c6edeed4b242dbc3e0e1c8
 #   ---
 #   channels:
 #     - conda-forge
 #     - bioconda
-#     - defaults
 #   dependencies:
 #     - htslib
-RUN mkdir -p /conda-envs/9444545a0ebc79ec516fa74514742720
-COPY workflow/envs/default.yaml /conda-envs/9444545a0ebc79ec516fa74514742720/environment.yaml
+RUN mkdir -p /conda-envs/899c0a8c04c6edeed4b242dbc3e0e1c8
+COPY workflow/envs/default.yaml /conda-envs/899c0a8c04c6edeed4b242dbc3e0e1c8/environment.yaml
 
 # Conda environment:
 #   source: workflow/envs/fastqsplitter.yaml
-#   prefix: /conda-envs/dc242c7dafc90db387bc0290c31dc7ae
+#   prefix: /conda-envs/e5aec3a0d6b8921994e5305d4f89e90f
 #   ---
 #   channels:
 #     - bioconda
-#     - defaults
 #     - conda-forge
 #   dependencies:
 #     - fastqsplitter
-RUN mkdir -p /conda-envs/dc242c7dafc90db387bc0290c31dc7ae
-COPY workflow/envs/fastqsplitter.yaml /conda-envs/dc242c7dafc90db387bc0290c31dc7ae/environment.yaml
+RUN mkdir -p /conda-envs/e5aec3a0d6b8921994e5305d4f89e90f
+COPY workflow/envs/fastqsplitter.yaml /conda-envs/e5aec3a0d6b8921994e5305d4f89e90f/environment.yaml
 
 # Conda environment:
 #   source: workflow/envs/python3.yaml
-#   prefix: /conda-envs/dadb883da8c83465d38f12e012df0cd0
+#   prefix: /conda-envs/a4e1b935cbca52df9b6f192ff86c464c
 #   ---
 #   channels:
 #     - conda-forge
 #     - bioconda
-#     - defaults
 #   dependencies:
 #     - biopython
 #     - click
@@ -95,84 +91,91 @@ COPY workflow/envs/fastqsplitter.yaml /conda-envs/dc242c7dafc90db387bc0290c31dc7
 #     - python
 #     - pysam
 #     - pyfastx
-RUN mkdir -p /conda-envs/dadb883da8c83465d38f12e012df0cd0
-COPY workflow/envs/python3.yaml /conda-envs/dadb883da8c83465d38f12e012df0cd0/environment.yaml
+RUN mkdir -p /conda-envs/a4e1b935cbca52df9b6f192ff86c464c
+COPY workflow/envs/python3.yaml /conda-envs/a4e1b935cbca52df9b6f192ff86c464c/environment.yaml
 
 # Conda environment:
 #   source: workflow/envs/r.yaml
-#   prefix: /conda-envs/e6c048b22dbbbe081b8d18143c20afe3
+#   prefix: /conda-envs/ae3e37bf43cbb30416a885168e10c552
 #   ---
 #   channels:
 #     - conda-forge
 #     - bioconda
-#     - defaults
 #   dependencies:
 #     - r-base
 #     - r-cowplot
 #     - r-cairo
 #     - r-optparse
 #     - r-tidyverse
-RUN mkdir -p /conda-envs/e6c048b22dbbbe081b8d18143c20afe3
-COPY workflow/envs/r.yaml /conda-envs/e6c048b22dbbbe081b8d18143c20afe3/environment.yaml
+RUN mkdir -p /conda-envs/ae3e37bf43cbb30416a885168e10c552
+COPY workflow/envs/r.yaml /conda-envs/ae3e37bf43cbb30416a885168e10c552/environment.yaml
 
 # Conda environment:
 #   source: workflow/envs/python27.yaml
-#   prefix: /conda-envs/c1d850971f4158052cd52615fbc1591a
+#   prefix: /conda-envs/cb972f023533b03e742da9095ce03b06
 #   ---
 #   channels:
 #     - bioconda
-#     - defaults
 #     - conda-forge
 #   dependencies:
 #     - htslib
 #     - pysam
 #     - python=2.7
 #     - samtools
-RUN mkdir -p /conda-envs/c1d850971f4158052cd52615fbc1591a
-COPY workflow/envs/python27.yaml /conda-envs/c1d850971f4158052cd52615fbc1591a/environment.yaml
+RUN mkdir -p /conda-envs/cb972f023533b03e742da9095ce03b06
+COPY workflow/envs/python27.yaml /conda-envs/cb972f023533b03e742da9095ce03b06/environment.yaml
 
 # Conda environment:
 #   source: workflow/envs/cutadapt.yaml
-#   prefix: /conda-envs/e6c048b22dbbbe081b8d18143c20afe3
+#   prefix: /conda-envs/a3e2fce7f2f6fdbe1aa97232e3def601
 #   ---
 #   channels:
 #     - conda-forge
 #     - bioconda
-#     - defaults
 #    dependencies:
 #     - cutadapt
-RUN mkdir -p /conda-envs/d49adba2589cd2a66656b9298acdbece
-COPY workflow/envs/cutadapt.yaml /conda-envs/d49adba2589cd2a66656b9298acdbece/environment.yaml
+RUN mkdir -p /conda-envs/a3e2fce7f2f6fdbe1aa97232e3def601
+COPY workflow/envs/cutadapt.yaml /conda-envs/a3e2fce7f2f6fdbe1aa97232e3def601/environment.yaml
 
 # Conda environment:
 #   source: workflow/envs/quarto.yaml
-#   prefix: /conda-envs/b8e51d222ab0d9caac2206a127729b1c
+#   prefix: /conda-envs/b933cc1aa7c25db04635e7ec0e37f80e
+#   ---
 #   channels:
 #      - conda-forge
 #      - bioconda
-#      - defaults
 #   dependencies:
 #      - python
 #      - quarto
 #      - jupyter
 #      - pandas
 #      - matplotlib
 #      - papermill
-RUN mkdir -p /conda-envs/b8e51d222ab0d9caac2206a127729b1c
-COPY workflow/envs/quarto.yaml /conda-envs/b8e51d222ab0d9caac2206a127729b1c/environment.yaml
+RUN mkdir -p /conda-envs/b933cc1aa7c25db04635e7ec0e37f80e
+COPY workflow/envs/quarto.yaml /conda-envs/b933cc1aa7c25db04635e7ec0e37f80e/environment.yaml
 
 
 
 # Step 2: Generate conda environments
 
-RUN mamba env create --prefix /conda-envs/c243bde7dc056785a077f6c33e56e8d6 --file /conda-envs/c243bde7dc056785a077f6c33e56e8d6/environment.yaml
-RUN mamba env create --prefix /conda-envs/575ebc82fb464fb2d0748323abbd3a13 --file /conda-envs/575ebc82fb464fb2d0748323abbd3a13/environment.yaml
-RUN mamba env create --prefix /conda-envs/f354d1f7a8fd64abb8ea8902ec91d399 --file /conda-envs/f354d1f7a8fd64abb8ea8902ec91d399/environment.yaml
-RUN mamba env create --prefix /conda-envs/9444545a0ebc79ec516fa74514742720 --file /conda-envs/9444545a0ebc79ec516fa74514742720/environment.yaml
-RUN mamba env create --prefix /conda-envs/dc242c7dafc90db387bc0290c31dc7ae --file /conda-envs/dc242c7dafc90db387bc0290c31dc7ae/environment.yaml
-RUN mamba env create --prefix /conda-envs/dadb883da8c83465d38f12e012df0cd0 --file /conda-envs/dadb883da8c83465d38f12e012df0cd0/environment.yaml
-RUN mamba env create --prefix /conda-envs/e6c048b22dbbbe081b8d18143c20afe3 --file /conda-envs/e6c048b22dbbbe081b8d18143c20afe3/environment.yaml
-RUN mamba env create --prefix /conda-envs/c1d850971f4158052cd52615fbc1591a --file /conda-envs/c1d850971f4158052cd52615fbc1591a/environment.yaml
-RUN mamba env create --prefix /conda-envs/d49adba2589cd2a66656b9298acdbece --file /conda-envs/d49adba2589cd2a66656b9298acdbece/environment.yaml
-RUN mamba env create --prefix /conda-envs/b8e51d222ab0d9caac2206a127729b1c --file /conda-envs/b8e51d222ab0d9caac2206a127729b1c/environment.yaml
-RUN mamba clean --all -y
+RUN <<EOR
+	conda config --add channels nodefaults
+	conda config --add channels bioconda
+	conda config --add channels conda-forge
+	conda config --set channel_priority strict
+EOR
+
+RUN conda env create --no-default-packages --prefix /conda-envs/8b5bbd33d7ccdbbe3a28773771abe2b3 --file /conda-envs/8b5bbd33d7ccdbbe3a28773771abe2b3/environment.yaml
+RUN conda env create --no-default-packages --prefix /conda-envs/f24b0ccfc23aadb93b466380cd592733 --file /conda-envs/f24b0ccfc23aadb93b466380cd592733/environment.yaml
+RUN conda env create --no-default-packages --prefix /conda-envs/a0a27bbdca540d02023c7fdcf819bfc1 --file /conda-envs/a0a27bbdca540d02023c7fdcf819bfc1/environment.yaml
+RUN conda env create --no-default-packages --prefix /conda-envs/a3e2fce7f2f6fdbe1aa97232e3def601 --file /conda-envs/a3e2fce7f2f6fdbe1aa97232e3def601/environment.yaml
+RUN conda env create --no-default-packages --prefix /conda-envs/899c0a8c04c6edeed4b242dbc3e0e1c8 --file /conda-envs/899c0a8c04c6edeed4b242dbc3e0e1c8/environment.yaml
+RUN conda env create --no-default-packages --prefix /conda-envs/e5aec3a0d6b8921994e5305d4f89e90f --file /conda-envs/e5aec3a0d6b8921994e5305d4f89e90f/environment.yaml
+RUN conda env create --no-default-packages --prefix /conda-envs/cb972f023533b03e742da9095ce03b06 --file /conda-envs/cb972f023533b03e742da9095ce03b06/environment.yaml
+RUN conda env create --no-default-packages --prefix /conda-envs/a4e1b935cbca52df9b6f192ff86c464c --file /conda-envs/a4e1b935cbca52df9b6f192ff86c464c/environment.yaml
+RUN conda env create --no-default-packages --prefix /conda-envs/b933cc1aa7c25db04635e7ec0e37f80e --file /conda-envs/b933cc1aa7c25db04635e7ec0e37f80e/environment.yaml
+RUN conda env create --no-default-packages --prefix /conda-envs/ae3e37bf43cbb30416a885168e10c552 --file /conda-envs/ae3e37bf43cbb30416a885168e10c552/environment.yaml
+
+# cleanup when version changed
+ARG VERSION
+RUN conda clean --all -y
diff --git a/README.md b/README.md
@@ -1,6 +1,6 @@
 # Snakemake workflow: MPRAsnakeflow
 
-[![Documentation Status](https://readthedocs.org/projects/mprasnakeflow/badge/?version=latest)](https://mprasnakeflow.readthedocs.io/en/latest/?badge=latest)
+[![Documentation Status](https://readthedocs.org/projects/mprasnakeflow/badge/?version=latest)](https://mprasnakeflow.readthedocs.io/latest/?badge=latest)
 [![Snakemake](https://img.shields.io/badge/snakemake-≥7.2.1-brightgreen.svg)](https://snakemake.bitbucket.io)
 [![Tests](https://github.com/kircherlab/MPRAsnakeflow/actions/workflows/main.yml/badge.svg)](https://github.com/kircherlab/MPRAsnakeflow/actions/workflows/main.yml)
 

diff --git a/docs/assignment.rst b/docs/assignment.rst
@@ -140,6 +140,10 @@ The output can be found in the folder defined by the option :code:`results/assig
 
 Files
 -------------
+Once the pipline is finished running then all the output files can be seen in the results folder. This pipline also generates a qc report. 
+For more details, refer to the `HTML QC report <https://kircherlab.github.io/mprasnakeflow/assignment.html>`_.
+
+
 
 File tree of the result folder (names in :code:`< >` can be specified in the config file.)
 

diff --git a/docs/cluster.rst b/docs/cluster.rst
@@ -11,12 +11,12 @@ Snakemake gives us the opportunity to run MPRAsnakeflow in a cluster environment
     :lines: 1-30
 
 
-We used this workflow successfully in a SLURM environment using the `slurm excecutor plugin <https://snakemake.github.io/snakemake-plugin-catalog/plugins/executor/slurm.html>`_ from snakemake. therfore the partition is set with :code:`slurm_partition` and has to be renamed maybe due to your environment.
+We used this workflow successfully in a SLURM environment using the `slurm excecutor plugin <https://snakemake.github.io/snakemake-plugin-catalog/plugins/executor/slurm.html>`_ from snakemake. Therfore the partition is set with :code:`slurm_partition` and has to be renamed or removed to fith with your own SLURM configuration.
 
 Running with resources
 ----------------------
 
-having 30 cores and 10GB of memory.
+Having 30 cores and 10GB of memory.
 
 .. code-block:: bash
 
@@ -35,13 +35,13 @@ Using the slurm excecutor plugin running 300 jobs in parallel.
 Snakemake 7
 -----------
 
-Here we used the :code:`cluster` option which is not anymore avialable in snakemake 8. You can also use the predefined `config/sbatch.yaml` but this might be outdated and we highly recommend to use resources with the workfloe profile. 
+Here we used the :code:`--cluster` option which is not anymo,onger available in snakemake 8. You can also use the predefined `config/sbatch.yaml` but this might be outdated and we highly recommend to use resources with the workfloe profile. 
 
 .. code-block:: bash
 
     snakemake --use-conda --configfile config/config.yaml --cluster "sbatch --nodes=1 --ntasks={cluster.threads} --mem={cluster.mem} -t {cluster.time} -p {cluster.queue} -o {cluster.output}" --jobs 100 --cluster-config config/sbatch.yaml
 
-Please note that the log folder of the cluster environment has to be generated first, e.g:
+Please note that with this :code:`--cluster` option the log folder of the cluster environment (see :code:` -o {cluster.output}`) has to be generated first, e.g:
 
 .. code-block:: bash
 

diff --git a/docs/combined_example1.rst b/docs/combined_example1.rst
@@ -174,7 +174,7 @@ When dry-drun does not give any errors we will run the workflow. We use a machin
 
 .. code-block:: bash
 
-    snakemake -c 30 --use-conda --snakefile /home/user/MPRAsnakeflow/workflow/Snakefile --configfile /home/user/MPRAsnakeflow/resources/combined_basic/config.yml
+    snakemake -c 30 --sdm conda --snakefile /home/user/MPRAsnakeflow/workflow/Snakefile --configfile /home/user/MPRAsnakeflow/resources/combined_basic/config.yml
 
 
 .. note:: Please modify your code when running in a cluster environment. We have an example SLURM config file here :code:`config/sbatch.yml`.

diff --git a/docs/conf.py b/docs/conf.py
@@ -23,10 +23,12 @@
 copyright = u'2024, Max Schubach'
 author = u'Max Schubach'
 
+f = open("../version.txt", "r")
+
 # The short X.Y version
-version = u'1.0'
+version = f.read().strip()
 # The full version, including alpha/beta/rc tags
-release = u'1.0'
+release = version
 
 
 # -- General configuration ---------------------------------------------------
@@ -161,8 +163,8 @@
 #  dir menu entry, description, category)
 texinfo_documents = [
     (master_doc, 'MPRAsnakeflow', u'MPRAsnakeflow Documentation',
-     author, 'MPRAsnakeflow', 'One line description of project.',
-     'miscellaneous'),
+     author, 'MPRAsnakeflow', 'Snakemake workflow to get assignments or counts from ´MPRA sequencing data.',
+     'MPRA sequencing data workflow'),
 ]
-Original file line number
+Diff line change
@@ Expand Up @@
     Files
     -------------
+    Once the pipline is finished running then all the output files can be seen in the results folder. This pipline also generates a qc report.
+    For more details, refer to the `HTML QC report <https://kircherlab.github.io/mprasnakeflow/assignment.html>`_.
     File tree of the result folder (names in :code:`< >` can be specified in the config file.)
@@ Expand Down @@