Merge pull request nf-core#3358 from mirpedrol/fix-pre-commit-template

Fix pre commit template

CHANGELOG.md

@@ -5,6 +5,7 @@
 ### Template
 
 - Use outputs instead of the environment to pass around values between steps in the Download Test Action ([#3351](https://github.com/nf-core/tools/pull/3351))
+- Fix pre commit template ([#3358](https://github.com/nf-core/tools/pull/3358))
 
 ### Download
 

nf_core/pipeline-template/.github/ISSUE_TEMPLATE/bug_report.yml

@@ -11,7 +11,6 @@ body:
         - [nf-core website: troubleshooting](https://nf-co.re/usage/troubleshooting)
         - [{{ name }} pipeline documentation](https://nf-co.re/{{ short_name }}/usage)
 {%- endif %}
-
   - type: textarea
     id: description
     attributes:

nf_core/pipeline-template/.gitpod.yml

@@ -3,7 +3,7 @@ tasks:
   - name: Update Nextflow and setup pre-commit
     command: |
       pre-commit install --install-hooks
-      nextflow self-update {% if code_linters %}
+      nextflow self-update {%- if code_linters %}
 
 vscode:
   extensions:

nf_core/pipeline-template/.prettierignore

@@ -16,3 +16,6 @@ testing/
 testing*
 *.pyc
 bin/
+{%- if rocrate %}
+ro-crate-metadata.json
+{%- endif %}

nf_core/pipeline-template/CITATIONS.md

@@ -18,7 +18,7 @@
 
 {%- endif %}
 
-{% if multiqc %}- [MultiQC](https://pubmed.ncbi.nlm.nih.gov/27312411/)
+{%- if multiqc %}- [MultiQC](https://pubmed.ncbi.nlm.nih.gov/27312411/)
 
 > Ewels P, Magnusson M, Lundin S, Käller M. MultiQC: summarize analysis results for multiple tools and samples in a single report. Bioinformatics. 2016 Oct 1;32(19):3047-8. doi: 10.1093/bioinformatics/btw354. Epub 2016 Jun 16. PubMed PMID: 27312411; PubMed Central PMCID: PMC5039924.
 

nf_core/pipeline-template/README.md

@@ -7,7 +7,7 @@
   </picture>
 </h1>
 
-{% else %}
+{%- else -%}
 
 # {{ name }}
 
@@ -48,13 +48,13 @@
      workflows use the "tube map" design for that. See https://nf-co.re/docs/contributing/design_guidelines#examples for examples.   -->
 <!-- TODO nf-core: Fill in short bullet-pointed list of the default steps in the pipeline -->
 
-{% if fastqc %}1. Read QC ([`FastQC`](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/)){% endif %}
-{% if multiqc %}2. Present QC for raw reads ([`MultiQC`](http://multiqc.info/)){% endif %}
+{%- if fastqc %}1. Read QC ([`FastQC`](https://www.bioinformatics.babraham.ac.uk/projects/fastqc/)){% endif %}
+{%- if multiqc %}2. Present QC for raw reads ([`MultiQC`](http://multiqc.info/)){% endif %}
 
 ## Usage
 
 > [!NOTE]
-> If you are new to Nextflow and nf-core, please refer to [this page](https://nf-co.re/docs/usage/installation) on how to set-up Nextflow. {% if test_config %}Make sure to [test your setup](https://nf-co.re/docs/usage/introduction#how-to-run-a-pipeline) with `-profile test` before running the workflow on actual data.{% endif %}
+> If you are new to Nextflow and nf-core, please refer to [this page](https://nf-co.re/docs/usage/installation) on how to set-up Nextflow. {%- if test_config %}Make sure to [test your setup](https://nf-co.re/docs/usage/introduction#how-to-run-a-pipeline) with `-profile test` before running the workflow on actual data.{% endif %}
 
 <!-- TODO nf-core: Describe the minimum required steps to execute the pipeline, e.g. how to prepare samplesheets.
      Explain what rows and columns represent. For instance (please edit as appropriate):
@@ -120,10 +120,11 @@ For further information or help, don't hesitate to get in touch on the [Slack `#
 <!-- TODO nf-core: Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file. -->
 <!-- If you use {{ name }} for your analysis, please cite it using the following doi: [10.5281/zenodo.XXXXXX](https://doi.org/10.5281/zenodo.XXXXXX) -->
 
-{% if citations %}<!-- TODO nf-core: Add bibliography of tools and data used in your pipeline -->
+{%- if citations %}<!-- TODO nf-core: Add bibliography of tools and data used in your pipeline -->
 
 An extensive list of references for the tools used by the pipeline can be found in the [`CITATIONS.md`](CITATIONS.md) file.
-{% endif %}
+{%- endif %}
+
 {% if is_nfcore -%}
 You can cite the `nf-core` publication as follows:
 

nf_core/pipeline-template/docs/output.md

@@ -2,7 +2,7 @@
 
 ## Introduction
 
-This document describes the output produced by the pipeline. {% if multiqc %}Most of the plots are taken from the MultiQC report, which summarises results at the end of the pipeline.{% endif %}
+This document describes the output produced by the pipeline.{% if multiqc %} Most of the plots are taken from the MultiQC report, which summarises results at the end of the pipeline.{% endif %}
 
 The directories listed below will be created in the results directory after the pipeline has finished. All paths are relative to the top-level results directory.
 
@@ -14,9 +14,8 @@ The pipeline is built using [Nextflow](https://www.nextflow.io/) and processes d
 
 {% if fastqc -%}
 
-- [FastQC](#fastqc) - Raw read QC
-  {%- endif %}
-  {%- if multiqc %}
+- [FastQC](#fastqc) - Raw read QC{% endif %}
+  {%- if multiqc -%}
 - [MultiQC](#multiqc) - Aggregate report describing results and QC from the whole pipeline
   {%- endif %}
 - [Pipeline information](#pipeline-information) - Report metrics generated during the workflow execution
@@ -35,7 +34,7 @@ The pipeline is built using [Nextflow](https://www.nextflow.io/) and processes d
 </details>
 
 [FastQC](http://www.bioinformatics.babraham.ac.uk/projects/fastqc/) gives general quality metrics about your sequenced reads. It provides information about the quality score distribution across your reads, per base sequence content (%A/T/G/C), adapter contamination and overrepresented sequences. For further reading and documentation see the [FastQC help pages](http://www.bioinformatics.babraham.ac.uk/projects/fastqc/Help/).
-{%- endif %}
+{%- endif -%}
 
 {% if multiqc -%}
 
@@ -54,7 +53,7 @@ The pipeline is built using [Nextflow](https://www.nextflow.io/) and processes d
 [MultiQC](http://multiqc.info) is a visualization tool that generates a single HTML report summarising all samples in your project. Most of the pipeline QC results are visualised in the report and further statistics are available in the report data directory.
 
 Results generated by MultiQC collate pipeline QC from supported tools e.g. FastQC. The pipeline has special steps which also allow the software versions to be reported in the MultiQC output for future traceability. For more information about how to use MultiQC reports, see <http://multiqc.info>.
-{%- endif %}
+{%- endif -%}
 
 ### Pipeline information
 

nf_core/pipeline-template/docs/usage.md

@@ -115,7 +115,7 @@ It is a good idea to specify the pipeline version when running the pipeline on y
 
 First, go to the [{{ name }} releases page](https://github.com/{{ name }}/releases) and find the latest pipeline version - numeric only (eg. `1.3.1`). Then specify this when running the pipeline with `-r` (one hyphen) - eg. `-r 1.3.1`. Of course, you can switch to another version by changing the number after the `-r` flag.
 
-This version number will be logged in reports when you run the pipeline, so that you'll know what you used when you look back in the future. {% if multiqc %}For example, at the bottom of the MultiQC reports.{% endif %}
+This version number will be logged in reports when you run the pipeline, so that you'll know what you used when you look back in the future.{% if multiqc %} For example, at the bottom of the MultiQC reports.{% endif %}
 
 To further assist in reproducibility, you can use share and reuse [parameter files](#running-the-pipeline) to repeat pipeline runs with the same settings without having to write out a command with every single parameter.
 

nf_core/pipeline-template/nextflow.config

@@ -306,7 +306,7 @@ validation {
         command = "nextflow run {{ name }} -profile <docker/singularity/.../institute> --input samplesheet.csv --outdir <OUTDIR>"
         fullParameter = "help_full"
         showHiddenParameter = "show_hidden"
-        {% if is_nfcore -%}
+        {%- if is_nfcore %}
         beforeText = """
 -\033[2m----------------------------------------------------\033[0m-
                                         \033[0;32m,--.\033[0;30m/\033[0;32m,-.\033[0m

nf_core/pipeline-template/nextflow_schema.json

@@ -243,10 +243,12 @@
         {
             "$ref": "#/$defs/input_output_options"
         },
-        {% if igenomes %}{
+        {%- if igenomes %}
+        {
             "$ref": "#/$defs/reference_genome_options"
         },{% endif %}
-        {% if nf_core_configs %}{
+        {%- if nf_core_configs %}
+        {
             "$ref": "#/$defs/institutional_config_options"
         },{% endif %}
         {

nf_core/pipeline-template/subworkflows/local/utils_nfcore_pipeline_pipeline/main.nf

@@ -143,7 +143,7 @@ workflow PIPELINE_COMPLETION {
     {%- if multiqc %}
     def multiqc_reports = multiqc_report.toList()
     {%- endif %}
-    
+
     //
     // Completion email and summary
     //

nf_core/pipelines/create/create.py

@@ -22,7 +22,7 @@
 from nf_core.pipelines.create_logo import create_logo
 from nf_core.pipelines.lint_utils import run_prettier_on_file
 from nf_core.pipelines.rocrate import ROCrate
-from nf_core.utils import NFCoreTemplateConfig, NFCoreYamlLintConfig
+from nf_core.utils import NFCoreTemplateConfig, NFCoreYamlLintConfig, custom_yaml_dumper
 
 log = logging.getLogger(__name__)
 
@@ -266,6 +266,14 @@ def init_pipeline(self):
         # Init the git repository and make the first commit
         if not self.no_git:
             self.git_init_pipeline()
+            # Run prettier on files
+            if self.config.skip_features is None or not (
+                "code_linters" in self.config.skip_features or "github" in self.config.skip_features
+            ):
+                current_dir = Path.cwd()
+                os.chdir(self.outdir)
+                run_prettier_on_file([str(f) for f in self.outdir.glob("**/*")])
+                os.chdir(current_dir)
 
         if self.config.is_nfcore and not self.is_interactive:
             log.info(
@@ -362,7 +370,7 @@ def render_template(self) -> None:
             # Make a logo and save it, if it is a nf-core pipeline
             self.make_pipeline_logo()
 
-        if self.config.skip_features is None or "ro-crate" not in self.config.skip_features:
+        if self.config.skip_features is None or "rocrate" not in self.config.skip_features:
             # Create the RO-Crate metadata file
             rocrate_obj = ROCrate(self.outdir)
             rocrate_obj.create_rocrate(json_path=self.outdir / "ro-crate-metadata.json")
@@ -375,12 +383,9 @@ def render_template(self) -> None:
             if config_fn is not None and config_yml is not None:
                 with open(str(config_fn), "w") as fh:
                     config_yml.template = NFCoreTemplateConfig(**self.config.model_dump(exclude_none=True))
-                    yaml.safe_dump(config_yml.model_dump(exclude_none=True), fh)
+                    yaml.dump(config_yml.model_dump(exclude_none=True), fh, Dumper=custom_yaml_dumper())
                     log.debug(f"Dumping pipeline template yml to pipeline config file '{config_fn.name}'")
 
-        # Run prettier on files
-        run_prettier_on_file([str(f) for f in self.outdir.glob("**/*")])
-
     def fix_linting(self):
         """
         Updates the .nf-core.yml with linting configurations
@@ -408,7 +413,13 @@ def fix_linting(self):
         if config_fn is not None and nf_core_yml is not None:
             nf_core_yml.lint = NFCoreYamlLintConfig(**lint_config)
             with open(self.outdir / config_fn, "w") as fh:
-                yaml.dump(nf_core_yml.model_dump(exclude_none=True), fh, default_flow_style=False, sort_keys=False)
+                yaml.dump(
+                    nf_core_yml.model_dump(exclude_none=True),
+                    fh,
+                    sort_keys=False,
+                    default_flow_style=False,
+                    Dumper=custom_yaml_dumper(),
+                )
 
     def make_pipeline_logo(self):
         """Fetch a logo for the new pipeline from the nf-core website"""

nf_core/pipelines/create/template_features.yml

@@ -148,6 +148,10 @@ is_nfcore:
       - "docs/images/nf-core-{{short_name}}_logo_light.png"
       - "docs/images/nf-core-{{short_name}}_logo_dark.png"
       - ".github/ISSUE_TEMPLATE/bug_report.yml"
+      - ".github/CONTRIBUTING.md"
+      - ".github/PULL_REQUEST_TEMPLATE.md"
+      - "assets/email_template.txt"
+      - "docs/README.md"
     nextflow_config:
       - "manifest.name"
       - "manifest.homePage"
@@ -445,6 +449,8 @@ rocrate:
   linting:
     files_warn:
       - "ro-crate-metadata.json"
+    files_unchanged:
+      - ".prettierignore"
 vscode:
   skippable_paths:
     - ".vscode"

nf_core/pipelines/lint_utils.py

@@ -97,7 +97,7 @@ def run_prettier_on_file(file: Union[Path, str, List[str]]) -> None:
             all_lines = [line for line in e.stdout.decode().split("\n")]
             files = "\n".join(all_lines[3:])
             log.debug(f"The following files were modified by prettier:\n {files}")
-        elif e.stderr.decode():
+        else:
             log.warning(
                 "There was an error running the prettier pre-commit hook.\n"
                 f"STDOUT: {e.stdout.decode()}\nSTDERR: {e.stderr.decode()}"

tests/utils.py

@@ -14,7 +14,7 @@
 import nf_core.modules
 import nf_core.pipelines.create.create
 from nf_core import __version__
-from nf_core.utils import NFCoreTemplateConfig, NFCoreYamlConfig
+from nf_core.utils import NFCoreTemplateConfig, NFCoreYamlConfig, custom_yaml_dumper
 
 TEST_DATA_DIR = Path(__file__).parent / "data"
 OLD_TRIMGALORE_SHA = "9b7a3bdefeaad5d42324aa7dd50f87bea1b04386"
@@ -136,7 +136,7 @@ def create_tmp_pipeline(no_git: bool = False) -> Tuple[Path, Path, str, Path]:
         bump_version=None,
     )
     with open(str(Path(pipeline_dir, ".nf-core.yml")), "w") as fh:
-        yaml.dump(nf_core_yml.model_dump(), fh)
+        yaml.dump(nf_core_yml.model_dump(), fh, Dumper=custom_yaml_dumper())
 
     nf_core.pipelines.create.create.PipelineCreate(
         pipeline_name, "it is mine", "me", no_git=no_git, outdir=pipeline_dir, force=True