v3.1.0 - Brass Boxfish

⭐ Highlights ⭐

• We added the new contributors field to the pipeline template manifest.
• The nf-core pipelines download command supports ORAS container URIs.
• New command nf-core subworkflows patch.

Template

• Keep pipeline name in version.yml file (#3223)
• Fix Manifest DOI text (#3224)
• Do not assume pipeline name is url (#3225)
• fix workflow_dispatch trigger and parse more review comments in awsfulltest (#3235)
• Add resource limits to Gitpod profile(#3255)
• Fix a typo (#3268)
• Remove def from nextflow.config and add trace_report_suffix param (#3296)
• Move includeConfig 'conf/modules.config' next to includeConfig 'conf/base.config' to not overwrite tests profiles configurations (#3301)
• Use params.monochrome_logs in the template and update nf-core components (#3310)
• Fix some typos and improve writing in usage.md and CONTRIBUTING.md (#3302)
• Add manifest.contributors to nextflow.config (#3311)
• Update template components (#3328)
• Template: Remove mention of GRCh37 if igenomes is skipped (#3330)
• Be more verbose in approval check action (#3338)

Download

• First steps towards fixing #3179: Modify prioritize_direct_download() to retain Seqera Singularity https:// Container URIs and hardcode Seqera Containers into gather_registries() (#3244).
• Further steps towards fixing #3179: Enable limited support for oras:// container paths (only absolute URIs, no flexible registries like with Docker) and prevent unnecessary image downloads for Seqera Container modules with reconcile_seqera_container_uris() (#3293).
• Update dawidd6/action-download-artifact action to v7 (#3306)

Linting

• allow mixed str and dict entries in lint config (#3228)
• fix meta_yml linting test failing due to module.process_name always being "" (#3317)
• fix module section regex matching wrong things (#3321)

Modules

• add a panel around diff previews when updating (#3246)

Subworkflows

• Add nf-core subworkflows patch command (#2861)
• Improve subworkflow nf-test migration warning (#3298)

General

• Include .nf-core.yml in nf-core pipelines bump-version (#3220)
• create: add shortcut to toggle all switches (#3226)
• Remove unrelated values when saving .nf-core file (#3227)
• use correct --profile options for nf-core subworkflows test (#3233)
• Update GitHub Actions (#3237)
• add --dir/-d option to schema commands (#3247)
• fix headers in api docs (#3323)
• handle new schema structure in nf-core pipelines create-params-file (#3276)
• Update Gitpod image to use Miniforge instead of Miniconda(#3274)
• Add hint to solve git errors with a synced repo (#3279)
• Run pre-commit when testing linting the template pipeline (#3280)
• Make CLI prompt less nf-core specific (#3326)
• Update gitpod vscode extensions to use nf-core extension pack (#3327)
• Remove toList() channel operation from inside onComplete block (#3304)
• build: Setup VS Code tests (#3292)
• Don't break gitpod.yml with template string (#3332)
• rocrate: remove duplicated entries for name and version (#3333)
• rocrate: Update crate with version bump and handle new contributor field (#3334)
• set default_branch to master for now (#3335)
• Set git defaultBranch to master in sync action (#3337)
• Add verbose mode to sync action (#3339)

Version updates

• chore(deps): update pre-commit hook pre-commit/mirrors-mypy to v1.12.0 (#3230)
• Update codecov/codecov-action action to v5 (#3283)
• Update gitpod/workspace-base Docker digest to 12853f7 (#3309)
• Update pre-commit hook astral-sh/ruff-pre-commit to v0.8.2 (#3325)

v3.0.2 - Titanium Tapir Patch

This patch release fixes some bugs found in the pipeline template.

Template

• Add null/ to .gitignore (#3191)
• Parallelize pipeline GHA tests over docker/conda/singularity (#3214)
• Fix template_version_comment.yml github action (#3212)
• Fix pre-commit linting on pipeline template (#3218)

Linting

• Fix bug when linting schema params and when using defaultIgnoreParams (#3213)

General

• Use updated pipeline commands in docstrings (#3215)
• Disable automatic sync on release, fix handling empty pipeline input (#3217)

v3.0.1 - Titanium Tapir Patch

Template

• Fixed an issue where the linting CI action didn't read the correct file (#3202)
• Fixed condition for awsfulltest to run (#3203)
• Fix too many empty lines added by jinja (#3204 and #3206)
• Fix header blocks in local subworkflow including git merge marker-like strings (#3201)
• Update included subworkflows and modules (#3208)

v3.0.0 - Titanium Tapir

Highlights

• Pipeline commands are renamed from nf-core <comand> to nf-core pipelines <command> to follow the same command structure as modules and subworkflows commands.
• More customisation for pipeline templates. The template has been divided into features which can be skipped, e.g. you can create a new pipeline without any traces of FastQC in it.
• A new Text User Interface app when running nf-core pipelines create to help us guide you through the process better (no worries, you can still use the cli if you give all values as parameters)
• We replaced nf-validation with nf-schema in the pipeline template
• CI tests now lint with the nf-core tools version matching the template version of the pipeline, to minimise errors in opened PRs with every new tools release.
• nf-core licences command is deprecated.
• The structure of nf-core/tools pytests has been updated
• The structure of the API docs has been updated

Template

• Change paths to test data (#2985)
• Run awsfulltest on PRs to master with two PR approvals (#3042)
• Remove deprecated syntax (#3046)
• Use filename in code block for params.yml (#3055)
• Remove release announcement for non nf-core pipelines (#3072)
• handle template features with a yaml file (#3108, #3112)
• add option to exclude code linters for custom pipeline template (#3084)
• add option to exclude citations for custom pipeline template (#3101 and #3169)
• add option to exclude gitpod for custom pipeline template (#3100)
• add option to exclude codespaces from pipeline template (#3105)
• add option to exclude multiqc from pipeline template (#3103)
• add option to exclude changelog from custom pipeline template (#3104)
• add option to exclude license from pipeline template (#3125)
• add option to exclude email from pipeline template (#3126)
• add option to exclude nf-schema from the template (#3116)
• add option to exclude fastqc from pipeline template (#3129)
• add option to exclude documentation from pipeline template (#3130)
• add option to exclude test configs from pipeline template (#3133)
• add option to exclude tower.yml from pipeline template (#3134)
• Use nf-schema instead of nf-validation (#3116)
• test pipeline with conda and singularity on PRs to master (#3149)
• run nf-core lint --release on PRs to master (#3148)
• Add tests to ensure all files are part of a template customisation group and all groups are tested (#3099)
• Update the syntax of utils_nfcore_pipeline_pipeline local subworkflow (#3166)
• Remove if/else block to include igenomes.config (#3168)
• Fixed release announcement hashtags for Mastodon (#3099)
• Remove try/catch blocks from nextflow.config (#3167)
• Extend download_pipeline.yml to count pre-downloaded container images. (#3182)

Linting

• Fix linting fail on nfcore_external_java_deps if nf_schema is used (#2976)
• Conda module linting: Include package name in log file (#3014)
• Remove defaults from conda environment.yml file. (#3029)
• Restructure pipeline tests and move pipeline linting into subfolder (#3070)
• Fix module linting warning for process_high_memory (#3086)
• Linting will now fail when an unpinned plugin is used (#3116)
• Linting will now check if the schema is correct for the used validation plugin ([#3116])(#3116)
• Linting will now check the use of the right validation plugin include statements in the workflow scripts ([#3116])(#3116)
• Full linting for correct use of nf-schema and nf-validation (#3116)
• Handle cases where the directory path contains the name of the component (#3147)
• Don't test conda environment.yml name attribute (which should no longer be there) (#3161)

Pipeline create command

• Allow more special characters on the pipeline name for non-nf-core pipelines (#3008)
• Mock git cretentials to generate stable textual snapshots (#3007)
• Display input textbox with equally spaced grid (#3038)
• Allow numbers in custom pipeline name (#3094)

Components

• The modules_nfcore tag in the main.nf.test file of modules/subworkflows now displays the organization name in custom modules repositories (#3005)
• Add --migrate_pytest option to nf-core <modules|subworkflows> test command (#3085)
• Allow spaces at the beginning of include statements (#3115)
• Add option --fix to update the meta.yml file of subworkflows (#3077)

Download

• Fully removed already deprecated -t / --tower flag.
• Refactored the CLI for consistency (short flag is usually second word, e.g. also for --container-library etc.):

Old parameter	New parameter
-d / --download-configuration	-c / --download-configuration
-p / --parallel-downloads	-d / --parallel-downloads
new parameter	-p / (--platform)

General

• Update output of generation script for API docs to new structure (#2988)
• Remove rich-codex.yml action, images are now generated on the website repo (#2989)
• Add no clobber and put bash options on their own line (#2991)
• move pipeline subcommands for v3.0 (#2983)
• return directory if base_dir is the root directory (#3003)
• Remove nf-core licences command (#3012)
• README - absolute image paths (#3013)
• Add warning deprecation message to top-level commands (#3036)
• move pipeline commands to functions to avoid duplication (#3039)
• update output_dir for api docs to new website structure (#3051)
• Add --limit-output argument for modules/subworkflow update (#3047)
• update api docs to new structure (#3054)
• handle new jsonschema error type (#3061)
• Fix number of arguments for pipelines_create within the command_create function (#3074)
• Add bot action to update textual snapshots and write bot documentation (#3102)
• Update gitpod setup (#3136)
• fix syncing a pipeline from current directory (#3143)
• Patch gitpod conda setup to not use defaults channel (#3159)

Version updates

• Update pre-commit hook astral-sh/ruff-pre-commit to v0.6.0 (#3122)
• Update gitpod/workspace-base Docker digest to 92dd1bc (#2982)
• Update python:3.12-slim Docker digest to 59c7332 (#3124)
• Update pre-commit hook pre-commit/mirrors-mypy to v1.11.1 (#3091)
• Update to pytest v8 and move it to dev dependencies (#3058)
• Update minimal textual version and snapshots (#2998)

v2.14.1 - Tantalum Toad - Patch

This is a patch release to fix a broken CI action.

Template

• Don't cache pip in linting.yml (#2961)

General

• Fix update github action for components in pipeline template (#2968)
• Run sync after release on self hosted runners (#2970)

v2.14.0 - Tantalum Toad

This release contains some template changes and nf-core/tools updates.

🌟 Highlights 🌟

nf-core/tools functionalities

• We included a new linting test to ensure that nf-test snapshots contain the versions.yml file.
• Components (modules and subworkflows) can now handle more possible git URLs (ssh:// and ftp://).
• The nf-core download command has a new argument --tag, which allows adding additional tags to select particular revisions in the Seqera Platform interface. For example, --tag "3.10=validated"{:bash} would allow you to quickly select the validated version of the pipeline.

Pipeline template

• We updated the GitHub Action which tests that the pipeline can be downloaded correctly (download_pipeline.yml)
• We removed pyproject.toml from the template.
• Pipeline-specific institutional configs support is now activated for all pipelines by default.
• The .nf-core.yml file contains now the version of the pipeline template.

Changelog

Template

• Remove fasta default from nextflow.config (#2828)
• Update templates to use nf-core/setup-nextflow v2 (#2818)
• Link to troubleshooting docs when pipeline fails (#2845)
• Add fallback to download_pipeline.yml in case the pipeline does not support stub runs (#2846)
• Set topic variable correctly in the mastodon announcement (#2848)
• Add a cleanup action to download_pipeline.yml to fix failures caused by inadequate storage space on the runner (#2849)
• Update python to 3.12 (#2805)
• Remove pyproject.toml from template root
• Shorten lines in pipeline template (#2908)
• Add a new hidden --pipelines_testdata_base_path parameter to more easily switch locations of test data in test configs (#2931)[https://github.com//pull/2931]
• Permanently activated pipeline-specific institutional configs support for all pipelines without need for manual intervention (#2936)
• Template config: conda.channels, not channels (#2950)
• Handles multiple DOIs + doi.org resolver from manifest.doi (#2946)
• Update included components (#2949)
• Update .editorconfig (#2953)

Linting

• Only match assignments of params in main.nf and not references like params.aligner == <something> (#2833)
• Include test for presence of versions in snapshot (#2888)
• Components: set correct sha before running component lint tests (#2952)
• Less strict logo comparison (#2956)
• Handle request errors more gracefully for actions validation (#2959)

Download

• Replace --tower with --platform. The former will remain for backwards compatability for now but will be removed in a future release. (#2853)
• Better error message when GITHUB_TOKEN exists but is wrong/outdated
• New --tag argument to add custom tags during a pipeline download (#2938)

Components

• Handle more complete list of possible git URL forms (ssh:// and ftp:// prefixes specifically) (#2945)
• Fix path in component update script (#2823)

General

• Update CI to use nf-core/setup-nextflow v2 (#2819)
• Changelog bot: handle also patch version before dev suffix (#2820)
• Add force_pr flag to sync, to force a PR even though there are no changes committed (#2822)
• Update prettier to 3.2.5 (#2830)
• Update GitHub Actions (#2827), (#2902), (#2927), (#2939)
• Switch to setup-nf-test (#2834)
• Add tests for assignment and referencing of params in main.nf (#2841)
• Optimize layers in dockerfile (#2842)
• Update python:3.11-slim Docker digest to a2eb07f (#2847)
• Strip out mention of "Nextflow Tower" and replace with "Seqera Platform" wherever possible
• Fix issue with config resolution that was causing nested configs to behave unexpectedly (#2862)
• Fix schema docs console output truncating (#2880)
• Ensure path object converted to string before stripping quotes (#2878)
• Fix incorrect assertions for called_with on mocks (#2891)
• Make cli-provided module/subworkflow names case insensitive (#2869)
• Get immediate parent path name for schema creation (#2886)
• Remove old references to CUSTOMDUMPSOFTWAREVERSIONS and add linting checks (#2897)
• Update pre-commit hook pre-commit/mirrors-mypy to v1.10.0 (#2933)
• Update codecov/codecov-action digest to 5ecb98a (#2948)
• Update gitpod/workspace-base Docker digest to 124f2b8 (#2943)
• fix(collectfile): sort true for methods_description_mqc.yaml (#2947)
• chore(deps): update pre-commit hook astral-sh/ruff-pre-commit to v0.4.3 (#2951)
• Restructure CHANGELOG.md (#2954)

v2.13.1 - Tin Puppy Patc

v2.13.1 - Tin Puppy Patch - [2024-02-29]

Patch release for failing AWS actions.

Template

• Remove obsolete editor settings in devcontainer.json and gitpod.yml (#2795)
• Add nf-test test instructions to contributing and PR template (#2807)
• Fix topic extraction step for hashtags in toots (#2810)
• Update modules and subworkflows in the template (#2811)
• Unpin setup-nextflow and action-tower-launch (#2806)

Download

• Improved offline container image resolution by introducing symlinks, fixes issues #2751, #2644 and demultiplex#164: (#2768)

General

• chore(deps): update codecov/codecov-action digest to 0cfda1d (#2794)
• chore(deps): update gitpod/workspace-base docker digest to c15ee2f (#2799)

v2.13 - Tin Puppy

v2.13 - Tin Puppy - [2024-02-20]

Highlights 🌟

This new version of nf-core/tools includes a refactoring of the pipelines template.

• The lib directory is removed: 👋 Groovy code
• We now use nf-core subworkflows for pipeline initialisation (replacing the code from the lib directory)
• The nf-validation plugin is now used to create an input channel from a sample sheet file.

Check the nf-core blog post for more information!

Template

• Add empty line in README.md to fix badges. (#2729)
• Replace automatic branch detection in nf-core download CI test with hardcoded dev and input. (#2727)
• Add Github Action to automatically cleanup ubuntu-latest runners to fix runner running out of diskspace errors(#2755)
• Fix GitHub Actions CI and Linting badges links (#2757)
• Add hashtags to release announcement on mastodon (#2761)
• update fastqc and multiqc in template (#2776)
• template refactoring: remove the lib directory and use nf-core subworkflows (#2736)
• use nf-validation to create an input channel from a sample sheet (#2736)

Linting

• Make creat-lint-wf composable (#2733)
• Add looser comparison when pipeline logos (#2744)
• Handle multiple aliases in module imports correctly during linting (#2762)
• Switch to markdown based API and error docs (#2758)

Modules

• Handle dirty local module repos by force checkout of commits and branches if needed (#2734)
• Patch: handle file not found when it is an added file to a module (#2771)
• Handle symlinks when migrating pytest (#2770)
• Add --profile parameter to nf-test command (#2767)

General

• fix ignoring changes in partially templated files (e.g. .gitignore) (#2722)
• update ruff to 0.2.0 and add it to pre-commit step (#2725)
• Update codecov/codecov-action digest to e0b68c6 (#2728)
• Update pre-commit hook astral-sh/ruff-pre-commit to v0.2.1 (#2730)
• Update python:3.11-slim Docker digest to 2a746e2 (#2743)
• Update actions/setup-python action to v5 (#2739)
• Update gitpod/workspace-base Docker digest to 45e7617 (#2747)
• chore(deps): pin jlumbroso/free-disk-space action to 54081f1 (#2756)
• chore(deps): update actions/github-script action to v7 (#2766)
• chore(deps): update pre-commit hook astral-sh/ruff-pre-commit to v0.2.2 (#2769)
• Update gitpod/workspace-base Docker digest to 728e1fa (#2780)

v2.12.1 - Aluminium Wolf - Patch

v2.12.1 - Aluminium Wolf - Patch - [2024-02-01]

Linting

• Handle default values of type number from nextflow schema (#2703)
• fix ignoring files_unchanged (#2707)

General

• Update pre-commit hook astral-sh/ruff-pre-commit to v0.1.15 (#2705)
• use types for default value comparison (#2712)
• fix changelog titles (#2708)
• Print relative path not absolute path in logo cmd log output (#2709)
• Update codecov/codecov-action action to v4 (#2713)
• Ignore nf-core-bot in renovate PRs (#2716)

2.12 - Aluminium Wolf

v2.12 - Aluminium Wolf - [2024-01-29]

Template

• Add a Github Action Workflow to the pipeline template that tests a successful download with 'nf-core download' (#2618)
• Use pre-commit to lint files in GitHub CI (#2635)
• Use pdiff also on gitpod for nf-test (#2640)
• switch to new image syntax in readme (#2645)
• Add conda channel order to nextflow.config (#2094)
• Fix tyop in pipeline nextflow.config (#2664)
• Remove nfcore_external_java_deps.jar from lib directory in pipeline template (#2675)
• Add function to check -profile is well formatted (#2678)
• Add new pipeline error message pointing to docs when 'requirement exceeds available memory' error message (#2680)
• add 👀👍🏻🎉😕 reactions to fix-linting-bot action (#2692)

Linting

• Fix linting of a pipeline with patched custom module (#2669)
• linting a pipeline also lints the installed subworkflows (#2677)
• environment.yml name must be lowercase (#2676)
• allow ignoring specific files when template_strings (#2686)
• lint nextflow.config default values match the ones specified in nextflow_schema.json (#2684)

Modules

• Fix empty json output for nf-core list local (#2668)

General

• Run CI-pytests for nf-core tools on self-hosted runners (#2550)
• Add Ruff linter and formatter replacing Black, isort and pyupgrade (#2620)
• Set pdiff as nf-test differ in Docker image for Gitpod (#2642)
• Fix Renovate Dockerfile updating issues (#2648 and #2651)
• Add new subcommand nf-core tui, which launches a TUI (terminal user interface) to intuitively explore the command line flags, built using Trogon (#2655)
• Add new subcommand: nf-core logo-create to output an nf-core logo for a pipeline (instead of going through the website) (#2662)
• Handle api redirects from the old site (#2672)
• Remove redundanct v in pipeline version for emails (#2667)
• add function to check -profile is well formatted (#2678)
• Update pre-commit hook astral-sh/ruff-pre-commit to v0.1.14 (#2674)
• Update pre-commit hook pre-commit/mirrors-mypy to v1.8.0 (#2630)
• Update mshick/add-pr-comment action to v2 (#2632)
• update python image version in docker file (#2636)
• Update actions/cache action to v4 (#2666)
• Update peter-evans/create-or-update-comment action to v4 (#2683) and (#2695)