Skip to content

Commit

Permalink
Merge branch 'dev' into fix-linting-error
Browse files Browse the repository at this point in the history
  • Loading branch information
lmReef authored Dec 5, 2024
2 parents c3fd9bb + 978f865 commit c5dc6ac
Show file tree
Hide file tree
Showing 26 changed files with 76 additions and 59 deletions.
1 change: 0 additions & 1 deletion .github/actions/create-lint-wf/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ runs:
cd create-lint-wf
export NXF_WORK=$(pwd)
# Set up Nextflow
- name: Install Nextflow
uses: nf-core/setup-nextflow@v2
with:
Expand Down
4 changes: 1 addition & 3 deletions .github/workflows/create-lint-wf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,12 @@ concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

env:
NXF_ANSI_LOG: false

jobs:
MakeTestWorkflow:
runs-on: ${{ github.event.inputs.runners || github.run_number > 1 && 'ubuntu-latest' || 'self-hosted' }}
env:
NXF_ANSI_LOG: false

strategy:
matrix:
NXF_VER:
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/create-test-lint-wf-template.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ jobs:
needs: prepare-matrix
env:
NXF_ANSI_LOG: false

strategy:
matrix:
TEMPLATE: ${{ fromJson(needs.prepare-matrix.outputs.all_features) }}
Expand Down
4 changes: 1 addition & 3 deletions .github/workflows/create-test-wf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,15 +27,13 @@ concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

env:
NXF_ANSI_LOG: false

jobs:
RunTestWorkflow:
# use the runner given by the input if it is dispatched manually, run on github if it is a rerun or on self-hosted by default
runs-on: ${{ github.event.inputs.runners || github.run_number > 1 && 'ubuntu-latest' || 'self-hosted' }}
env:
NXF_ANSI_LOG: false

strategy:
matrix:
NXF_VER:
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.8.1
rev: v0.8.2
hooks:
- id: ruff # linter
args: [--fix, --exit-non-zero-on-fix] # sort imports and fix
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
### Subworkflows

- Add `nf-core subworkflows patch` command ([#2861](https://github.com/nf-core/tools/pull/2861))
- Improve subworkflow nf-test migration warning ([#3298](https://github.com/nf-core/tools/pull/3298))

### General

Expand All @@ -57,6 +58,7 @@
- Update pre-commit hook astral-sh/ruff-pre-commit to v0.8.0 ([#3299](https://github.com/nf-core/tools/pull/3299))
- Update gitpod/workspace-base Docker digest to 12853f7 ([#3309](https://github.com/nf-core/tools/pull/3309))
- Run pre-commit when testing linting the template pipeline ([#3280](https://github.com/nf-core/tools/pull/3280))
- Update pre-commit hook astral-sh/ruff-pre-commit to v0.8.2 ([#3325](https://github.com/nf-core/tools/pull/3325))

## [v3.0.2 - Titanium Tapir Patch](https://github.com/nf-core/tools/releases/tag/3.0.2) - [2024-10-11]

Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM python:3.12-slim@sha256:2a6386ad2db20e7f55073f69a98d6da2cf9f168e05e7487d2670baeb9b7601c5
FROM python:3.12-slim@sha256:2b0079146a74e23bf4ae8f6a28e1b484c6292f6fb904cbb51825b4a19812fcd8
LABEL authors="phil.ewels@seqera.io,erik.danielsson@scilifelab.se" \
description="Docker image containing requirements for nf-core/tools"

Expand Down
4 changes: 2 additions & 2 deletions nf_core/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -288,7 +288,7 @@ def command_pipelines_create(ctx, name, description, author, version, force, out
@click.option(
"--release",
is_flag=True,
default=os.path.basename(os.path.dirname(os.environ.get("GITHUB_REF", "").strip(" '\""))) == "master"
default=Path(os.environ.get("GITHUB_REF", "").strip(" '\"")).parent.name in ["master", "main"]
and os.environ.get("GITHUB_REPOSITORY", "").startswith("nf-core/")
and not os.environ.get("GITHUB_REPOSITORY", "") == "nf-core/tools",
help="Execute additional checks for release-ready workflows.",
Expand Down Expand Up @@ -2240,7 +2240,7 @@ def command_download(
@click.option(
"--release",
is_flag=True,
default=os.path.basename(os.path.dirname(os.environ.get("GITHUB_REF", "").strip(" '\""))) == "master"
default=Path(os.environ.get("GITHUB_REF", "").strip(" '\"")).parent.name in ["master", "main"]
and os.environ.get("GITHUB_REPOSITORY", "").startswith("nf-core/")
and not os.environ.get("GITHUB_REPOSITORY", "") == "nf-core/tools",
help="Execute additional checks for release-ready workflows.",
Expand Down
4 changes: 2 additions & 2 deletions nf_core/pipeline-template/.github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,9 @@ These tests are run both with the latest available version of `Nextflow` and als

:warning: Only in the unlikely and regretful event of a release happening with a bug.

- On your own fork, make a new branch `patch` based on `upstream/master`.
- On your own fork, make a new branch `patch` based on `upstream/main` or `upstream/master`.
- Fix the bug, and bump version (X.Y.Z+1).
- Open a pull-request from `patch` to `master` with the changes.
- Open a pull-request from `patch` to `main`/`master` with the changes.

{% if is_nfcore -%}

Expand Down
4 changes: 2 additions & 2 deletions nf_core/pipeline-template/.github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ These are the most common things requested on pull requests (PRs).
Remember that PRs should be made against the dev branch, unless you're preparing a pipeline release.
Learn more about contributing: [CONTRIBUTING.md](https://github.com/{{ name }}/tree/master/.github/CONTRIBUTING.md)
Learn more about contributing: [CONTRIBUTING.md](https://github.com/{{ name }}/tree/{{ default_branch }}/.github/CONTRIBUTING.md)
-->

## PR checklist

- [ ] This comment contains a description of changes (with reason).
- [ ] If you've fixed a bug or added code that should be tested, add tests!
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/{{ name }}/tree/master/.github/CONTRIBUTING.md)
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/{{ name }}/tree/{{ default_branch }}/.github/CONTRIBUTING.md)
{%- if is_nfcore %}
- [ ] If necessary, also make a PR on the {{ name }} _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
{%- endif %}
Expand Down
3 changes: 2 additions & 1 deletion nf_core/pipeline-template/.github/workflows/awsfulltest.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
name: nf-core AWS full size tests
# This workflow is triggered on PRs opened against the master branch.
# This workflow is triggered on PRs opened against the main/master branch.
# It can be additionally triggered manually with GitHub actions workflow dispatch button.
# It runs the -profile 'test_full' on AWS batch

on:
pull_request:
branches:
- main
- master
workflow_dispatch:
pull_request_review:
Expand Down
18 changes: 10 additions & 8 deletions nf_core/pipeline-template/.github/workflows/branch.yml
Original file line number Diff line number Diff line change
@@ -1,15 +1,17 @@
name: nf-core branch protection
# This workflow is triggered on PRs to master branch on the repository
# It fails when someone tries to make a PR against the nf-core `master` branch instead of `dev`
# This workflow is triggered on PRs to `main`/`master` branch on the repository
# It fails when someone tries to make a PR against the nf-core `main`/`master` branch instead of `dev`
on:
pull_request_target:
branches: [master]
branches:
- main
- master

jobs:
test:
runs-on: ubuntu-latest
steps:
# PRs to the nf-core repo master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches
# PRs to the nf-core repo main/master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches
- name: Check PRs
if: github.repository == '{{ name }}'
run: |
Expand All @@ -22,7 +24,7 @@ jobs:
uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2
with:
message: |
## This PR is against the `master` branch :x:
## This PR is against the `${{github.event.pull_request.base.ref}}` branch :x:
* Do not close this PR
* Click _Edit_ and change the `base` to `dev`
Expand All @@ -32,9 +34,9 @@ jobs:
Hi @${{ github.event.pull_request.user.login }},
It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `master` branch.
The `master` branch on nf-core repositories should always contain code from the latest release.
Because of this, PRs to `master` are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch.
It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) ${{github.event.pull_request.base.ref}} branch.
The ${{github.event.pull_request.base.ref}} branch on nf-core repositories should always contain code from the latest release.
Because of this, PRs to ${{github.event.pull_request.base.ref}} are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch.
You do not need to close this PR, you can change the target branch to `dev` by clicking the _"Edit"_ button at the top of this page.
Note that even after this, the test will continue to show as failing until you push a new commit.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Test successful pipeline download with 'nf-core pipelines download'

# Run the workflow when:
# - dispatched manually
# - when a PR is opened or reopened to master branch
# - when a PR is opened or reopened to main/master branch
# - the head branch of the pull request is updated, i.e. if fixes for a release are pushed last minute to dev.
on:
workflow_dispatch:
Expand All @@ -17,9 +17,11 @@ on:
- edited
- synchronize
branches:
- main
- master
pull_request_target:
branches:
- main
- master

env:
Expand Down
2 changes: 1 addition & 1 deletion nf_core/pipeline-template/assets/schema_input.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://raw.githubusercontent.com/{{ name }}/master/assets/schema_input.json",
"$id": "https://raw.githubusercontent.com/{{ name }}/{{ default_branch }}/assets/schema_input.json",
"title": "{{ name }} pipeline - params.input schema",
"description": "Schema for the file provided with params.input",
"type": "array",
Expand Down
1 change: 1 addition & 0 deletions nf_core/pipeline-template/nextflow.config
Original file line number Diff line number Diff line change
Expand Up @@ -291,6 +291,7 @@ manifest {
homePage = 'https://github.com/{{ name }}'
description = """{{ description }}"""
mainScript = 'main.nf'
defaultBranch = '{{ default_branch }}'
nextflowVersion = '!>=24.04.2'
version = '{{ version }}'
doi = ''
Expand Down
2 changes: 1 addition & 1 deletion nf_core/pipeline-template/nextflow_schema.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://raw.githubusercontent.com/{{ name }}/master/nextflow_schema.json",
"$id": "https://raw.githubusercontent.com/{{ name }}/{{ default_branch }}/nextflow_schema.json",
"title": "{{ name }} pipeline parameters",
"description": "{{ description }}",
"type": "object",
Expand Down
46 changes: 28 additions & 18 deletions nf_core/pipelines/create/create.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ def __init__(
template_config: Optional[Union[CreateConfig, str, Path]] = None,
organisation: str = "nf-core",
from_config_file: bool = False,
default_branch: Optional[str] = None,
default_branch: str = "main",
is_interactive: bool = False,
) -> None:
if isinstance(template_config, CreateConfig):
Expand Down Expand Up @@ -87,8 +87,17 @@ def __init__(
# Read features yaml file
self.template_features_yml = load_features_yaml()

# Set fields used by the class methods
self.no_git = no_git
self.default_branch = default_branch
self.is_interactive = is_interactive

if self.config.outdir is None:
self.config.outdir = str(Path.cwd())

# Get the default branch name from the Git configuration
self.get_default_branch()

self.jinja_params, self.skip_areas = self.obtain_jinja_params_dict(
self.config.skip_features or [], str(self.config.outdir)
)
Expand All @@ -107,11 +116,6 @@ def __init__(

# Set convenience variables
self.name = self.config.name

# Set fields used by the class methods
self.no_git = no_git
self.default_branch = default_branch
self.is_interactive = is_interactive
self.force = self.config.force

if self.config.outdir == ".":
Expand Down Expand Up @@ -233,6 +237,7 @@ def obtain_jinja_params_dict(
jinja_params["name_docker"] = jinja_params["name"].replace(jinja_params["org"], jinja_params["prefix_nodash"])
jinja_params["logo_light"] = f"{jinja_params['name_noslash']}_logo_light.png"
jinja_params["logo_dark"] = f"{jinja_params['name_noslash']}_logo_dark.png"
jinja_params["default_branch"] = self.default_branch
if config_yml is not None:
if (
hasattr(config_yml, "lint")
Expand All @@ -254,6 +259,7 @@ def obtain_jinja_params_dict(

def init_pipeline(self):
"""Creates the nf-core pipeline."""

# Make the new pipeline
self.render_template()

Expand Down Expand Up @@ -421,33 +427,37 @@ def make_pipeline_logo(self):
force=bool(self.force),
)

def git_init_pipeline(self) -> None:
"""Initialises the new pipeline as a Git repository and submits first commit.
Raises:
UserWarning: if Git default branch is set to 'dev' or 'TEMPLATE'.
"""
default_branch: Optional[str] = self.default_branch
def get_default_branch(self) -> None:
"""Gets the default branch name from the Git configuration."""
try:
default_branch = default_branch or str(git.config.GitConfigParser().get_value("init", "defaultBranch"))
self.default_branch = (
str(git.config.GitConfigParser().get_value("init", "defaultBranch")) or "main"
) # default to main
except configparser.Error:
log.debug("Could not read init.defaultBranch")
if default_branch in ["dev", "TEMPLATE"]:
if self.default_branch in ["dev", "TEMPLATE"]:
raise UserWarning(
f"Your Git defaultBranch '{default_branch}' is incompatible with nf-core.\n"
f"Your Git defaultBranch '{self.default_branch}' is incompatible with nf-core.\n"
"'dev' and 'TEMPLATE' can not be used as default branch name.\n"
"Set the default branch name with "
"[white on grey23] git config --global init.defaultBranch <NAME> [/]\n"
"Or set the default_branch parameter in this class.\n"
"Pipeline git repository will not be initialised."
)

def git_init_pipeline(self) -> None:
"""Initialises the new pipeline as a Git repository and submits first commit.
Raises:
UserWarning: if Git default branch is set to 'dev' or 'TEMPLATE'.
"""

log.info("Initialising local pipeline git repository")
repo = git.Repo.init(self.outdir)
repo.git.add(A=True)
repo.index.commit(f"initial template build from nf-core/tools, version {nf_core.__version__}")
if default_branch:
repo.active_branch.rename(default_branch)
if self.default_branch:
repo.active_branch.rename(self.default_branch)
try:
repo.git.branch("TEMPLATE")
repo.git.branch("dev")
Expand Down
2 changes: 1 addition & 1 deletion nf_core/pipelines/download.py
Original file line number Diff line number Diff line change
Expand Up @@ -1744,7 +1744,7 @@ def tidy_tags_and_branches(self):
for tag in tags_to_remove:
self.repo.delete_tag(tag)

# switch to a revision that should be kept, because deleting heads fails, if they are checked out (e.g. "master")
# switch to a revision that should be kept, because deleting heads fails, if they are checked out (e.g. "main")
self.checkout(self.revision[0])

# delete unwanted heads/branches from repository
Expand Down
2 changes: 1 addition & 1 deletion nf_core/pipelines/lint/actions_awsfulltest.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ def actions_awsfulltest(self) -> Dict[str, List[str]]:

# Check that the action is only turned on for published releases
try:
if wf[True]["pull_request"]["branches"] != ["master"]:
if wf[True]["pull_request"]["branches"] != ["main", "master"]:
raise AssertionError()
if wf[True]["pull_request_review"]["types"] != ["submitted"]:
raise AssertionError()
Expand Down
2 changes: 1 addition & 1 deletion nf_core/pipelines/lint/version_consistency.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ def version_consistency(self):
"""Pipeline and container version number consistency.
.. note:: This test only runs when the ``--release`` flag is set for ``nf-core pipelines lint``,
or ``$GITHUB_REF`` is equal to ``master``.
or ``$GITHUB_REF`` is equal to ``main``.
This lint fetches the pipeline version number from three possible locations:
Expand Down
8 changes: 5 additions & 3 deletions nf_core/pipelines/schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -517,11 +517,13 @@ def validate_schema_title_description(self, schema=None):
if "title" not in self.schema:
raise AssertionError("Schema missing top-level `title` attribute")
# Validate that id, title and description match the pipeline manifest
id_attr = "https://raw.githubusercontent.com/{}/master/nextflow_schema.json".format(
id_attr = "https://raw.githubusercontent.com/{}/main/nextflow_schema.json".format(
self.pipeline_manifest["name"].strip("\"'")
)
if self.schema["$id"] != id_attr:
raise AssertionError(f"Schema `$id` should be `{id_attr}`\n Found `{self.schema['$id']}`")
if self.schema["$id"] not in [id_attr, id_attr.replace("/main/", "/master/")]:
raise AssertionError(
f"Schema `$id` should be `{id_attr}` or {id_attr.replace('/main/', '/master/')}. \n Found `{self.schema['$id']}`"
)

title_attr = "{} pipeline parameters".format(self.pipeline_manifest["name"].strip("\"'"))
if self.schema["title"] != title_attr:
Expand Down
2 changes: 1 addition & 1 deletion nf_core/subworkflows/lint/subworkflow_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ def subworkflow_tests(_, subworkflow: NFCoreComponent):
subworkflow.warned.append(
(
"test_dir_exists",
"nf-test directory is missing",
"Migrate pytest-workflow to nf-test",
subworkflow.nftest_testdir,
)
)
Expand Down
Loading

0 comments on commit c5dc6ac

Please sign in to comment.