Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update mock_snakemake #1345

Merged
merged 3 commits into from
Oct 10, 2024
Merged

Update mock_snakemake #1345

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8f1a6b1
test: log env diff (#1338)
lkstrp Oct 1, 2024
dc6ea50
Update mock_snakemake
davide-f Oct 2, 2024
b4905e1
Merge branch 'remove-madd' into fix_mock
fneum Oct 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 27 additions & 4 deletions .github/workflows/test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ concurrency:

env:
ENV_FILE: envs/environment.yaml
ENV_FIXED_FILE: envs/environment.fixed.yaml

jobs:
run-tests:
Expand Down Expand Up @@ -62,16 +63,39 @@ jobs:
uses: actions/cache@v4
with:
path: ${{ env.CONDA }}/envs
key: conda-${{ runner.os }}-${{ runner.arch }}-${{ env.today }}-${hashFiles('${{ env.ENV_FILE }}')}
key: conda-${{ runner.os }}-${{ runner.arch }}-${{ env.today }}-${{ hashFiles(format('{0}', env.ENV_FILE)) }}
id: cache-env

- name: Update environment
if: steps.cache-env.outputs.cache-hit != 'true'
run: conda env update -n pypsa-eur -f ${{ env.ENV_FILE }}

- name: Log env diff to environment.fixed.yaml
if: ${{ matrix.os == 'ubuntu' }}
run: |
# Get fixed environment of current env
conda env export --name ${{ github.event.repository.name }} --no-builds | sed 's/^name: ${{ github.event.repository.name }}$/name: ${{ github.event.repository.name }}-fixed/' > current-env.yaml

# Add SPDX header
SPDX_HEADER="# SPDX-FileCopyrightText: : 2017-2024 The PyPSA-Eur Authors\n# SPDX-License-Identifier: CC0-1.0\n"
echo -e "$SPDX_HEADER" | cat - current-env.yaml > temp && mv temp current-env.yaml

# Format with pre-commit (it differs from the conda output)
pip install pre-commit
pre-commit run --files current-env.yaml || true

# Get diff
diff ${{ env.ENV_FIXED_FILE }} current-env.yaml > diff.txt || true

# Format
{ echo -e "**Environment diff**\n\`\`\` diff\n"; cat diff.txt; echo -e "\n\`\`\`"; } > temp && mv temp diff.txt

# Write to summary
cat diff.txt >> $GITHUB_STEP_SUMMARY
shell: bash

- name: Run snakemake test workflows
run: |
conda activate pypsa-eur
make test

- name: Upload artifacts
Expand Down Expand Up @@ -137,7 +161,7 @@ jobs:
uses: actions/cache@v4
with:
path: ${{ env.CONDA }}/envs
key: conda-${{ runner.os }}-${{ runner.arch }}-${{ matrix.inhouse }}-${{ env.today }}-${hashFiles('${{ env.ENV_FILE }}')}
key: conda-${{ runner.os }}-${{ runner.arch }}-${{ matrix.inhouse }}-${{ env.today }}-${{ hashFiles(format('{0}', env.ENV_FILE)) }}
id: cache-env

- name: Update environment
Expand All @@ -152,7 +176,6 @@ jobs:
- name: Run snakemake test workflows
if: env.pinned == 'false'
run: |
conda activate pypsa-eur
make test

- name: Upload artifacts
Expand Down
2 changes: 1 addition & 1 deletion scripts/_helpers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -434,7 +434,7 @@ def mock_snakemake(
import os

import snakemake as sm
from pypsa.descriptors import Dict
from pypsa.definitions.structures import Dict
from snakemake.api import Workflow
from snakemake.common import SNAKEFILE_CHOICES
from snakemake.script import Snakemake
Expand Down