Skip to content

Commit

Permalink
Don't run tests twice in CI
Browse files Browse the repository at this point in the history
As a first draft, tests ran twice (one for the "test" and once for the
skipped test collator). Switch to upload/download of junitxml report
data to pass the reports from the matrix to the collator.
  • Loading branch information
sirosen committed Feb 8, 2023
1 parent d183045 commit 42d21dc
Show file tree
Hide file tree
Showing 3 changed files with 19 additions and 29 deletions.
34 changes: 14 additions & 20 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -75,35 +75,29 @@ jobs:
- name: test
run: |
python -m tox run-parallel -m ci
python -m tox run-parallel -m ci -- --junitxml pytest.{envname}.xml
python -m tox run -e cov
- uses: actions/upload-artifact@v3
with:
name: pytest-report-py${{ matrix.py }}-${{ matrix.os }}
path: pytest.*.xml

collate-tests:
needs: [ci-test-matrix]
runs-on: ubuntu-latest
name: "Collate results to check for skipped tests"
steps:
- uses: actions/checkout@v3

- name: get date for caching
run: /bin/date -u "+%U" > cachedate.txt
shell: bash

- uses: actions/setup-python@v4
with:
python-version: |
3.7
3.10
3.11
cache: "pip"
cache-dependency-path: |
.github/workflows/build.yaml
setup.cfg
tox.ini
cachedate.txt
- run: python -m pip install tox

- run: make collated-test-report
python-version: "3.x"
# download everything
- uses: actions/download-artifact@v3
with:
path: artifacts/
# collate and report
- run: python ./scripts/aggregate-pytest-reports.py artifacts/*/pytest.*.xml

self-check:
name: "Self-Check"
Expand Down
9 changes: 2 additions & 7 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,8 @@ release:

.PHONY: collated-test-report
collated-test-report:
tox p -e py37-mindeps,py311,py310-notoml,py310-tomli-format,py311-json5 -- '--junitxml={envdir}/pytest.xml'
python ./scripts/aggregate-pytest-reports.py \
.tox/py37-mindeps/pytest.xml \
.tox/py311/pytest.xml \
.tox/py310-notoml/pytest.xml \
.tox/py310-tomli-format/pytest.xml \
.tox/py311-json5/pytest.xml
tox p
python ./scripts/aggregate-pytest-reports.py .tox/*/pytest.xml

.PHONY: clean
clean:
Expand Down
5 changes: 3 additions & 2 deletions tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ skip_missing_interpreters = true
minversion = 4.0.0

labels =
ci = py-notoml, py-tomli-format, py-json5, py-pyjson5
ci = py, py-notoml, py-tomli-format, py-json5, py-pyjson5

[testenv]
description = "run tests with pytest"
Expand All @@ -28,7 +28,8 @@ deps =
format: jsonschema[format]
set_env =
notoml: FORCE_TOML_DISABLED=1
commands = coverage run -m pytest {posargs}
commands =
coverage run -m pytest {posargs:--junitxml={envdir}/pytest.xml}

[testenv:cov_clean]
description = "erase coverage data to prepare for a new run"
Expand Down

0 comments on commit 42d21dc

Please sign in to comment.