Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove codecov, solely use coverage. #1542

Merged
merged 6 commits into from
Dec 18, 2021

Conversation

tim-schilling
Copy link
Member

CodeCov has become a bit flaky in their evaluation of code coverage.
Using GitHub actions, we can utilize coverage to combine multiple reports.

Thanks to Hynek Schlawack for the approach.
https://hynek.me/articles/ditch-codecov-python/

CodeCov has become a bit flaky in their evaluation of code coverage.
Using GitHub actions, we can utilize coverage to combine multiple reports.

Thanks to Hynek Schlawack for the approach.
https://hynek.me/articles/ditch-codecov-python/

Remove .coveragerc file in favor of configuration in setup.cfg

Also applies coverage's parallel mode always.
@tim-schilling
Copy link
Member Author

@matthiask what do you think about dropping codecov? It's probably possible to get the github actions to create/update a comment on the PR with the report generated.

I'm a bit unsure of all this, but I do know we've had a few PRs fail CI due to codecov saying it didn't cover everything.

Additionally, we'd probably need to lower the failure percentage and/or cover existing code paths.

@matthiask
Copy link
Member

I read the post by Hynek too. It makes a lot of sense to me. Thank you!

@auvipy
Copy link
Contributor

auvipy commented Dec 14, 2021

how is the report looks like?

@tim-schilling
Copy link
Member Author

@auvipy in this implementation you'd have to click on the "Check coverage" action's details link: https://github.com/jazzband/django-debug-toolbar/runs/4514568613?check_suite_focus=true

It contains a table that looks like the following:

Name                                                 Stmts   Miss Branch BrPart  Cover   Missing
------------------------------------------------------------------------------------------------
debug_toolbar/__init__.py                                7      0      2      0   100%
debug_toolbar/apps.py                                   58      3     25      1    95%   78, 116-117
debug_toolbar/decorators.py                             21      0      4      0   100%
debug_toolbar/forms.py                                  25      0      8      0   100%
debug_toolbar/management/__init__.py                     0      0      0      0   100%
debug_toolbar/management/commands/__init__.py            0      0      0      0   100%
debug_toolbar/management/commands/debugsqlshell.py      20      0      4      0   100%
debug_toolbar/middleware.py                             60      1     26      1    98%   32
debug_toolbar/panels/__init__.py                        65      2      6      1    96%   82, 92, 103->exit
debug_toolbar/panels/cache.py                          164     22     44      7    84%   46, 72, 75-77, 80, 83, 87, 113, 117, 121, 125, 129, 133, 137, 141, 163, 221-225, 278->277, 282, 298->297, 300->exit
debug_toolbar/panels/headers.py                         22      0      8      0   100%
debug_toolbar/panels/history/__init__.py                 2      0      0      0   100%
debug_toolbar/panels/history/forms.py                    3      0      2      0   100%
debug_toolbar/panels/history/panel.py                   52      2      8      0    97%   63-65
debug_toolbar/panels/history/views.py                   33      2     12      2    91%   37, 71
debug_toolbar/panels/logging.py                         49      2      8      0    96%   11-12
debug_toolbar/panels/profiling.py                       96      6     32      3    93%   29, 32-33, 43, 102, 110
debug_toolbar/panels/redirects.py                       18      0      6      0   100%
debug_toolbar/panels/request.py                         32      0     10      0   100%
debug_toolbar/panels/settings.py                        16      0      6      0   100%
debug_toolbar/panels/signals.py                         44      3     19      3    90%   56, 79-80, 89->91
debug_toolbar/panels/sql/__init__.py                     2      0      0      0   100%
debug_toolbar/panels/sql/forms.py                       36      4      6      2    86%   31, 40-41, 47
debug_toolbar/panels/sql/panel.py                      164     46     72      7    68%   20-32, 36-48, 68-92, 182-187, 201-205, 207, 211, 215, 223->225, 228-229, 239, 281-282
debug_toolbar/panels/sql/tracking.py                   125      9     48      3    93%   42->exit, 71->exit, 159-160, 193-197, 210, 216
debug_toolbar/panels/sql/utils.py                       51      2     22      2    95%   75, 82
debug_toolbar/panels/sql/views.py                       67      3     22      3    93%   33, 69, 122
debug_toolbar/panels/staticfiles.py                    112      3     36      2    97%   16-17, 42, 177->176
debug_toolbar/panels/templates/__init__.py               2      0      0      0   100%
debug_toolbar/panels/templates/panel.py                124      6     46      7    92%   25->34, 37, 108, 112, 127, 181-182, 185->188
debug_toolbar/panels/templates/views.py                 43     11     10      3    74%   19, 22-23, 30->29, 44-47, 51-56
debug_toolbar/panels/timer.py                           59      8     12      4    80%   11-12, 29-32, 57-59, 63->65, 69->71, 71->93
debug_toolbar/panels/versions.py                        41      1     20      1    97%   58
debug_toolbar/settings.py                               17      0      0      0   100%
debug_toolbar/templatetags/__init__.py                   0      0      0      0   100%
debug_toolbar/toolbar.py                                87      5     28      2    92%   71-79, 98, 127->139
debug_toolbar/urls.py                                    3      0      0      0   100%
debug_toolbar/utils.py                                 153     23     71     13    80%   18-19, 29-30, 78-81, 108, 112->115, 129->128, 158->162, 179-180, 184, 191-192, 200, 204-212, 217, 227, 250, 261->263, 268->270
debug_toolbar/views.py                                  16      0      2      0   100%
------------------------------------------------------------------------------------------------
TOTAL                                                 1889    164    625     67    89%
Coverage failure: total of 89 is less than fail-under=100

@tim-schilling tim-schilling marked this pull request as ready for review December 14, 2021 22:25
Copy link
Member

@pauloxnet pauloxnet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it woul be better start using a pyproject.toml to store configurations.

setup.cfg Outdated Show resolved Hide resolved
.github/workflows/test.yml Outdated Show resolved Hide resolved
.github/workflows/test.yml Outdated Show resolved Hide resolved
setup.cfg Show resolved Hide resolved
auvipy and others added 4 commits December 16, 2021 16:55
Co-authored-by: Paolo Melchiorre <paolo@melchiorre.org>
Co-authored-by: Paolo Melchiorre <paolo@melchiorre.org>
Co-authored-by: Paolo Melchiorre <paolo@melchiorre.org>
Co-authored-by: Paolo Melchiorre <paolo@melchiorre.org>
@auvipy
Copy link
Contributor

auvipy commented Dec 16, 2021

I think it woul be better start using a pyproject.toml to store configurations.

I quite agree with you, but may be in another PR

@tim-schilling
Copy link
Member Author

Any qualms with me setting the failure percentage to 89%?

@matthiask
Copy link
Member

Any qualms with me setting the failure percentage to 89%?

I think that's fine. Maybe you could insert a // TODO Raise this, when the testsuite allows it or something so that the percentage appears in editors' to do lists.

@pauloxnet
Copy link
Member

Any qualms with me setting the failure percentage to 89% ?

I agree with that. We can also define a coverage goal fro the next release (e.g. 90%)

@tim-schilling
Copy link
Member Author

@pauloxnet can you confirm that all of your concerns with the PR were addressed? I'll squash and merge our commits after I get that green checkmark.

Copy link
Member

@pauloxnet pauloxnet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything ok for me.

@tim-schilling tim-schilling merged commit 83e3d1c into django-commons:main Dec 18, 2021
@pauloxnet
Copy link
Member

I noticed only now that in this PR we forgot to update the coverage badge in the README.

matthiask added a commit to matthiask/django-debug-toolbar that referenced this pull request Dec 19, 2021
matthiask added a commit to matthiask/django-debug-toolbar that referenced this pull request Dec 19, 2021
adamchainz added a commit to adamchainz/django-debug-toolbar that referenced this pull request Dec 20, 2021
Since django-commons#1542, coverage creates parallel files, with names like `.coverage.<hostname>.<pid>.<someothernumber>`, which need ignoring too.
tim-schilling pushed a commit that referenced this pull request Dec 20, 2021
Since #1542, coverage creates parallel files, with names like `.coverage.<hostname>.<pid>.<someothernumber>`, which need ignoring too.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants