Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better test coverage reports #3498

Merged
merged 14 commits into from
Sep 5, 2024
Merged

Conversation

antonpirker
Copy link
Member

@antonpirker antonpirker commented Sep 4, 2024

Our coverage reports are broken. This PR tries to fix them.

  • Sometimes the coverage report XML files contain references to files in /tmp/... (this can happen if dependencies write those files) so the first change is to omit those files.
  • We created our coverage reports with coverage xml -i where the -i means "ignore errors". This is why we never found out about problems generating coverage reports. Report generation fails now verbose (everywhere except in Python 3.6, because there are always some errors there because it can not parse python files with async code, but I guess those can be savely ignored)
  • For Python 3.6 we know have a special coverage config (.coveragerc36) because the option exclude_also was named exclude_lines in older coverage.py versions.

Copy link

codecov bot commented Sep 4, 2024

❌ 4 Tests Failed:

Tests completed Failed Passed Skipped
11541 4 11537 1637
View the top 3 failed tests by shortest run time
tests.integrations.aws_lambda.test_aws test_init_error[python3.9]
Stack Traces | 10.9s run time
.../integrations/aws_lambda/test_aws.py:334: in test_init_error
    (event,) = envelope_items
E   ValueError: not enough values to unpack (expected 1, got 0)
tests.integrations.aws_lambda.test_aws test_init_error[python3.11]
Stack Traces | 11.3s run time
.../integrations/aws_lambda/test_aws.py:334: in test_init_error
    (event,) = envelope_items
E   ValueError: not enough values to unpack (expected 1, got 0)
tests.integrations.aws_lambda.test_aws test_init_error[python3.10]
Stack Traces | 16.4s run time
.../integrations/aws_lambda/test_aws.py:334: in test_init_error
    (event,) = envelope_items
E   ValueError: not enough values to unpack (expected 1, got 0)

To view individual test run time comparison to the main branch, go to the Test Analytics Dashboard

@antonpirker antonpirker changed the title Maybe better coverage config Better test coverage reports Sep 5, 2024
@antonpirker antonpirker marked this pull request as ready for review September 5, 2024 11:33
@antonpirker antonpirker merged commit e99873d into master Sep 5, 2024
124 of 126 checks passed
@antonpirker antonpirker deleted the antonpirker/better-coverage branch September 5, 2024 12:13
arjennienhuis pushed a commit to arjennienhuis/sentry-python that referenced this pull request Sep 30, 2024
Our coverage reports are broken. This PR tries to fix them.

-  Sometimes the coverage report XML files contain references to files in `/tmp/...` (this can happen if dependencies write those files) so the first change is to omit those files.
- We created our coverage reports with `coverage xml -i` where the `-i` means "ignore errors". This is why we never found out about problems generating coverage reports. Report generation fails now verbose (everywhere except in Python 3.6, because there are always some errors there because it can not parse python files with async code, but I guess those can be savely ignored)
- For Python 3.6 we know have a special coverage config (`.coveragerc36`) because the option `exclude_also` was named `exclude_lines` in older coverage.py versions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants