Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Started to raise "AssertionError: Oops, byte_incr = 2, line_incr = 2" with switch to coverage 7.6.5 py 3.9 #1891

Closed
yarikoptic opened this issue Nov 14, 2024 · 4 comments
Labels
bug Something isn't working fixed

Comments

@yarikoptic
Copy link

Describe the bug

We finally decided to ditch python 3.8 and thus switched CI runs to 3.9 in

but many runs failed with similar problem(s), e.g. from https://github.com/datalad/datalad/actions/runs/11844567886/job/33008095384?pr=7682

2024-11-14T20:05:30.1119963Z �[36;1mpython -m coverage xml�[0m
2024-11-14T20:05:30.1177922Z shell: /usr/bin/bash -e {0}
2024-11-14T20:05:30.1178206Z env:
2024-11-14T20:05:30.1178509Z   pythonLocation: /opt/hostedtoolcache/Python/3.9.20/x64
2024-11-14T20:05:30.1179046Z   PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.9.20/x64/lib/pkgconfig
2024-11-14T20:05:30.1179571Z   Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.20/x64
2024-11-14T20:05:30.1180054Z   Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.20/x64
2024-11-14T20:05:30.1180527Z   Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.9.20/x64
2024-11-14T20:05:30.1181008Z   LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.9.20/x64/lib
2024-11-14T20:05:30.1181401Z   DL_PIP_INSTALLS: mock
2024-11-14T20:05:30.1181667Z   DL_APT_INSTALLS: dcm2niix
2024-11-14T20:05:30.1181972Z   DL_REPO: datalad/datalad-container
2024-11-14T20:05:30.1182325Z   DL_TESTER: pytest
2024-11-14T20:05:30.1182569Z   DL_NEED_SINGULARITY: 1
2024-11-14T20:05:30.1182840Z   DL_PACKAGE: datalad_container
2024-11-14T20:05:30.1183131Z ##[endgroup]
2024-11-14T20:05:32.0063609Z Traceback (most recent call last):
2024-11-14T20:05:32.0065160Z   File "/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/runpy.py", line 197, in _run_module_as_main
2024-11-14T20:05:32.0066400Z     return _run_code(code, main_globals, None,
2024-11-14T20:05:32.0067530Z   File "/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/runpy.py", line 87, in _run_code
2024-11-14T20:05:32.0068599Z     exec(code, run_globals)
2024-11-14T20:05:32.0070188Z   File "/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/coverage/__main__.py", line 10, in <module>
2024-11-14T20:05:32.0071416Z     sys.exit(main())
2024-11-14T20:05:32.0075327Z   File "/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/coverage/cmdline.py", line 970, in main
2024-11-14T20:05:32.0076573Z     status = CoverageScript().command_line(argv)
2024-11-14T20:05:32.0078003Z   File "/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/coverage/cmdline.py", line 730, in command_line
2024-11-14T20:05:32.0079144Z     total = self.coverage.xml_report(
2024-11-14T20:05:32.0080387Z   File "/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/coverage/control.py", line 1210, in xml_report
2024-11-14T20:05:32.0081912Z     return render_report(self.config.xml_output, XmlReporter(self), morfs, self._message)
2024-11-14T20:05:32.0083963Z   File "/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/coverage/report_core.py", line 62, in render_report
2024-11-14T20:05:32.0085247Z     ret = reporter.report(morfs, outfile=outfile)
2024-11-14T20:05:32.0086608Z   File "/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/coverage/xmlreport.py", line 106, in report
2024-11-14T20:05:32.0087913Z     for fr, analysis in get_analysis_to_report(self.coverage, morfs):
2024-11-14T20:05:32.0089595Z   File "/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/coverage/report_core.py", line 101, in get_analysis_to_report
2024-11-14T20:05:32.0090928Z     analysis = coverage._analyze(morf)
2024-11-14T20:05:32.0092220Z   File "/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/coverage/control.py", line 946, in _analyze
2024-11-14T20:05:32.0093805Z     return analysis_from_file_reporter(data, self.config.precision, file_reporter, filename)
2024-11-14T20:05:32.0095699Z   File "/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/coverage/results.py", line 31, in analysis_from_file_reporter
2024-11-14T20:05:32.0096977Z     statements = file_reporter.lines()
2024-11-14T20:05:32.0098246Z   File "/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/coverage/python.py", line 195, in lines
2024-11-14T20:05:32.0099401Z     return self.parser.statements
2024-11-14T20:05:32.0101464Z   File "/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/coverage/python.py", line 190, in parser
2024-11-14T20:05:32.0102689Z     self._parser.parse_source()
2024-11-14T20:05:32.0104027Z   File "/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/coverage/parser.py", line 267, in parse_source
2024-11-14T20:05:32.0105164Z     self._raw_parse()
2024-11-14T20:05:32.0106337Z   File "/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/coverage/parser.py", line 199, in _raw_parse
2024-11-14T20:05:32.0107634Z     self.raw_statements.update(byte_parser._find_statements())
2024-11-14T20:05:32.0109207Z   File "/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/coverage/parser.py", line 459, in _find_statements
2024-11-14T20:05:32.0110469Z     yield from bp._line_numbers()
2024-11-14T20:05:32.0111801Z   File "/opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/coverage/parser.py", line 440, in _line_numbers
2024-11-14T20:05:32.0113208Z     assert line_num != last_line_num, f"Oops, {byte_incr = }, {line_incr = }"
2024-11-14T20:05:32.0114062Z AssertionError: Oops, byte_incr = 2, line_incr = 2
2024-11-14T20:05:32.1348945Z ##[error]Process completed with exit code 1.

in the logs before running that command, while running actual tests

2024-11-14T20:05:25.0073912Z bb -> .datalad/environments/bb/image
2024-11-14T20:05:27.0085054Z /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/coverage/report_core.py:116: CoverageWarning: Couldn't parse '/home/runner/work/datalad/datalad/datalad/distribution/tests/test_siblings.py': Oops, byte_incr = 2, line_incr = 2 (couldnt-parse)
2024-11-14T20:05:27.0087672Z   coverage._warn(msg, slug="couldnt-parse")
2024-11-14T20:05:27.8205359Z /opt/hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/coverage/report_core.py:116: CoverageWarning: Couldn't parse '/home/runner/work/datalad/datalad/datalad/local/subdatasets.py': Oops, byte_incr = 2, line_incr = 2 (couldnt-parse)
2024-11-14T20:05:27.8207952Z   coverage._warn(msg, slug="couldnt-parse")
2024-11-14T20:05:29.9715809Z PASSED

so most likely coverage has issues parsing these two files

but unclear with what and why not before.

To Reproduce
How can we reproduce the problem? Please be specific. Don't link to a failing CI job. Answer the questions below:

  1. What version of Python are you using?

3.9

  1. What version of coverage.py shows the problem? The output of coverage debug sys is helpful.
2024-11-14T20:02:02.1498025Z Collecting coverage (from datalad==1.1.3+35.gab7380b9e)
2024-11-14T20:02:02.1649949Z   Downloading coverage-7.6.5-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (8.2 kB)

before this we had 3.8 and it was installing e.g coverage-7.6.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2 and we had no such issues

  1. What versions of what packages do you have installed? The output of pip freeze is helpful.

here is some

dependencies

  • annexremote: 1.6.6
  • boto3: 1.35.60
  • humanize: 4.11.0
  • iso8601: 2.1.0
  • keyring: 25.5.0
  • keyrings.alt: 5.0.2
  • msgpack: 1.1.0
  • platformdirs: 4.3.6
  • requests: 2.32.3
  1. What code shows the problem? Give us a specific commit of a specific repo that we can check out. If you've already worked around the problem, please provide a commit before that fix.
  2. What commands should we run to reproduce the problem? Be specific. Include everything, even git clone, pip install, and so on. Explain like we're five!

failed to replicate locally.

Expected behavior

Not to raise AssertionError ;)

More information on problem with parsing -- at least point to the line numbers or even visualize them.

Additional context
Add any other context about the problem here.

@yarikoptic yarikoptic added the bug Something isn't working label Nov 14, 2024
@yarikoptic
Copy link
Author

FWIW, pinning coverage to 7.6.1 provided a workaround, so should be related to diff between 7.6.1 and 7.6.5.

yarikoptic added a commit to yarikoptic/datalad that referenced this issue Nov 15, 2024
Underlying issue is with 7.6.5: reported to
nedbat/coveragepy#1891 . First 7.6.1 was tried
and it worked. Next bisection is 7.6.3. We will stop on highest working.
@nedbat
Copy link
Owner

nedbat commented Nov 15, 2024

Yes, the changelog for 7.6.5 mentions:

  • refactor: some code unreachable code paths in parser.py were changed to asserts. If you encounter any of these, please let me know!

I'll take a look at the files you've indicated to understand why they are different than most.

yarikoptic added a commit to yarikoptic/datalad that referenced this issue Nov 15, 2024
Underlying issue is with 7.6.5: reported to
nedbat/coveragepy#1891 . First 7.6.1 was tried
and it worked. Next bisection is 7.6.3 was ok.
nedbat clarified that indeed in 7.6.5 some assertions were added:
nedbat/coveragepy#1891 (comment)

So we will just keep it that ceiled until issue resolved.
jvesely added a commit to jvesely/PsyNeuLink that referenced this issue Nov 15, 2024
nedbat/coveragepy#1891

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
jvesely added a commit to jvesely/PsyNeuLink that referenced this issue Nov 15, 2024
nedbat/coveragepy#1891

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
wpbonelli added a commit to modflowpy/flopy that referenced this issue Nov 15, 2024
@nedbat
Copy link
Owner

nedbat commented Nov 15, 2024

This is fixed in commit 98939c9 and released as part of coverage 7.6.6.

@nedbat nedbat closed this as completed Nov 15, 2024
@nedbat nedbat added the fixed label Nov 15, 2024
@nedbat
Copy link
Owner

nedbat commented Nov 15, 2024

If anyone finds another similar assert in 7.6.6, please provide details.

jvesely added a commit to PrincetonUniversity/PsyNeuLink that referenced this issue Nov 15, 2024
nedbat/coveragepy#1891

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working fixed
Projects
None yet
Development

No branches or pull requests

2 participants