Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI Build&Test: Split test-long into multiple jobs, repair Coverage.py upload #38090

Merged
merged 6 commits into from
Jun 1, 2024

Conversation

mkoeppe
Copy link
Member

@mkoeppe mkoeppe commented May 26, 2024

test-long (90 minutes) runs after a successful test-new (10 minutes). This is the critical path (100 minutes), longer than build-docs-pdf (85 minutes). We split it into multiple parallel jobs (33 minutes), merging the coverage files in a job afterwards (6 minutes).

This also repairs the Coverage.py upload, which apparently got broken.

📝 Checklist

  • The title is concise and informative.
  • The description explains in detail what this PR is about.
  • I have linked a relevant issue or discussion.
  • I have created tests covering the changes.
  • I have updated the documentation and checked the documentation preview.

⌛ Dependencies

@mkoeppe mkoeppe self-assigned this May 26, 2024
Copy link

github-actions bot commented May 26, 2024

Documentation preview for this PR (built with commit a0262b6; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

@mkoeppe mkoeppe force-pushed the ci_split_long_tests branch 5 times, most recently from 53727f8 to d24f52f Compare May 26, 2024 19:04
@mkoeppe mkoeppe force-pushed the ci_split_long_tests branch 4 times, most recently from c1177cc to 2d31669 Compare May 26, 2024 20:31
@mkoeppe mkoeppe changed the title CI Build&Test: Split test-long into multiple jobs CI Build&Test: Split test-long into multiple jobs, repair Coverage.py upload May 26, 2024
@mkoeppe mkoeppe added this to the sage-10.4 milestone May 26, 2024
@mkoeppe mkoeppe marked this pull request as ready for review May 26, 2024 22:44
@kwankyu
Copy link
Collaborator

kwankyu commented May 28, 2024

Is there an example run?

This is to save 15 minutes. Is it worth the effort?

Workflow scripts are the second-class citizen. But in the long run, they are still burden of the community.

The workflow script gets more complicated. Saving 15 minutes is worth it?

@mkoeppe
Copy link
Member Author

mkoeppe commented May 28, 2024

Is there an example run?

Yes, right here on the PR

@mkoeppe
Copy link
Member Author

mkoeppe commented May 28, 2024

This is to save 15 minutes.

... to completion of all jobs, yes. But the time to completion of test-long is cut in half from 100 minutes to 50 minutes

@kwankyu
Copy link
Collaborator

kwankyu commented May 28, 2024

From the example run,

+ ./sage -python -m coverage combine --rcfile=src/tox.ini .coverage/coverage-src_doc--src_sage_docbuild--src_sage_setup/.coverage .coverage/coverage-src_sage_a-f/.coverage .coverage/coverage-src_sage_g-o/.coverage .coverage/coverage-src_sage_p-z/.coverage
Combined data file .coverage/coverage-src_doc--src_sage_docbuild--src_sage_setup/.coverage
Combined data file .coverage/coverage-src_sage_a-f/.coverage
Combined data file .coverage/coverage-src_sage_g-o/.coverage
Combined data file .coverage/coverage-src_sage_p-z/.coverage
+ ./sage -python -m coverage xml --rcfile=src/tox.ini
/sage/local/var/lib/sage/venv-python3.10/lib/python3.10/site-packages/coverage/report_core.py:115: CoverageWarning: Couldn't parse '/tmp/tmp06qizzie/tmp_fte2alrs.py': No source for code: '/tmp/tmp06qizzie/tmp_fte2alrs.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
/sage/local/var/lib/sage/venv-python3.10/lib/python3.10/site-packages/coverage/report_core.py:115: CoverageWarning: Couldn't parse '/tmp/tmp06qizzie/tmp_ldpu46ob.py': No source for code: '/tmp/tmp06qizzie/tmp_ldpu46ob.py'. (couldnt-parse)
...

Could we remove the warnings?

@kwankyu
Copy link
Collaborator

kwankyu commented May 28, 2024

... the time to completion of test-long is cut in half from 100 minutes to 50 minutes

OK. Certainly nice reward for the effort.

@mkoeppe
Copy link
Member Author

mkoeppe commented May 28, 2024

Could we remove the warnings?

Trying in a0262b6

Copy link
Collaborator

@kwankyu kwankyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. LGTM.

@mkoeppe
Copy link
Member Author

mkoeppe commented May 28, 2024

Thank you!

vbraun pushed a commit to vbraun/sage that referenced this pull request May 29, 2024
…repair Coverage.py upload

    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->

test-long (90 minutes) runs after a successful test-new (10 minutes).
This is the critical path (100 minutes), longer than build-docs-pdf (85
minutes). We split it into multiple parallel jobs (33 minutes), merging
the coverage files in a job afterwards (6 minutes).

This also repairs the Coverage.py upload, which apparently got broken.

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [x] The title is concise and informative.
- [x] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#38090
Reported by: Matthias Köppe
Reviewer(s): Kwankyu Lee
@vbraun vbraun merged commit aa8e10f into sagemath:develop Jun 1, 2024
20 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants