Document building from the monorepo using `PIP_CONSTRAINT=$SAGE_ROOT/constraints_pkgs.txt` #37434

mkoeppe · 2024-02-23T06:55:20Z

This is aimed at the intermediate level, for users who wish to build a set of distribution packages as supplied in pkgs/ but without using the wheelhouse infrastructure of Sage-the-distribution.

Documentation included in the files.

It can also be tested using new tox environments defined in pkgs/sagemath-standard/tox.ini, and is tested on GH Actions: https://github.com/mkoeppe/sage/actions/workflows/ci-linux-incremental.yml?query=branch%3Aconstraints_pkgs

this also provides the missing automatic test for the sagemath-standard distribution built out of pkgs/sagemath-standard/

Marked as critical because it is hoped to ease the adoption of modularized distribution packages by downstream packagers.

📝 Checklist

The title is concise, informative, and self-explanatory.
The description explains in detail what this PR is about.
I have linked a relevant issue or discussion.
I have created tests covering the changes.
I have updated the documentation accordingly.

⌛ Dependencies

Depends on src/pyproject.toml: Add missing build dep jinja2 #38515 (merged here)
Depends on cython 3.0.11 compatibility #38500 (merged here)

src/pyproject.toml.m4

github-actions · 2024-04-01T10:14:29Z

Documentation preview for this PR (built with commit 3ab2a18; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

kiwifb · 2024-04-15T04:47:53Z

Yes, I understand how this would have failed if the sdist had no .py files like it is happening for 10.3.beta4 right now. For that reason, CI will probably fail if we commit this PR now. I cannot see any obvious problem. I would consider a clean CI run to be enough to decide this can be included.

mkoeppe · 2024-08-17T01:18:57Z

Did you run it within sage -sh? That's needed so that the lrcalc Python bindings can find the lrcalc library -- unless you have a systemwide installation of it.

No. I ran it in a new venv. What is the point of installing the sagemath-standard wheel within sage -sh?

Within sage -sh, environment variables are set that are needed to discover the libraries installed in SAGE_LOCAL.

You can activate your venv within the sage -sh.

kwankyu · 2024-08-17T01:24:54Z

I see. My venvs are outside of sage -sh. There was no problem except lrcalc.

kwankyu · 2024-08-17T01:27:49Z

The step pip install lrcalc was not really necessary.

This is the first time that I successfully installed sagemath-standard distribution package. Personally, it is the most impressive product of the modularization efforts :-)

kwankyu · 2024-08-17T13:05:42Z

Perhaps not an issue directly related with this PR, but I wonder if lrcalc (and others if any) is a prerequisite of sagemath-standard and if that should be documented somewhere.

mkoeppe · 2024-08-17T17:20:57Z

Basically all standard packages are dependencies of sagemath-standard, so https://doc.sagemath.org/html/en/reference/spkg/index.html#standard-packages provides this documentation.

The direct non-Python dependencies of sagemath-standard are also declared in https://github.com/sagemath/sage/blob/develop/src/pyproject.toml#L25, but this is at the moment only decoration because there is no tooling that reads this information.

Note that the user-facing documentation of installation with Python tools is in https://github.com/sagemath/sage/blob/develop/README.md#alternative-installation-using-pypi - it provisions the non-Python dependencies via sage_conf.

kwankyu · 2024-08-18T00:24:10Z

Note that the user-facing documentation of installation with Python tools is in https://github.com/sagemath/sage/blob/develop/README.md#alternative-installation-using-pypi - it provisions the non-Python dependencies via sage_conf.

So Python dependencies are automatically resolved and installed via pip, and non-Python dependencies should be provisioned by the system or by sage_conf. For my case, my system only lacked lrcalc, rather fortunately.

https://github.com/sagemath/sage/blob/develop/README.md#alternative-installation-using-pypi perfectly explains the installation procedure of sagemath-standard. I think that there should be a link to it or a copy in the Project description of sagemath-standard at https://pypi.org/project/sagemath-standard.

…emath/sage/blob/develop/README.md\#alternative-installation-using-pypi

mkoeppe · 2024-08-18T00:34:03Z

Good idea, done in 02f6e88

pkgs/sagemath-standard/constraints_pkgs.txt

…experimental'

…ecific to sagemath-standard

…ecific to sagemath-standard (fixup)

kwankyu · 2024-08-18T02:29:09Z

Perhaps another test run of the changed workflows?

Otherwise lgtm.

mkoeppe · 2024-08-18T03:06:52Z

Thanks. I think I'll iterate on the workflows in a follow up, when changes such as #38518 have been merged

kwankyu · 2024-08-18T04:08:01Z

OK, as the changes on the workflows are just additions and can be tested on the battleground.

mkoeppe · 2024-08-18T04:38:45Z

Thanks for the review!

…TRAINT=$SAGE_ROOT/constraints_pkgs.txt`   This is aimed at the intermediate level, for users who wish to build a set of distribution packages as supplied in `pkgs/` but without using the wheelhouse infrastructure of Sage-the-distribution. Documentation included in the files. It can also be tested using new tox environments defined in `pkgs/sagemath-standard/tox.ini`, and is tested on GH Actions: https://github.com/mkoeppe/sage/actions/workflows/ci-linux- incremental.yml?query=branch%3Aconstraints_pkgs - this also provides the missing automatic test for the **sagemath- standard** distribution built out of `pkgs/sagemath-standard/` Marked as critical because it is hoped to ease the adoption of modularized distribution packages by downstream packagers.    ### 📝 Checklist     - [x] The title is concise, informative, and self-explanatory. - [x] The description explains in detail what this PR is about. - [ ] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies   - Depends on sagemath#38515 (merged here) - Depends on sagemath#38500 (merged here) URL: sagemath#37434 Reported by: Matthias Köppe Reviewer(s): François Bissey, Kwankyu Lee, Matthias Köppe

- Broken in sagemath#37287 - Reported in sagemath#37434 (comment) - Reported in sagemath#36524 (comment) ### 📝 Checklist  - [x] The title is concise and informative. - [x] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation and checked the documentation preview. ### ⌛ Dependencies    URL: sagemath#38515 Reported by: Matthias Köppe Reviewer(s): Kwankyu Lee

Thanks to the more powerful runners in use since early 2024, we can do away with the splits between - `standard-pre` and `standard` - `minimal-pre` and `minimal` - `optional-0-o` and `optional-p-z` - `experimental-0-o` and `experimental-p-z` This makes the CI runs a bit easier to navigate for those who look at them. - Test run: https://github.com/mkoeppe/sage/actions/runs/10438362191 This should also fix "No space left on device" aborts as seen for example in https://github.com/mkoeppe/sage/actions/runs/10426436105 for standard-pre (where we did not use to maximize the available disk space). ### 📝 Checklist  - [x] The title is concise and informative. - [x] The description explains in detail what this PR is about. - [ ] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation and checked the documentation preview. ### ⌛ Dependencies    - Depends on sagemath#37434 (merged here) URL: sagemath#38518 Reported by: Matthias Köppe Reviewer(s): Kwankyu Lee, Matthias Köppe

- Broken in sagemath#37287 - Reported in sagemath#37434 (comment) - Reported in sagemath#36524 (comment) ### 📝 Checklist  - [x] The title is concise and informative. - [x] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation and checked the documentation preview. ### ⌛ Dependencies    URL: sagemath#38515 Reported by: Matthias Köppe Reviewer(s): Kwankyu Lee

Thanks to the more powerful runners in use since early 2024, we can do away with the splits between - `standard-pre` and `standard` - `minimal-pre` and `minimal` - `optional-0-o` and `optional-p-z` - `experimental-0-o` and `experimental-p-z` This makes the CI runs a bit easier to navigate for those who look at them. - Test run: https://github.com/mkoeppe/sage/actions/runs/10438362191 This should also fix "No space left on device" aborts as seen for example in https://github.com/mkoeppe/sage/actions/runs/10426436105 for standard-pre (where we did not use to maximize the available disk space). ### 📝 Checklist  - [x] The title is concise and informative. - [x] The description explains in detail what this PR is about. - [ ] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation and checked the documentation preview. ### ⌛ Dependencies    - Depends on sagemath#37434 (merged here) URL: sagemath#38518 Reported by: Matthias Köppe Reviewer(s): Kwankyu Lee, Matthias Köppe

mkoeppe self-assigned this Feb 23, 2024

mkoeppe added s: needs review c: distribution labels Feb 23, 2024

mkoeppe requested a review from tornaria February 23, 2024 06:56

mkoeppe commented Feb 23, 2024

View reviewed changes

src/pyproject.toml.m4 Outdated Show resolved Hide resolved

mkoeppe mentioned this pull request Feb 23, 2024

pkgs/sagemath-standard: Remove self-reference at build time #37292

Open

5 tasks

mkoeppe added the p: critical / 2 label Feb 23, 2024

mkoeppe changed the title ~~SAGE_ROOT/constraints_pkgs.txt, pkgs/sagemath-standard/constraints_pkgs.txt~~ Building from the monorepo using PIP_CONSTRAINT=$SAGE_ROOT/constraints_pkgs.txt Feb 24, 2024

mkoeppe mentioned this pull request Feb 24, 2024

Generate {constraints,requirements}_sagewheels.txt #37448

Open

5 tasks

mkoeppe requested a review from kiwifb February 24, 2024 22:45

mkoeppe force-pushed the constraints_pkgs branch 2 times, most recently from c75ddba to a59d7d5 Compare March 2, 2024 05:37

mkoeppe requested a review from kwankyu March 6, 2024 07:39

mkoeppe requested a review from saraedum March 17, 2024 16:55

mkoeppe mentioned this pull request Mar 17, 2024

sagemath-standard: include sage_setup in sdist #37287

Merged

2 tasks

mkoeppe force-pushed the constraints_pkgs branch from a59d7d5 to b9fcd14 Compare March 17, 2024 18:06

mkoeppe added c: documentation c: scripts labels Mar 17, 2024

mkoeppe changed the title ~~Building from the monorepo using PIP_CONSTRAINT=$SAGE_ROOT/constraints_pkgs.txt~~ Document building from the monorepo using PIP_CONSTRAINT=$SAGE_ROOT/constraints_pkgs.txt Mar 23, 2024

mkoeppe force-pushed the constraints_pkgs branch 2 times, most recently from 29702ff to 0b1acff Compare April 1, 2024 00:44

mkoeppe mentioned this pull request Apr 15, 2024

sagemath-standard: fix manifest #37286

Merged

2 tasks

mkoeppe force-pushed the constraints_pkgs branch 3 times, most recently from 29958c1 to 3b4d95d Compare April 15, 2024 05:35

mkoeppe added the s: needs work label May 25, 2024

mkoeppe force-pushed the constraints_pkgs branch from 0da65cc to 6437b40 Compare May 31, 2024 20:05

github-actions bot added the v: moderate label May 31, 2024

mkoeppe marked this pull request as ready for review August 17, 2024 01:39

github-actions bot added the s: needs review label Aug 17, 2024

pkgs/sagemath-standard/README.rst: Add link to https://github.com/sag…

02f6e88

…emath/sage/blob/develop/README.md\#alternative-installation-using-pypi

kwankyu reviewed Aug 18, 2024

View reviewed changes

pkgs/sagemath-standard/constraints_pkgs.txt Show resolved Hide resolved

mkoeppe added 4 commits August 17, 2024 17:45

pkgs/sagemath-standard/README.rst: Update copyright year; no longer '…

524dc88

…experimental'

pkgs/sagemath-standard/constraints_pkgs.txt: Make explanation more sp…

368e9f6

…ecific to sagemath-standard

pkgs/sagemath-standard/constraints_pkgs.txt: Make explanation more sp…

4204f2d

…ecific to sagemath-standard (fixup)

pkgs/sagemath-standard/constraints_pkgs.txt: Change to just 'Example:'

3ab2a18

kwankyu approved these changes Aug 18, 2024

View reviewed changes

kwankyu added s: positive review and removed s: needs review labels Aug 18, 2024

kwankyu mentioned this pull request Aug 18, 2024

CI Linux: Consolidate jobs #38518

Merged

5 tasks

vbraun merged commit 5ee375c into sagemath:develop Sep 3, 2024
17 of 20 checks passed

github-actions bot removed the s: positive review label Sep 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Document building from the monorepo using `PIP_CONSTRAINT=$SAGE_ROOT/constraints_pkgs.txt` #37434

Document building from the monorepo using `PIP_CONSTRAINT=$SAGE_ROOT/constraints_pkgs.txt` #37434

mkoeppe commented Feb 23, 2024 •

edited

Loading

github-actions bot commented Apr 1, 2024 •

edited

Loading

kiwifb commented Apr 15, 2024

mkoeppe commented Aug 17, 2024

kwankyu commented Aug 17, 2024 •

edited

Loading

kwankyu commented Aug 17, 2024 •

edited

Loading

kwankyu commented Aug 17, 2024

mkoeppe commented Aug 17, 2024 •

edited

Loading

kwankyu commented Aug 18, 2024

mkoeppe commented Aug 18, 2024

kwankyu commented Aug 18, 2024 •

edited

Loading

mkoeppe commented Aug 18, 2024

kwankyu commented Aug 18, 2024

mkoeppe commented Aug 18, 2024

Document building from the monorepo using PIP_CONSTRAINT=$SAGE_ROOT/constraints_pkgs.txt #37434

Document building from the monorepo using PIP_CONSTRAINT=$SAGE_ROOT/constraints_pkgs.txt #37434

Conversation

mkoeppe commented Feb 23, 2024 • edited Loading

📝 Checklist

⌛ Dependencies

github-actions bot commented Apr 1, 2024 • edited Loading

kiwifb commented Apr 15, 2024

mkoeppe commented Aug 17, 2024

kwankyu commented Aug 17, 2024 • edited Loading

kwankyu commented Aug 17, 2024 • edited Loading

kwankyu commented Aug 17, 2024

mkoeppe commented Aug 17, 2024 • edited Loading

kwankyu commented Aug 18, 2024

mkoeppe commented Aug 18, 2024

kwankyu commented Aug 18, 2024 • edited Loading

mkoeppe commented Aug 18, 2024

kwankyu commented Aug 18, 2024

mkoeppe commented Aug 18, 2024

Document building from the monorepo using `PIP_CONSTRAINT=$SAGE_ROOT/constraints_pkgs.txt` #37434

Document building from the monorepo using `PIP_CONSTRAINT=$SAGE_ROOT/constraints_pkgs.txt` #37434

mkoeppe commented Feb 23, 2024 •

edited

Loading

github-actions bot commented Apr 1, 2024 •

edited

Loading

kwankyu commented Aug 17, 2024 •

edited

Loading

kwankyu commented Aug 17, 2024 •

edited

Loading

mkoeppe commented Aug 17, 2024 •

edited

Loading

kwankyu commented Aug 18, 2024 •

edited

Loading