pkgs/sagemath-*/tox.ini: Create environment that uses Sage's python as the basepython, fix sagemath-standard deps

[mkoeppe]

Newer versions of tox seem to provision python 3.10 when the environment python is used, but then our wheels don't necessarily fit.

For this reason, the examples shown in the developer's manual hardcoded the Python version (tox -e py39).

We now define tox environments sagepython in tox.ini, which takes Python as SAGE_VENV/bin/python3 and change the manual to use this environment in examples, such as:

./bootstrap && ./sage -sh -c '(cd pkgs/sagemath-standard && SAGE_NUM_THREADS=16 tox -v -v -v -e sagepython-sagewheels-nopypi)'

(to be run after make build)

Depends on #33936

CC: @dimpase @kwankyu

Component: refactoring

Author: Matthias Koeppe

Branch: ca9d6ce

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/32716

[mkoeppe]

Branch: u/mkoeppe/pkgs_sagemath_standard_tox_ini__use_sage_s_python_by_default

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

Commit: 9db6b2f

[mkoeppe]

New commits:

5ce30b8	pkgs/sage-setup, sagemath-{objects,categories,standard}: Add tox environment sagepython
9db6b2f	src/doc/en/developer/packaging_sage_library.rst: Use sagepython instead of hardcoded py39

[dimpase]

comment:8

command in the ticket description fails with

WARNING: Url 'file:///home/scratch/scratch2/dimpase/sage/sagetrac-mirror/pkgs/sagemath-standard/../../../../venv/var/lib/sage/wheels' is ignored: it is neither a file nor a directory.
ERROR: Could not find a version that satisfies the requirement sage-conf==9.7.beta1
ERROR: No matching distribution found for sage-conf==9.7.beta1
ERROR: invocation failed (exit code 1)
ERROR: could not install deps [-rrequirements.txt]; v = InvocationError('/home/scratch/scratch2/dimpase/sage/sagetrac-mirror/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/bin/python -m pip install -rrequirements.txt', 1)
sagepython-sagewheels-nopypi finish: getenv /home/scratch/scratch2/dimpase/sage/sagetrac-mirror/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi after 0.69 seconds
...

[mkoeppe]

comment:9

Did you run make build as the manual explains?

[dimpase]

comment:11

besides make build, I also needed to apply

diff --git a/build/pkgs/sage_conf/install-requires.txt b/build/pkgs/sage_conf/install-requires.txt
index 83ad7e641fa..3d4ff51d565 100644
--- a/build/pkgs/sage_conf/install-requires.txt
+++ b/build/pkgs/sage_conf/install-requires.txt
@@ -1 +1 @@
-sage-conf ~= 9.5.b6
+sage-conf
diff --git a/build/pkgs/sage_setup/install-requires.txt b/build/pkgs/sage_setup/install-requires.txt
index 01a931f5eb5..9d4666c07f4 100644
--- a/build/pkgs/sage_setup/install-requires.txt
+++ b/build/pkgs/sage_setup/install-requires.txt
@@ -1 +1 @@
-sage-setup ~= 9.5.b6
+sage-setup

that 9.5.b6 seemed to come from an obsolete workaround anyway.

[dimpase]

comment:12

And after these changes, it errors out (much later) with

ModuleNotFoundError: No module named 'primecountpy'
ERROR: InvocationError for command /home/scratch/scratch2/dimpase/sage/sagetrac-mirror/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/bin/python -c 'import sys; "" in sys.path and sys.path.remove(""); import sage.all; print(sage.all.__file__)' (exited with code 1)

[dimpase]

comment:13

the latter is probably a config error, as the module is there and well:

sage: import primecountpy
sage: primecountpy.primecount.prime_pi(100)
25

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

b030e7a

Merge tag '9.7.beta2' into t/32716/pkgs_sagemath_standard_tox_ini__use_sage_s_python_by_default

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

5ea0035	src/requirements.txt.m4, src/setup.cfg.m4 (install_requires): Add lrcalc_python
7b17c2d	Split sage_docbuild.conf to sage.misc.sagedoc_conf
282270f	Fix docs
c2766bf	Move skip_member() to sage_docbuild
4296851	Move the remnants of skip_member from sagedoc_conf
45f9ff3	Merge branch 'develop'
0a02496	Change assert back to if
1c2f41b	Merge #33936

[mkoeppe]

comment:43

Replying to @dimpase:

I'm trying the change for lrcalc analogous to the one for primecountpy.

Indeed, that was missing too, fixed now.

[dimpase]

comment:46

Replying to @dimpase:

I'm trying the change for lrcalc analogous to the one for primecountpy.

that appeared to suffice (but now the branch got updated)

[dimpase]

comment:47

what I don't like are explicit versions of sage-conf and sage-setup in requirements.txt

Obviously there should be an automated way.

[dimpase]

Reviewer: Dima Pasechnik

[dimpase]

comment:48

OK, otherwise it looks good. Perhaps deal with versions mentioned in the last comment on another ticket (unless it can be trivially fixed).

[mkoeppe]

comment:49

Replying to @dimpase:

what I don't like are explicit versions of sage-conf and sage-setup in requirements.txt

Obviously there should be an automated way.

src/requirements.txt is already generated from src/requirements.txt.m4

If you mean build/pkgs/sage_conf/install-requires.txt and build/pkgs/sage_setup/install-requires.txt, yes, that will need a better solution but it's way beyond the scope of this ticket

[dimpase]

comment:51

OK

[mkoeppe]

comment:52

Thanks!

[vbraun]

comment:53

Merge failure on top of:

662aaf2 Trac #32423: Update numpy to 1.22.x, scipy 1.8.x, networkx 2.8.x

d98aa73 Trac #29549: bootstrap: Clean up use of gettextize

24b4eeb Trac #13321: FreeModule.hom stores its matrix over the wrong ring

93357f3 Trac #34104: Build & Test workflow: Run apt-get update first

23e4bbe Trac #34092: Docbuild workflow broken because furo is too new

9440e71 Trac #34088: cygwin python3 missing ensurepip

9879ff4 Trac #33936: Remove runtime dependency on sage_docbuild introduced in #33763

55d05c9 Trac #31403: giac: Make cliquer a dependency, libnauty an optional dependency

843eb03 Updated SageMath version to 9.7.beta4

merge was not clean: conflicts in pkgs/sagemath-standard/tox.ini

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

ca9d6ce

Merge tag '9.7.beta4' into t/32716/pkgs_sagemath_standard_tox_ini__use_sage_s_python_by_default

[sagetrac-git]

Changed commit from 1c2f41b to ca9d6ce

[vbraun]

Changed branch from u/mkoeppe/pkgs_sagemath_standard_tox_ini__use_sage_s_python_by_default to ca9d6ce

[mkoeppe]

Changed commit from ca9d6ce to none

[mkoeppe]

comment:57

Replying to @mkoeppe:

Replying to @dimpase:

what I don't like are explicit versions of sage-conf and sage-setup in requirements.txt

Obviously there should be an automated way.

src/requirements.txt is already generated from src/requirements.txt.m4

If you mean build/pkgs/sage_conf/install-requires.txt and build/pkgs/sage_setup/install-requires.txt, yes, that will need a better solution but it's way beyond the scope of this ticket

Implemented now in #33817