build/bin/sage-spkg: Add support for installing script packages

[mkoeppe]

Split out and updated from #29386.

The section of build/make/Makefile.in that deals with script packages can be consolidated with the section on normal packages later, in a follow-up PR after #36740, #36738.

We switch the spkg-install scripts of the sage* script packages to the templated versions (spkg-install.in), which is a simplification. spkg-check remains as is because we still invoke it directly (changing this will need #36738).

Resolves #29386.

📝 Checklist

• The title is concise, informative, and self-explanatory.
• The description explains in detail what this PR is about.
• I have linked a relevant issue or discussion.
• I have created tests covering the changes.
• I have updated the documentation accordingly.

⌛ Dependencies

• Depends on Fix make SPKG-uninstall for Python packages after #36452 #36737 (for testing)

[orlitzky]

I get two errors under "normal" circumstances that don't hurt anything:

$ sage-spkg sage_conf
...
Package 'sage_conf' is currently not installed
No legacy uninstaller found for 'sage_conf'; nothing to do
Installing sage_conf-10.2.rc4
Looking in links: /home/mjo/src/sage.git/local/var/lib/sage/wheels
Processing /home/mjo/src/sage.git/pkgs/sage-conf
  Running command pip subprocess to install build dependencies
  Looking in links: /home/mjo/src/sage.git/local/var/lib/sage/wheels
  WARNING: Location '/home/mjo/src/sage.git/local/var/lib/sage/wheels' is ignored: it is either a non-existing path or lacks a specific scheme.
  ERROR: Could not find a version that satisfies the requirement setuptools (from versions: none)
  ERROR: No matching distribution found for setuptools
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/mjo/src/sage.git/local/var/lib/sage/venv-python3.11/bin/python3 /usr/lib/python3.11/site-packages/pip/__pip-runner__.py install --ignore-installed --no-user --prefix /tmp/pip-build-env-eto9101s/overlay --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links /home/mjo/src/sage.git/local/var/lib/sage/wheels -- setuptools wheel
  cwd: [inherit]
  Installing build dependencies ... error
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
Warning: building with "python3 -m pip wheel --wheel-dir=dist --verbose --no-deps --no-index --isolated --ignore-requires-python --find-links=/home/mjo/src/sage.git/local/var/lib/sage/wheels" failed.
Retrying with "python3 -m pip wheel --wheel-dir=dist --verbose --no-deps --no-index --isolated --ignore-requires-python --no-build-isolation --no-binary :all:".

The retry works, but the error is in bright red and always happens AFAICT so it would be better if, well, it didn't. It may have something to do with --enable-system-site-packages since I'm using almost everything (and in particular setuptools and wheel) from the system.

The other error can probably be fixed with an -f test?

$ sage-spkg-uninstall sage_conf
Uninstalling existing 'sage_conf'
Running pip-uninstall script for 'sage_conf'
ERROR: Could not open requirements file: [Errno 2] No such file or directory: '/home/mjo/src/sage.git/local/var/lib/sage/scripts/sage_conf/spkg-requirements.txt'
Warning: pip exited with status 0
Removing stamp file '/home/mjo/src/sage.git/local/var/lib/sage/installed/sage_conf-10.2.rc4'

[mkoeppe]

It may have something to do with --enable-system-site-packages since I'm using almost everything (and in particular setuptools and wheel) from the system.

The first try uses build isolation, so it needs the wheel file of setuptools.

[mkoeppe]

... so perhaps when using --enable-system-site-packages, you would want to disable build isolation?

[mkoeppe]

The other error can probably be fixed with an -f test?

$ sage-spkg-uninstall sage_conf
Uninstalling existing 'sage_conf'
Running pip-uninstall script for 'sage_conf'
ERROR: Could not open requirements file: [Errno 2] No such file or directory: '/home/mjo/src/sage.git/local/var/lib/sage/scripts/sage_conf/spkg-requirements.txt'
Warning: pip exited with status 0
Removing stamp file '/home/mjo/src/sage.git/local/var/lib/sage/installed/sage_conf-10.2.rc4'

How do you provoke this error? I don't see it

[orlitzky]

How do you provoke this error? I don't see it

$ ./bootstrap
$ ./configure <my long list of configure flags>
$ make sage_conf  # install python deps to prevent pip fails
$ sage -sh
$ sage-spkg sage_conf  # reinstall
$ sage-spkg-uninstall sage_conf

The reinstall may not be necessary, but those were the steps I used.

[orlitzky]

... so perhaps when using --enable-system-site-packages, you would want to disable build isolation?

I think so. Is there an easy way to do this across the board? If not, it's not that big of a deal. The make that most people use outputs the error in black and white and is a lot less scary.

[orlitzky]

My ./configure flags can be found here:

https://gitweb.michael.orlitzky.com/?p=bash.d.git;a=blob;f=sage.sh;hb=HEAD

I've obviously got --enable-system-site-packages, but maybe --disable-editable is relevant too, who knows.

[mkoeppe]

Oh, that's #36737. I'll merge it here

[mkoeppe]

... so perhaps when using --enable-system-site-packages, you would want to disable build isolation?

I think so. Is there an easy way to do this across the board?

It all goes through build/bin/sage-dist-helpers (sdh_pip_install, sdh_pip_editable_install); for the first one, just set build_isolation_option to a different default for this case. (Separate PR)

[orlitzky]

Yep, that fixed it.

[orlitzky]

$$$$PATH 💪

[mkoeppe]

Thank you!

[vbraun]

merge conflict

[github-actions]

Documentation preview for this PR (built with commit 32d95d2; changes) is ready! 🎉