sagemathgh-37447: Replace bootstrap-conda by grayskull and update con…

…da lock files

    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

Using the metadata added in sagemath#37446,
we automatically generate the conda environment files. This no longer
makes any reference to the `conda.txt` files contained in sage-the-
distribution. Thus sage-on-top-of-conda is now completely independent of
sage-the-distribution (only relying on information specified by sage-
the-library). In particular, after this PR is merged the `conda.txt`
files could be deleted from sage-the-distribution.

In particular, we no longer need to maintain the mapping of pypi
packages to conda packages but instead can rely on the offical mappings
maintained by the conda team (https://github.com/regro/cf-graph-
countyfair/tree/master/mappings/pypi).

To test:
```
pip install grayskull conda-lock
python tools/update-conda.py
```

The updated conda files are committed here as well.

Moreover, the `environment-dev` files have been deleted as there was
only little difference between these files and some people rightfully
complained that having too many files in the root folder is distracting.

As a byproduct, this fixes sagemath#34626.

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [ ] The title is concise, informative, and self-explanatory.
- [ ] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

- sagemath#37446: specifies additional
requirements in the pyproject.toml which are used to generate the conda
env files
- sagemath#38983: for the update of numpy
- sagemath#38982: to fix meson build

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#37447
Reported by: Tobias Diez
Reviewer(s): Dima Pasechnik, Julian Rüth, Tobias Diez

.ci/write-dockerfile.sh

@@ -275,7 +275,7 @@ cat <<EOF
 FROM with-system-packages AS bootstrapped
 #:bootstrapping:
 RUN rm -rf /new /sage/.git
-$ADD Makefile VERSION.txt COPYING.txt condarc.yml README.md bootstrap bootstrap-conda configure.ac sage .homebrew-build-env tox.ini .gitignore /new/
+$ADD Makefile VERSION.txt COPYING.txt condarc.yml README.md bootstrap configure.ac sage .homebrew-build-env tox.ini .gitignore /new/
 $ADD config/config.rpath /new/config/config.rpath
 $ADD src/doc/bootstrap /new/src/doc/bootstrap
 $ADD src/bin /new/src/bin

.devcontainer/onCreate-conda.sh

@@ -6,7 +6,7 @@ conda config --env --add channels conda-forge
 conda config --env --set channel_priority strict
 conda update -y --all --override-channels -c conda-forge
 conda install mamba=1 -n base -y
-mamba env create -y --file environment-dev-3.11-linux.yml || mamba env update -y --file environment-dev-3.11-linux.yml
+mamba env create -y --file environment-3.11-linux.yml || mamba env update -y --file environment-3.11-linux.yml
 conda init bash
 
 # Build sage

.github/workflows/ci-conda.yml

@@ -69,7 +69,7 @@ jobs:
           use-mamba: true
           channels: conda-forge
           channel-priority: true
-          activate-environment: sage
+          activate-environment: sage-dev
           environment-file: ${{ matrix.conda-env }}-${{ matrix.python }}-${{ startsWith(matrix.os, 'macos') && (startsWith(runner.arch, 'ARM') && 'macos' || 'macos-x86_64') || 'linux' }}.yml
 
       - name: Print Conda environment

.github/workflows/ci-meson.yml

@@ -54,7 +54,7 @@ jobs:
           use-mamba: true
           channels: conda-forge
           channel-priority: true
-          activate-environment: sage
+          activate-environment: sage-dev
           environment-file: environment-${{ matrix.python }}-${{ startsWith(matrix.os, 'macos') && (startsWith(runner.arch, 'ARM') && 'macos' || 'macos-x86_64') || 'linux' }}.yml
 
       - name: Print Conda environment

.gitignore

@@ -27,8 +27,8 @@
 # no longer generated, but may still be in user worktrees
 /src/lib/pkgconfig
 
-# Environment files generated by bootstrap-conda.
-# The files without Python version and in src are no longer generated
+# Conda environment files
+# The files without Python version, with -dev or in src are no longer generated
 # but may still be in users' directories.
 /environment.yml
 /environment-3.9.yml

.gitpod.yml

@@ -7,7 +7,7 @@ tasks:
   - name: Setup
     # Create conda environment, then configure and build sage
     init: >-
-      && mamba env create --file environment-dev-3.11-linux.yml --prefix venv
+      && mamba env create --file environment-3.11-linux.yml --prefix venv
       && conda config --append envs_dirs $(pwd)
       && conda activate $(pwd)/venv
       && ./bootstrap

Makefile

@@ -174,7 +174,7 @@ bootstrap-clean:
 	rm -rf config/install-sh config/compile config/config.guess config/config.sub config/missing configure build/make/Makefile-auto.in
 	rm -f src/doc/en/installation/*.txt
 	find src/doc/en/reference/spkg -name index.rst -prune -o -maxdepth 1 -name "*.rst" -exec rm -f {} \+
-	for a in environment environment-optional src/environment src/environment-dev src/environment-optional; do rm -f $$a.yml $$a-3.[89].yml $$a-3.1[0-9].yml; done
+	for a in environment environment-optional src/environment src/environment-optional; do rm -f $$a.yml $$a-3.[89].yml $$a-3.1[0-9].yml; done
 	rm -f src/requirements.txt
 	rm -f src/setup.cfg
 	rm -f build/pkgs/cypari/version_requirements.txt

bootstrap

@@ -139,7 +139,6 @@ EOF
     # ONLY stderr, and to re-output the results back to stderr leaving
     # stdout alone. Basically we swap the two descriptors using a
     # third, filter, and then swap them back.
-    ./bootstrap-conda && \
     aclocal -I m4 && \
     automake --add-missing --copy build/make/Makefile-auto 3>&1 1>&2 2>&3 \
         | sed "${QUIET_SED_FILTER}" 3>&1 1>&2 2>&3 && \
@@ -226,7 +225,7 @@ save () {
 	build/make/Makefile-auto.in \
 	src/doc/en/installation/*.txt \
 	$(find src/doc/en/reference/spkg -name index.rst -prune -o -maxdepth 1 -name "*.rst" -print) \
-	environment-3.[89].yml environment-3.1[0-9].yml \
+	environment-3.[89]-*.yml environment-3.1[0-9]-*.yml \
 	src/pyproject.toml \
 	src/requirements.txt \
 	src/setup.cfg \
@@ -236,6 +235,7 @@ save () {
 	build/pkgs/gmpy2/version_requirements.txt \
 	build/pkgs/jupyter_core/version_requirements.txt \
 	build/pkgs/memory_allocator/version_requirements.txt \
+        build/pkgs/meson/version_requirements.txt \
 	build/pkgs/numpy/version_requirements.txt \
 	build/pkgs/pkgconfig/version_requirements.txt \
 	build/pkgs/pplpy/version_requirements.txt \