Move SAGE_ROOT/build/pkgs//src to SAGE_ROOT/pkgs/ #31577

mkoeppe · 2021-03-29T04:50:47Z

In the current layout, these embedded source trees are a little difficult to find / complicated to type.

The proposed location is:

SAGE_ROOT
- pkgs
  - sage-conf
    - setup.py
    - bin/
    - sage_conf.py.in
  - sage-docbuild
    - setup.py
    - sage_docbuild   -> symlinks to SAGE_ROOT/sage_docbuild/
  - sage-sws2rst
    - setup.py
    - bin/
    - sage_sws2rst/
  - sagemath-standard
    - setup.py
    - bin             -> symlinks to SAGE_ROOT/src/bin/        
    - sage            -> symlinks to SAGE_ROOT/src/sage/
    - sage_setup      -> symlinks to SAGE_ROOT/src/sage_setup/ (removed in #29847)

that is, the new directory pkgs will be on the same level as

SAGE_ROOT
- src

the unchanged monolithic sagelib source tree, which will continue to contain subdirectories

  - bin/ 
  - doc/
  - sage/
  - sage_docbuild/      # was sage_setup/docbuild/ in Sage 9.2
  - sage_setup/

We add symlinks from the previous locations SAGE_ROOT/build/pkgs/sage[math]_*/src to the new locations SAGE_ROOT/pkgs/sage[math]-*

#29847 will add:

SAGE_ROOT
- pkgs
  - sage-setup
    - setup.py
    - sage_setup         -> symlinks to src/sage_setup/

and remove sage_setup from the sagemath-standard distribution.

Other modularization tickets (#29705) will add

SAGE_ROOT
- pkgs
  - sagemath-core/
  - sagemath-brial/
  - sagemath-giac/
  - sagemath-meataxe/
  - sagemath-tdlib/

etc.

#29868 will add

SAGE_ROOT
- pkgs
  - sagemath-doc-html/
  - sagemath-doc-pdf/

All files that contain Sage doctests will remain in the monolithic src/ source tree; the source trees of the distributions symlink there. This may be changed in a follow-up ticket regarding the modularization of doctesting.

Before merging this branch into a non-distclean worktree, it is useful to use

  git clean -f -x build/pkgs/*/src

CC: @jhpalmieri @dimpase @kiwifb

Component: build

Author: Matthias Koeppe

Branch/Commit: 7568dc6

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/31577

The text was updated successfully, but these errors were encountered:

mkoeppe · 2021-03-29T15:33:28Z

comment:1

Alternatively, these directories could just be put into SAGE_ROOT:

SAGE_ROOT
 - sage_conf
 - sage_docbuild
 - sage_sws2rst
 - sagemath_standard

mkoeppe · 2021-03-30T23:07:59Z

comment:2

(made this change in the ticket description)

mkoeppe · 2021-04-12T04:13:35Z

Dependencies: #30913

mkoeppe · 2021-04-12T04:39:13Z

Branch: u/mkoeppe/move_sage_root_build_pkgs___src_to_sage_root____rename_sage_root_build_to_sage_root_sage_bootstrap

mkoeppe · 2021-04-12T04:39:42Z

Last 10 new commits:

`0283da5`	`build/make/Makefile.in: Add wheel, setuptools_wheel to PYTHON_TOOLCHAIN to make sure that PEP 517 packages have a complete build system`
`f720722`	`build/pkgs/sagelib/src/tox.ini: Add factor nobuildisolation`
`c451b31`	`src/setup.cfg.m4 (install_requires): Add sage_conf`
`6700223`	`Merge tag '9.3.rc0' into t/30913/sagelib__add_setup_cfg__install_requires_`
`04da2c6`	`build/pkgs/ipywidgets: Patch out declaring install-requires of nbformat and jupyterlab-widgets`
`68dc845`	`Merge branch 't/30913/sagelib__add_setup_cfg__install_requires_' into t/31577/move_sage_root_build_pkgs___src_to_sage_root____rename_sage_root_build_to_sage_root_sage_bootstrap`
`9181571`	`Move /build/pkgs/sage_conf/src to /sage_conf, leave symlink behind`
`4f480b4`	`Move /build/pkgs/sage_docbuild/src to /sage_docbuild, leave symlink behind`
`cc04151`	`Move /build/pkgs/sage_sws2rst/src to /sage_sws2rst, leave symlink behind`
`8862b55`	`Move /build/pkgs/sagelib/src to /sagemath_standard, leave symlink behind`

mkoeppe · 2021-04-12T04:39:42Z

Commit: 8862b55

mkoeppe · 2021-04-12T04:39:42Z

Author: Matthias Koeppe

sagetrac-git · 2021-04-12T04:48:04Z

Changed commit from 8862b55 to 73d83c2

sagetrac-git · 2021-04-12T04:48:04Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`73d83c2`	`Move /build/pkgs/sagelib/src to /sagemath_standard, leave symlink behind`

sagetrac-git · 2021-04-12T04:54:52Z

Branch pushed to git repo; I updated commit sha1. New commits:

`223e2ba`	`configure.ac: Use shorter path to sage_conf/`
`9bbb57b`	`Fixup: Add build/pkgs/sage_sws2rst/src symlink`
`ee0a21e`	`Fixup: Add build/pkgs/sagelib/src symlink`
`6ff4ba1`	`.gitignore: Update for new source tree locations`

sagetrac-git · 2021-04-12T04:54:52Z

Changed commit from 73d83c2 to 6ff4ba1

sagetrac-git · 2021-04-12T05:47:57Z

Changed commit from 6ff4ba1 to ef2fee8

sagetrac-git · 2021-04-12T05:47:57Z

Branch pushed to git repo; I updated commit sha1. New commits:

`ef2fee8`	`WIP: Fix up docker build`

jhpalmieri · 2021-04-12T18:33:57Z

comment:13

sage_docbuild   -> symlinks to SAGE_ROOT/src/sage_docbuild

Would it make sense to reverse this: put the files in SAGE_ROOT/sage_docbuild and then make SAGE_ROOT/src/sage_docbuild into a link? Is there any reason to keep SAGE_ROOT/src/sage_docbuild? If so, why also create the top-level directory?

What about SAGE_ROOT/src/sage_setup, along the same lines?

mkoeppe · 2021-04-12T18:50:35Z

comment:14

This is a great question.

There is no technical reason for keeping SAGE_ROOT/src/sage_docbuild at all.
Likewise, after #29847, there will be no technical reason for keeping SAGE_ROOT/src/sage_setup.

But in #29705, I promised not to change the structure of the source tree, which I interpret as keeping the monolithic tree SAGE_ROOT/src unchanged. It is so that we don't overwhelm the developer community by making too many changes at the same time.

Of course, it could be argued that sage_docbuild is a new name anyway, and very few people will know or care about this package.

So if you think that's better, I can just move the sage_docbuild sources to SAGE_ROOT/sage_docbuild/sage_docbuild on this ticket.

(I'd rather not put reverse symlinks.)

sagetrac-git · 2021-04-12T19:11:58Z

Branch pushed to git repo; I updated commit sha1. New commits:

`12df870`	`No more src/sage_docbuild`

sagetrac-git · 2021-04-12T19:11:58Z

Changed commit from ef2fee8 to 12df870

sagetrac-git · 2021-06-25T01:48:40Z

Branch pushed to git repo; I updated commit sha1. New commits:

`380b42d`	`Update paths in comments`
`a7d6a3e`	`build/bin/write-dockerfile.sh: ADD pkgs`

dimpase · 2021-06-25T19:40:45Z

Reviewer: Dima Pasechnik

dimpase · 2021-06-25T19:40:45Z

comment:81

Looks good.

mkoeppe · 2021-06-25T19:44:40Z

comment:82

Thanks!

vbraun · 2021-06-29T17:56:17Z

comment:83

Merge conflict

mkoeppe · 2021-06-29T21:42:44Z

comment:84

Merges cleanly with af326a8 (vbraun/develop)

vbraun · 2021-06-29T23:10:43Z

comment:85

then wait for the next beta

sagetrac-git · 2021-07-01T22:05:16Z

Changed commit from a7d6a3e to 7568dc6

sagetrac-git · 2021-07-01T22:05:16Z

Branch pushed to git repo; I updated commit sha1. New commits:

`7568dc6`	`Merge tag '9.4.beta4' into t/31577/move_sage_root_build_pkgs___src_to_sage_root____rename_sage_root_build_to_sage_root_sage_bootstrap`

vbraun · 2021-07-06T21:50:00Z

comment:88

whats going on here?

    STDOUT: CONFLICT (directory/file): There is a directory with name build/pkgs/sagelib/src in HEAD. Adding build/pkgs/sagelib/src as build/pkgs/sagelib/src~7568dc6dc7a2325525aaa4618ec1bc0bc417917b
    STDOUT: Removing build/pkgs/sage_sws2rst/src/sage_sws2rst/__init__.py
    STDOUT: Adding build/pkgs/sage_sws2rst/src
    STDOUT: Removing build/pkgs/sage_docbuild/src/sage_docbuild
    STDOUT: Removing build/pkgs/sage_docbuild/src/VERSION.txt
    STDOUT: CONFLICT (directory/file): There is a directory with name build/pkgs/sage_docbuild/src in HEAD. Adding build/pkgs/sage_docbuild/src as build/pkgs/sage_docbuild/src~7568dc6dc7a2325525aaa4618ec1bc0bc417917b
    STDOUT: Removing build/pkgs/sage_conf/src/bin/sage-env-config
    STDOUT: Removing build/pkgs/sage_conf/src/README.rst
    STDOUT: CONFLICT (directory/file): There is a directory with name build/pkgs/sage_conf/src in HEAD. Adding build/pkgs/sage_conf/src as build/pkgs/sage_conf/src~7568dc6dc7a2325525aaa4618ec1bc0bc417917b
    STDOUT: Auto-merging build/bin/write-dockerfile.sh
    STDOUT: Automatic merge failed; fix conflicts and then commit the result.
[release@zen Sage]$ git --version
git version 2.31.1

vbraun · 2021-07-06T21:51:39Z

comment:89

ok those were previously not-checked in build artifacts

vbraun · 2021-07-09T20:23:50Z

Changed branch from u/mkoeppe/move_sage_root_build_pkgs___src_to_sage_root____rename_sage_root_build_to_sage_root_sage_bootstrap to 7568dc6

mkoeppe added this to the sage-9.4 milestone Mar 29, 2021

mkoeppe added c: build labels Mar 29, 2021

This comment has been minimized.

Sign in to view

mkoeppe changed the title ~~Move SAGE_ROOT/build/pkgs/{sagelib,sage_conf,sage_sws2rst,sage_docbuild}/src to SAGE_ROOT/src/pkgs/...~~ Move SAGE_ROOT/build/pkgs/*/src to SAGE_ROOT/* Mar 30, 2021

This comment has been minimized.

Sign in to view

mkoeppe changed the title Move SAGE_ROOT/build/pkgs/*/src to SAGE_ROOT/* Move SAGE_ROOT/build/pkgs/*/src to SAGE_ROOT/*, rename SAGE_ROOT/build to SAGE_ROOT/sage_bootstrap Apr 10, 2021

This comment has been minimized.

Sign in to view

mkoeppe changed the title ~~Move SAGE_ROOT/build/pkgs/*/src to SAGE_ROOT/*, rename SAGE_ROOT/build to SAGE_ROOT/sage_bootstrap~~ Move SAGE_ROOT/build/pkgs/*/src to SAGE_ROOT/* Apr 12, 2021

mkoeppe added the s: needs review label Apr 12, 2021

dimpase added s: positive review and removed s: needs review labels Jun 25, 2021

vbraun added s: needs work and removed s: positive review labels Jun 29, 2021

mkoeppe added s: positive review and removed s: needs work labels Jun 29, 2021

vbraun added s: needs work and removed s: positive review labels Jun 29, 2021

mkoeppe added s: positive review and removed s: needs work labels Jul 1, 2021

vbraun added s: needs work and removed s: positive review labels Jul 6, 2021

vbraun added s: positive review and removed s: needs work labels Jul 6, 2021

This comment has been minimized.

Sign in to view

vbraun removed the s: positive review label Jul 9, 2021

vbraun closed this as completed in 912b83d Jul 9, 2021

Move SAGE_ROOT/build/pkgs/*/src to SAGE_ROOT/pkgs/* #31577

Move SAGE_ROOT/build/pkgs/*/src to SAGE_ROOT/pkgs/* #31577

Comments

mkoeppe commented Mar 29, 2021

mkoeppe commented Mar 29, 2021

mkoeppe commented Mar 30, 2021

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

mkoeppe commented Apr 12, 2021

mkoeppe commented Apr 12, 2021

mkoeppe commented Apr 12, 2021

mkoeppe commented Apr 12, 2021

mkoeppe commented Apr 12, 2021

sagetrac-git mannequin commented Apr 12, 2021

sagetrac-git mannequin commented Apr 12, 2021

sagetrac-git mannequin commented Apr 12, 2021

sagetrac-git mannequin commented Apr 12, 2021

sagetrac-git mannequin commented Apr 12, 2021

sagetrac-git mannequin commented Apr 12, 2021

jhpalmieri commented Apr 12, 2021

mkoeppe commented Apr 12, 2021

sagetrac-git mannequin commented Apr 12, 2021

sagetrac-git mannequin commented Apr 12, 2021

sagetrac-git mannequin commented Jun 25, 2021

dimpase commented Jun 25, 2021

dimpase commented Jun 25, 2021

mkoeppe commented Jun 25, 2021

vbraun commented Jun 29, 2021

mkoeppe commented Jun 29, 2021

vbraun commented Jun 29, 2021

sagetrac-git mannequin commented Jul 1, 2021

sagetrac-git mannequin commented Jul 1, 2021

vbraun commented Jul 6, 2021

vbraun commented Jul 6, 2021

This comment has been minimized.

This comment has been minimized.

vbraun commented Jul 9, 2021

Move SAGE_ROOT/build/pkgs//src to SAGE_ROOT/pkgs/ #31577

Move SAGE_ROOT/build/pkgs//src to SAGE_ROOT/pkgs/ #31577