Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add spkg-src and tox.ini #29950

mkoeppe · 2020-06-24T02:12:57Z

As a follow-up to #29411, we set up a separate directory build/pkgs/sagelib/src with symlinks into SAGE_ROOT/src so that sagelib's setup.py does not have access to the whole tree SAGE_ROOT/src.

We fix setup.py sdist, which can be tested using spkg-src (which builds a pip-installable tarball in upstream/) and tox:

./sage -sh -c '(cd build/pkgs/sagelib/src && tox -v -v)'
./sage -sh -c 'build/pkgs/sagelib/src/.tox/python/bin/sage-ipython'

The call of ./sage -sh is necessary so that SAGE_LOCAL is set.
Moreover, for scripts that use sage-env, also SAGE_ENV_SOURCED is crucial for the correct operation; otherwise sage-env sets things based on SAGE_LOCAL again.

Depends on #29701

CC: @kiwifb @jhpalmieri @dimpase @kliem @isuruf @vbraun

Component: refactoring

Author: Matthias Koeppe

Branch/Commit: ccc67b0

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/29950

The text was updated successfully, but these errors were encountered:

mkoeppe · 2020-06-24T02:28:00Z

Branch: u/mkoeppe/build_sagelib_using_installed_sage_setup

sagetrac-git · 2020-06-24T02:32:23Z

Commit: 1ab75a9

sagetrac-git · 2020-06-24T02:32:23Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`0459006`	`.gitignore: Ignore generated files in build/pkgs/*/src`
`1ab75a9`	`Build sagelib using the installed sage_setup`

mkoeppe · 2020-06-24T02:47:50Z

Work Issues: add spkg-src

mkoeppe · 2020-06-24T05:04:05Z

Author: Matthias Koeppe

sagetrac-git · 2020-06-25T06:32:34Z

Branch pushed to git repo; I updated commit sha1. New commits:

`3d83c12`	`build/pkgs/sagelib/spkg-src: New`
`a2bde50`	`src/MANIFEST.in: prune .tox`
`4b258d3`	`src/MANIFEST.in: Remove bitrotted file`
`a29cd2d`	`build/pkgs/sagelib/MANIFEST.in: New`

sagetrac-git · 2020-06-25T06:32:34Z

Changed commit from 1ab75a9 to a29cd2d

mkoeppe · 2020-06-25T06:45:16Z

Changed work issues from add spkg-src to Fixup MANIFEST.in

mkoeppe · 2020-06-28T16:25:48Z

comment:8

sdist is broken: Files named ** end up in the tar file.

mkoeppe · 2020-06-28T16:26:49Z

Changed dependencies from #29411, #29847 to #29411, #29847, #29701

mkoeppe · 2020-06-28T18:18:26Z

comment:10

sagemath/cypari2#93

sagetrac-git · 2020-06-28T18:26:50Z

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

`7f8850a`	`src/module_list.py: Document that it is obsolete`
`26a85a4`	`Merge branch 't/29702/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup' into t/29701/replace_use_of_module_list_optionalextension`
`5d5803e`	`src/sage/graphs/planarity.pyx: Add forgotten distutils directive`
`a0be9b6`	`Merge branch 't/29790/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_5__sage_graphs_' into t/29701/replace_use_of_module_list_optionalextension`
`14396af`	`Merge tag '9.2.beta2' into t/29701/replace_use_of_module_list_optionalextension`
`821e337`	`Merge branch 't/29701/replace_use_of_module_list_optionalextension' into t/29950/build_sagelib_using_installed_sage_setup`
`eeb6ba4`	`build/pkgs/sagelib/src/requirements.txt, tox.ini: New`
`ce8b84d`	`build/pkgs/sagelib/src/MANIFEST.in: Prune sage/ext/interpreters`
`a9a1335`	`build/pkgs/sagelib/src/requirements.txt: Add packages required for runtime`
`c3b8f8f`	`Suggest tox -v -v`

sagetrac-git · 2020-06-28T18:26:50Z

Changed commit from a29cd2d to c3b8f8f

mkoeppe · 2020-06-28T18:29:11Z

Changed work issues from Fixup MANIFEST.in to Rebase away from #29847

sagetrac-git · 2020-06-28T19:04:05Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`cdf501f`	`build/pkgs/sagelib/spkg-src: New`
`ef4f5b8`	`Move src/MANIFEST.in to build/pkgs/sagelib/MANIFEST.in, fix up`
`233f6ba`	`build/pkgs/sagelib/src/requirements.txt, tox.ini: New`
`177cd4c`	`build/pkgs/sagelib/src/MANIFEST.in: Prune sage/ext/interpreters`
`07b29fb`	`build/pkgs/sagelib/src/requirements.txt: Add packages required for runtime`
`ad33705`	`Suggest tox -v -v`
`481e212`	`.gitignore: Ignore generated files in build/pkgs/*/src`
`9955a3f`	`Build sagelib from build/pkgs/sagelib/src`

sagetrac-git · 2020-06-28T19:04:05Z

Changed commit from c3b8f8f to 9955a3f

mkoeppe · 2020-06-28T19:04:51Z

Changed dependencies from #29411, #29847, #29701 to #29411, #29701

mkoeppe · 2020-06-28T19:04:51Z

Changed work issues from Rebase away from #29847 to none

mkoeppe · 2020-06-28T19:07:03Z

comment:15

    from .maxima import maxima, Maxima
  File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/build/pkgs/sagelib/src/.tox/python/lib/python3.7/site-packages/sage/interfaces/maxima.py", line 1236, in <module>
    script_subdirectory=None)
  File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/build/pkgs/sagelib/src/.tox/python/lib/python3.7/site-packages/sage/interfaces/maxima.py", line 528, in __init__
    raise RuntimeError('You must get the file local/bin/sage-maxima.lisp')
RuntimeError: You must get the file local/bin/sage-maxima.lisp

mkoeppe · 2020-06-28T19:14:35Z

Changed dependencies from #29411, #29701 to #29411, #29701, #27171

mkoeppe · 2020-06-28T19:18:51Z

Changed dependencies from #29411, #29701, #27171 to #29411, #29701, #21559

sagetrac-git · 2020-06-28T19:19:31Z

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

`875940d`	`trac 29111: more tinkering. Fix doctests in tests/cmdline.py.`
`4a3d36e`	`Move 'sage -app' back to src/bin/sage`
`3a0193c`	`src/bin/sage: Remove handling of 'sage -axiom'`
`6b04075`	`Merge branch 't/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution' into t/21559/change-src-bin-installation`
`9c7116b`	`src/bin/sage-list-optional, sage-list-experimental, sage-list-standard: Remove deprecated scripts`
`831cc09`	`Merge branch 't/29920/remove_deprecated_scripts_sage_list_optional__sage_list_experimental__sage_list_standard' into t/21559/change-src-bin-installation`
`a56dc35`	`Merge tag '9.2.beta1' into t/29702/public/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup`
`e3eca85`	`Merge branch 'public/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup' of git://trac.sagemath.org/sage into t/21559/change-src-bin-installation`
`7d29141`	`src/setup.py: Do not install removed script sage-rsyncdist`
`08f5175`	`Merge branch 'u/mkoeppe/change-src-bin-installation' of git://trac.sagemath.org/sage into t/29950/build_sagelib_using_installed_sage_setup`

sagetrac-git · 2020-06-28T19:19:31Z

Changed commit from 9955a3f to 08f5175

sagetrac-git · 2020-06-29T00:11:18Z

Changed commit from eaced09 to b8e366e

mkoeppe · 2020-07-02T20:14:58Z

comment:31

All dependencies of this ticket are green. Ready for review.

sagetrac-git · 2020-07-04T16:30:44Z

Branch pushed to git repo; I updated commit sha1. New commits:

`c5f7f2f`	`Merge tag '9.2.beta3' into t/21559/change-src-bin-installation`
`0473ef3`	`Merge branch 't/21559/change-src-bin-installation' into t/29950/build_sagelib_using_installed_sage_setup`
`06a3609`	`sage_setup.find.find_python_sources: Add benchmark doctest`
`174626c`	`Merge branch 't/29786/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_4__sage_rings_' into t/29701/replace_use_of_module_list_optionalextension`
`034a7f7`	`Merge branch 'public/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup' of git://trac.sagemath.org/sage into t/29701/replace_use_of_module_list_optionalextension`
`d3c608b`	`Merge tag '9.2.beta3' into t/29701/replace_use_of_module_list_optionalextension`
`7244371`	`Merge branch 't/29701/replace_use_of_module_list_optionalextension' into t/29950/build_sagelib_using_installed_sage_setup`

sagetrac-git · 2020-07-04T16:30:44Z

Changed commit from b8e366e to 7244371

mkoeppe · 2020-07-04T16:31:17Z

comment:33

Merged in latest versions #29701, #21559 (on top of 9.2.beta3). Needs review

sagetrac-git · 2020-07-08T23:26:36Z

Changed commit from 7244371 to 4344f89

sagetrac-git · 2020-07-08T23:26:36Z

Branch pushed to git repo; I updated commit sha1. New commits:

`3953671`	`Merge tag '9.2.beta4' into t/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution`
`19d7f67`	`Merge branch 't/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution' into t/21559/change-src-bin-installation`
`4344f89`	`Merge branch 't/21559/change-src-bin-installation' into t/29950/build_sagelib_using_installed_sage_setup`

sagetrac-git · 2020-08-03T00:53:28Z

Changed commit from 4344f89 to 01b96b0

sagetrac-git · 2020-08-03T00:53:28Z

Branch pushed to git repo; I updated commit sha1. New commits:

`fcad518`	`Merge tag '9.2.beta7' into t/29701/replace_use_of_module_list_optionalextension`
`55c3fbc`	`src/sage_setup/clean.py: Fix doctest`
`01b96b0`	`Merge branch 't/29701/replace_use_of_module_list_optionalextension' into t/29950/build_sagelib_using_installed_sage_setup`

mkoeppe · 2020-08-03T00:54:01Z

Changed dependencies from #29411, #29701, #21559 to #29701

dimpase · 2020-08-05T08:22:44Z

comment:37

lgtm

dimpase · 2020-08-05T08:22:44Z

Reviewer: Dima Pasechnik

mkoeppe · 2020-08-08T20:50:05Z

comment:38

Failing doctest needs work as noted in #29701.

sagetrac-git · 2020-08-08T22:50:02Z

Branch pushed to git repo; I updated commit sha1. New commits:

`8a19fe2`	`build/make/Makefile.in (sagelib-clean): Clean the new build location`

sagetrac-git · 2020-08-08T22:50:02Z

Changed commit from 01b96b0 to 8a19fe2

sagetrac-git · 2020-08-08T23:08:03Z

Branch pushed to git repo; I updated commit sha1. New commits:

`ccc67b0`	`src/sage_setup: Update cythonized_dir in doctests`

sagetrac-git · 2020-08-08T23:08:03Z

Changed commit from 8a19fe2 to ccc67b0

dimpase · 2020-08-10T16:45:25Z

comment:42

ok

mkoeppe · 2020-08-10T16:51:20Z

comment:43

Thanks!

vbraun · 2020-08-12T19:53:18Z

Changed branch from u/mkoeppe/build_sagelib_using_installed_sage_setup to ccc67b0

mkoeppe added this to the sage-9.2 milestone Jun 24, 2020

mkoeppe added t: refactoring labels Jun 24, 2020

mkoeppe changed the title ~~Build sagelib using the installed sage_setup~~ Build sagelib using the installed sage_setup, add spkg-src Jun 24, 2020

This comment has been minimized.

Sign in to view

mkoeppe changed the title ~~Build sagelib using the installed sage_setup, add spkg-src~~ Build sagelib from build/pkgs/sagelib/src, fix setup.py sdist, add spkg-src and tox.ini Jun 28, 2020

dimpase added s: positive review and removed s: needs review labels Aug 5, 2020

mkoeppe added s: needs work and removed s: positive review labels Aug 8, 2020

mkoeppe added s: needs review and removed s: needs work labels Aug 8, 2020

dimpase added s: positive review and removed s: needs review labels Aug 10, 2020

vbraun removed the s: positive review label Aug 12, 2020

vbraun closed this as completed in eaa2f0a Aug 12, 2020

Build sagelib from build/pkgs/sagelib/src, fix setup.py sdist, add spkg-src and tox.ini #29950

Build sagelib from build/pkgs/sagelib/src, fix setup.py sdist, add spkg-src and tox.ini #29950

Comments

mkoeppe commented Jun 24, 2020

mkoeppe commented Jun 24, 2020

sagetrac-git mannequin commented Jun 24, 2020

sagetrac-git mannequin commented Jun 24, 2020

mkoeppe commented Jun 24, 2020

mkoeppe commented Jun 24, 2020

sagetrac-git mannequin commented Jun 25, 2020

sagetrac-git mannequin commented Jun 25, 2020

mkoeppe commented Jun 25, 2020

mkoeppe commented Jun 28, 2020

mkoeppe commented Jun 28, 2020

mkoeppe commented Jun 28, 2020

sagetrac-git mannequin commented Jun 28, 2020

sagetrac-git mannequin commented Jun 28, 2020

This comment has been minimized.

mkoeppe commented Jun 28, 2020

sagetrac-git mannequin commented Jun 28, 2020

sagetrac-git mannequin commented Jun 28, 2020

mkoeppe commented Jun 28, 2020

mkoeppe commented Jun 28, 2020

mkoeppe commented Jun 28, 2020

mkoeppe commented Jun 28, 2020

mkoeppe commented Jun 28, 2020

sagetrac-git mannequin commented Jun 28, 2020

sagetrac-git mannequin commented Jun 28, 2020

sagetrac-git mannequin commented Jun 29, 2020

mkoeppe commented Jul 2, 2020

sagetrac-git mannequin commented Jul 4, 2020

sagetrac-git mannequin commented Jul 4, 2020

mkoeppe commented Jul 4, 2020

sagetrac-git mannequin commented Jul 8, 2020

sagetrac-git mannequin commented Jul 8, 2020

sagetrac-git mannequin commented Aug 3, 2020

sagetrac-git mannequin commented Aug 3, 2020

mkoeppe commented Aug 3, 2020

dimpase commented Aug 5, 2020

dimpase commented Aug 5, 2020

mkoeppe commented Aug 8, 2020

sagetrac-git mannequin commented Aug 8, 2020

sagetrac-git mannequin commented Aug 8, 2020

sagetrac-git mannequin commented Aug 8, 2020

sagetrac-git mannequin commented Aug 8, 2020

dimpase commented Aug 10, 2020

mkoeppe commented Aug 10, 2020

vbraun commented Aug 12, 2020

Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add spkg-src and tox.ini #29950

Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add spkg-src and tox.ini #29950