Use the virtual environment to generate the sdist

[dstufft]

Currently it appears that tox uses the system Python when creating the sdist (specifically, /usr/bin/python if that's where tox is installed). This means that if you're trying to use a setuptools feature that is newer than what your system provides in your setup.py, then you're bound by what your current environment provides (which is often times fairly old). I think it would be a great idea if tox re-used one of the virtual environments it's creating to build the sdist (honestly it could be any of them) which would give more control over what version of setuptools is being used.

Additionally, I think this would also affect someone trying to use tox with a package whose setup.py only runs on Python 3.

For reference, here is the output I'm seeing that suggests this is what is happening:

+ tox -r -e docs -- --color=yes
GLOB sdist-make: /var/lib/jenkins/jobs/cryptography-pr-docs/workspace/cryptography/setup.py
ERROR: invocation failed (exit code 1), logfile: /var/lib/jenkins/jobs/cryptography-pr-docs/workspace/cryptography/.tox/log/tox-0.log
ERROR: actionid: tox
msg: packaging
cmdargs: ['/usr/bin/python', local('/var/lib/jenkins/jobs/cryptography-pr-docs/workspace/cryptography/setup.py'), 'sdist', '--formats=zip', '--dist-dir', local('/var/lib/jenkins/jobs/cryptography-pr-docs/workspace/cryptography/.tox/dist')]
env: None

Traceback (most recent call last):
  File "setup.py", line 332, in <module>
    **keywords_with_side_effects(sys.argv)
  File "/usr/lib/python2.7/distutils/core.py", line 111, in setup
    _setup_distribution = dist = klass(attrs)
  File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 239, in __init__
    self.fetch_build_eggs(attrs.pop('setup_requires'))
  File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 264, in fetch_build_eggs
    replace_conflicting=True
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 595, in resolve
    requirements = list(requirements)[::-1]  # set up the stack
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2605, in parse_requirements
    line, p, specs = scan_list(VERSION,LINE_END,line,p,(1,2),"version spec")
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2583, in scan_list
    "Expected ',' or end-of-list in",line,"at",line[p:]
ValueError: ("Expected ',' or end-of-list in", "cffi>=1.4.1 ; platform_python_implementation != 'PyPy'", 'at', " ; platform_python_implementation != 'PyPy'")

ERROR: FAIL could not package project - v = InvocationError('/usr/bin/python /var/lib/jenkins/jobs/cryptography-pr-docs/workspace/cryptography/setup.py sdist --formats=zip --dist-dir /var/lib/jenkins/jobs/cryptography-pr-docs/workspace/cryptography/.tox/dist (see /var/lib/jenkins/jobs/cryptography-pr-docs/workspace/cryptography/.tox/log/tox-0.log)', 1)

[warner]

Additionally, I think this would also affect someone trying to use tox with a package whose setup.py only runs on Python 3.

I can confirm this. Tahoe-LAFS is python2-only. We recently changed the tahoe-lafs setup.py to throw a clean exception if loaded with python3 (because the alternative was for zfec, our dependency, to fail with an inscrutable SyntaxError because of a py2-only print "foo" statement). When people accidentally try to install or build tahoe with py3, we want to teach them to use py2, instead of having them wonder why a perfectly valid-looking print statement in an apparently-unrelated package is being treated as a syntax error.

Our tox.ini calls for python2.7, and on platforms where tox uses python2, that works fine. But on most recent Ubuntu platforms, the system /usr/bin/tox is py3-based, and those platforms can no longer run the tests with our recommended instructions ("git clone REPOURL; cd tahoe-lafs; tox").

We don't want to tell users that they must switch to a py2-based tox first (if they knew how to build a py2 virtualenv and install tox into it, they would know enough to just install tahoe into that virtualenv and run trial directly). So we'd really like it if a py3-based tox were still able to run our tests.

Our workaround is to change the tahoe setup.py to only throw the not-on-py3 exception when sys.argv[1] is "install" or "egg-info" (it appears that tox only runs setup.py sdist under the wrong python). But we'd much prefer to see tox use the target python when building the sdist for a target, since there are ways that users could mistakenly use python3 to build tahoe that might bypass the instructional error message we're trying to show them.

(for reference, the tahoe bug is at https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2876)

[zozzz]

And set environment variables that defined in setenv or passenv

[gaborbernat]

So the solution could be to regenerate the package creation for each and every environment. setenv and passenv are set once loaded into the virtualenv, so that would be relatively already provided

[obestwalter]

I think we only should build the package under test once, but the Python building that package needs to fullfill the needs of the setup.py (e.g. needs to be Python 3.whatever). The idea from @gaborbernat would solve the problem of knowing what env to use, but would introduce other problems (after all, we want to build one package that we want to test, not several and then some of those envs might not even be able to build the package).

Maybe we need to introduce a venv that is only there to build the package in cases where this is necessary? So as a default, package is built by the interpreter that tox is installed in, but if you need another interpreter you can ask for it as a global config option?

[gaborbernat]

@dstufft this now is resolved with pep 517 implementation released with 3.3..0