Clean sdist apparently missing egg-info?

[lvh]

This issue started on a different ticket.

check-manifest 0.21 fails on lvh/jorji:

listing source files under version control: 12 files and directories
building an sdist: jorji-0.1.2.4.g7fded09.tar.gz: 16 files and directories
copying source files to a temporary directory
building a clean sdist
['/usr/local/opt/python/bin/python2.7', 'setup.py', 'sdist', '-d', '/var/folders/dd/lh02zlnn1hg7cgs85w8qq6c00000gn/T/check-manifest-ER_Wa7-sdist'] failed (status 1):
error in setup command: Error parsing /private/var/folders/dd/lh02zlnn1hg7cgs85w8qq6c00000gn/T/check-manifest-sscwne-sources/setup.cfg: Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. Are you sure that git is installed?

This is quite strange, because manually creating an sdist, unpacking it, and then creating and sdist within the unpacked sdist works totally fine:

[jorji] python setup.py sdist && mv dist/jorji-0.1.2.4.g7fded09.tar.gz ~/tmp && cd ~/tmp && tar -xvpzf jorji-0.1.2.4.g7fded09.tar.gz && cd jorji-0.1.2.4.g7fded09             10:03:46  ☁  master ☂ 𝝙 ⚡ ✭
running sdist
[pbr] Writing ChangeLog
[pbr] Generating AUTHORS
running egg_info
writing requirements to jorji.egg-info/requires.txt
writing jorji.egg-info/PKG-INFO
writing top-level names to jorji.egg-info/top_level.txt
writing dependency_links to jorji.egg-info/dependency_links.txt
[pbr] Processing SOURCES.txt
[pbr] In git context, generating filelist from git
warning: no previously-included files found matching '.gitreview'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
writing manifest file 'jorji.egg-info/SOURCES.txt'
running check
warning: check: missing required meta-data: url
creating jorji-0.1.2.4.g7fded09
creating jorji-0.1.2.4.g7fded09/jorji
creating jorji-0.1.2.4.g7fded09/jorji.egg-info
creating jorji-0.1.2.4.g7fded09/jorji/test
making hard links in jorji-0.1.2.4.g7fded09...
hard linking .coveragerc -> jorji-0.1.2.4.g7fded09
hard linking .travis.yml -> jorji-0.1.2.4.g7fded09
hard linking AUTHORS -> jorji-0.1.2.4.g7fded09
hard linking CONTRIBUTING.rst -> jorji-0.1.2.4.g7fded09
hard linking ChangeLog -> jorji-0.1.2.4.g7fded09
hard linking LICENSE -> jorji-0.1.2.4.g7fded09
hard linking MANIFEST.in -> jorji-0.1.2.4.g7fded09
hard linking README.rst -> jorji-0.1.2.4.g7fded09
hard linking requirements.txt -> jorji-0.1.2.4.g7fded09
hard linking setup.cfg -> jorji-0.1.2.4.g7fded09
hard linking setup.py -> jorji-0.1.2.4.g7fded09
hard linking tox.ini -> jorji-0.1.2.4.g7fded09
hard linking jorji/__init__.py -> jorji-0.1.2.4.g7fded09/jorji
hard linking jorji/_keys.py -> jorji-0.1.2.4.g7fded09/jorji
hard linking jorji/certificate.py -> jorji-0.1.2.4.g7fded09/jorji
hard linking jorji/specifications.py -> jorji-0.1.2.4.g7fded09/jorji
hard linking jorji.egg-info/PKG-INFO -> jorji-0.1.2.4.g7fded09/jorji.egg-info
hard linking jorji.egg-info/SOURCES.txt -> jorji-0.1.2.4.g7fded09/jorji.egg-info
hard linking jorji.egg-info/dependency_links.txt -> jorji-0.1.2.4.g7fded09/jorji.egg-info
hard linking jorji.egg-info/not-zip-safe -> jorji-0.1.2.4.g7fded09/jorji.egg-info
hard linking jorji.egg-info/requires.txt -> jorji-0.1.2.4.g7fded09/jorji.egg-info
hard linking jorji.egg-info/top_level.txt -> jorji-0.1.2.4.g7fded09/jorji.egg-info
hard linking jorji/test/__init__.py -> jorji-0.1.2.4.g7fded09/jorji/test
hard linking jorji/test/test_specifications.py -> jorji-0.1.2.4.g7fded09/jorji/test
copying setup.cfg -> jorji-0.1.2.4.g7fded09
Writing jorji-0.1.2.4.g7fded09/setup.cfg
Creating tar archive
removing 'jorji-0.1.2.4.g7fded09' (and everything under it)
x jorji-0.1.2.4.g7fded09/
x jorji-0.1.2.4.g7fded09/.coveragerc
x jorji-0.1.2.4.g7fded09/.travis.yml
x jorji-0.1.2.4.g7fded09/AUTHORS
x jorji-0.1.2.4.g7fded09/ChangeLog
x jorji-0.1.2.4.g7fded09/CONTRIBUTING.rst
x jorji-0.1.2.4.g7fded09/jorji/
x jorji-0.1.2.4.g7fded09/jorji/__init__.py
x jorji-0.1.2.4.g7fded09/jorji/_keys.py
x jorji-0.1.2.4.g7fded09/jorji/certificate.py
x jorji-0.1.2.4.g7fded09/jorji/specifications.py
x jorji-0.1.2.4.g7fded09/jorji/test/
x jorji-0.1.2.4.g7fded09/jorji/test/__init__.py
x jorji-0.1.2.4.g7fded09/jorji/test/test_specifications.py
x jorji-0.1.2.4.g7fded09/jorji.egg-info/
x jorji-0.1.2.4.g7fded09/jorji.egg-info/dependency_links.txt
x jorji-0.1.2.4.g7fded09/jorji.egg-info/not-zip-safe
x jorji-0.1.2.4.g7fded09/jorji.egg-info/PKG-INFO
x jorji-0.1.2.4.g7fded09/jorji.egg-info/requires.txt
x jorji-0.1.2.4.g7fded09/jorji.egg-info/SOURCES.txt
x jorji-0.1.2.4.g7fded09/jorji.egg-info/top_level.txt
x jorji-0.1.2.4.g7fded09/LICENSE
x jorji-0.1.2.4.g7fded09/MANIFEST.in
x jorji-0.1.2.4.g7fded09/PKG-INFO
x jorji-0.1.2.4.g7fded09/README.rst
x jorji-0.1.2.4.g7fded09/requirements.txt
x jorji-0.1.2.4.g7fded09/setup.cfg
x jorji-0.1.2.4.g7fded09/setup.py
x jorji-0.1.2.4.g7fded09/tox.ini

This isn't in a git repository anymore:

[jorji-0.1.2.4.g7fded09] git status                                                                                                                                                               10:03:54
fatal: Not a git repository (or any of the parent directories): .git

Then build a new sdist:

[jorji-0.1.2.4.g7fded09] python setup.py sdist                                                                                                                                                    10:04:01
running sdist
[pbr] Writing ChangeLog
[pbr] Generating AUTHORS
running egg_info
writing requirements to jorji.egg-info/requires.txt
writing jorji.egg-info/PKG-INFO
writing top-level names to jorji.egg-info/top_level.txt
writing dependency_links to jorji.egg-info/dependency_links.txt
[pbr] Processing SOURCES.txt
warning: no previously-included files found matching '.gitignore'
warning: no previously-included files found matching '.gitreview'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
writing manifest file 'jorji.egg-info/SOURCES.txt'
running check
warning: check: missing required meta-data: url

creating jorji-0.1.2.4.g7fded09
creating jorji-0.1.2.4.g7fded09/jorji
creating jorji-0.1.2.4.g7fded09/jorji.egg-info
creating jorji-0.1.2.4.g7fded09/jorji/test
making hard links in jorji-0.1.2.4.g7fded09...
hard linking AUTHORS -> jorji-0.1.2.4.g7fded09
hard linking ChangeLog -> jorji-0.1.2.4.g7fded09
hard linking LICENSE -> jorji-0.1.2.4.g7fded09
hard linking MANIFEST.in -> jorji-0.1.2.4.g7fded09
hard linking README.rst -> jorji-0.1.2.4.g7fded09
hard linking requirements.txt -> jorji-0.1.2.4.g7fded09
hard linking setup.cfg -> jorji-0.1.2.4.g7fded09
hard linking setup.py -> jorji-0.1.2.4.g7fded09
hard linking jorji/__init__.py -> jorji-0.1.2.4.g7fded09/jorji
hard linking jorji/_keys.py -> jorji-0.1.2.4.g7fded09/jorji
hard linking jorji/certificate.py -> jorji-0.1.2.4.g7fded09/jorji
hard linking jorji/specifications.py -> jorji-0.1.2.4.g7fded09/jorji
hard linking jorji.egg-info/PKG-INFO -> jorji-0.1.2.4.g7fded09/jorji.egg-info
hard linking jorji.egg-info/SOURCES.txt -> jorji-0.1.2.4.g7fded09/jorji.egg-info
hard linking jorji.egg-info/dependency_links.txt -> jorji-0.1.2.4.g7fded09/jorji.egg-info
hard linking jorji.egg-info/not-zip-safe -> jorji-0.1.2.4.g7fded09/jorji.egg-info
hard linking jorji.egg-info/requires.txt -> jorji-0.1.2.4.g7fded09/jorji.egg-info
hard linking jorji.egg-info/top_level.txt -> jorji-0.1.2.4.g7fded09/jorji.egg-info
hard linking jorji/test/__init__.py -> jorji-0.1.2.4.g7fded09/jorji/test
hard linking jorji/test/test_specifications.py -> jorji-0.1.2.4.g7fded09/jorji/test
copying setup.cfg -> jorji-0.1.2.4.g7fded09
Writing jorji-0.1.2.4.g7fded09/setup.cfg
creating dist
Creating tar archive
removing 'jorji-0.1.2.4.g7fded09' (and everything under it)

This is most likely related to jorji using pbr. pbr is used by most openstack projects, so hopefully we can get it working for those and put check-manifest in all the builds :)

It grabs the version from version control (something akin to git describe) when creating the original sdist, and grabs it from egg-info when installing from an sdist. I have no idea why it fails when check-manifest does it.

[lvh]

@mgedmin: Am I parsing your previous comment correctly that the "clean sdist" does not have egg-info and that that's intentional?

[mgedmin]

"Building a clean sdist" == copy all versioned files into an empty temporary directory, then run python setup.py sdist in there. It will not have *.egg-info (unless you add it to your version control system for some reason).

This is more or less what you'd get if you exported the most recent VCS revision, except it also contains uncommitted modifications and files that are locally added but not committed yet.

This behavior is intentional. It was intended to support the "building an sdist from a VCS export" use-case, as well as making sure "building an sdist from a VCS checkout that setuptools doesn't understand" isn't broken when your current setuptools version actually understands your VCS (see issue #1).

[lvh]

I've found a sort-of workaround for jorji, because pbr supports overriding the version-finding logic with an environment variable. (See reffed commit.)

[mgedmin]

I think I'll close this issue, since you've a workaround, and I've no ideas how to make your use case work without regressing either bug #1 (if I copy *.egg-info to the temp dir) or failing to detect gaps in MANIFEST.in when setuptools-git is installed (if I copy .git metadata to the temp dir).