Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

check-manifest no longer finds files in sdist but not in VCS #27

Closed
mgedmin opened this issue Feb 4, 2014 · 5 comments
Closed

check-manifest no longer finds files in sdist but not in VCS #27

mgedmin opened this issue Feb 4, 2014 · 5 comments

Comments

@mgedmin
Copy link
Owner

mgedmin commented Feb 4, 2014

A while ago I changed check-manifest to build a source distribution in a temporary directory, to avoid setuptools plugins from butting in and overriding MANIFEST.in (issue #1). The temporary directory gets a copy of all the source files from the VCS (plus MANIFEST.in, even if it's not in the VCS -- since that's what we're checking).

Unfortunately as a result you can no longer discover that your MANIFEST.in includes files that aren't present in the version control system -- they can't possibly be included during the test since they're not copied to the temporary directory.

I'm afraid this means check-manifest will need to build the sdist twice: once in the temporary directory, and once directly in the source tree.

I'm also toying with an alternative, simpler option, of having check-manifest --inplace skip the copying step.

@mgedmin
Copy link
Owner Author

mgedmin commented Feb 7, 2014

Suppose I have three sets of files:

  • source_files (extracted from the VCS),
  • sdist_files (produced by running setup.py sdist in the source tree, where setuptools might make use of VCS data or an old .egg-info/SOURCES.txt)
  • clean_sdist_files (produced by running setup.py sdist in a temp directory that contains only a copy of source_files plus MANIFEST.in).

I need to draw a Venn diagram of the three sets and figure out what it all means.

@mgedmin
Copy link
Owner Author

mgedmin commented Feb 9, 2014

(I'm blocked by not having paper and pencils at home, and by being busy at work.)

@mgedmin
Copy link
Owner Author

mgedmin commented May 14, 2014

venn

Areas:

  1. missing from MANIFEST.in
  2. missing from VCS (not really possible to have files there, except maybe MANIFEST.in if it somehow gets excluded from the "dirty" sdist)
  3. missing from VCS
  4. missing from VCS (the only file that can be here is MANIFEST.in itself)
  5. missing from MANIFEST.in
  6. is it even possible? I don't think so
  7. all good

@mgedmin
Copy link
Owner Author

mgedmin commented May 14, 2014

So,

missing from MANIFEST.in = 1 and 5 = source_files - clean_sdist_files
missing from VCS = 2 and 3 and 4 = (sdist_files | clean_sdist_files) - source_files

It may be also interesting to compute 6 just to see if it is non-empty, because I don't think that's possible: (clean_sdist_files & source_files) - sdist_files.

mgedmin added a commit that referenced this issue May 14, 2014
This check requires that we build an sdist directly in the original
source tree, without making a clean copy.  But we need a clean copy in
order to check MANIFEST.in when setuptools VCS logic (or legacy
SOURCES.txt parsing) kicks in!  To satisfy both requirements we end up
having to build the sdist twice: once in a clean tree, and once in the
original, dirty tree.

The

  missing_from_VCS = set(sdist_files + clean_sdist_files) - set(source_files)

check could probably have been written as

  missing_from_VCS = set(sdist_files) - set(source_files)

because I'm 99.9% sure sdist_files is a superset of clean_sdist_files.
But just in case I'm wrong I thought it's better to be sure.
@mgedmin
Copy link
Owner Author

mgedmin commented May 14, 2014

This will be fixed in version 0.20, out later today.

(I wish to fix #28 before making a release.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant