Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

include dotfiles in packages #7680

Merged
merged 7 commits into from
Dec 12, 2019
Merged

include dotfiles in packages #7680

merged 7 commits into from
Dec 12, 2019

Conversation

stefanhoelzl
Copy link

This PR solves #7183

It changes the behavior of cargo package to also include dotfiles by default.

It should be discussed if this is intended or if the implementation should be changed to only include dotfiles which are specified in the include section.

From the existing comment it is a little bit unclear to me, but I supposed it was intended only to exclude directories starting with a dot?

@rust-highfive
Copy link

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @Eh2406 (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Dec 7, 2019
@Eh2406
Copy link
Contributor

Eh2406 commented Dec 7, 2019

Thanks for the PR! I read it to be only ones explicitly in the include list. what to others think (@ehuss / @alexcrichton ...)?

got rid of extra test for dotfiles
@stefanhoelzl
Copy link
Author

I tried to implement a variant where dot-files/dirs are excluded by default (current behavior) and found some other unexpected behavior which is not documented here

  • If a git-repo is detected and no include-section in the manifest
    • then all tracked and untracked files (which means no files on .gitignore) are used for packaging
    • except if they are mentioned in the exclude section
    • => tracked dot-files/dirs are included
  • otherwise
    • the whole directory tree is searched for files
    • except dot-files/dirs
    • => dot-files/dirs are always excluded even if they are listed in the include section

the documentation does not mention a different behavior when a git repo is present nor does it mention that dot-files/dirs are always ignored when there is not git repo present.

@ehuss
Copy link
Contributor

ehuss commented Dec 8, 2019

Yea, I would expect if a dotfile is listed in the include list, it should be included.

the documentation does not mention a different behavior when a git repo is present nor does it mention that dot-files/dirs are always ignored when there is not git repo present.

It does say "If git is being used for a package, the exclude field will be seeded with the gitignore settings from the repository."

It definitely would be good to mention the dot exclusion behavior when git is not used.

@stefanhoelzl
Copy link
Author

Yea, I would expect if a dotfile is listed in the include list, it should be included.

With my latest changes dotfiles in then include list are getting included, all the other behavior is still unchanged

It definitely would be good to mention the dot exclusion behavior when git is not used.

yes, that's what I meant, this behavior change is not mentioned in the documentation.

let root = pkg.root();
let mut exclude_dot_files_dir_builder = GitignoreBuilder::new(root);
exclude_dot_files_dir_builder.add_line(None, ".*")?;
exclude_dot_files_dir_builder.add_line(None, "*/.*/*")?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this line necessary?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, it is already covered with the other pattern

@ehuss
Copy link
Contributor

ehuss commented Dec 11, 2019

Seems ok to me, would like to see if @alexcrichton has any opinion.

@alexcrichton
Copy link
Member

I don't really remember the reason for original inclusion of this rule, I'd sort of expect that if git is present we use that exclusively to guide what files are included (dotfiles or not) and only if that's not present do we try to have some other heuristics. Directives like include should always take precedent as well though.

Copy link
Contributor

@ehuss ehuss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, will merge once CI is fixed by #7699.

@alexcrichton
Copy link
Member

@bors: r=ehuss

@bors
Copy link
Contributor

bors commented Dec 12, 2019

📌 Commit 5efda6a has been approved by ehuss

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 12, 2019
@bors
Copy link
Contributor

bors commented Dec 12, 2019

⌛ Testing commit 5efda6a with merge c8595e8...

bors added a commit that referenced this pull request Dec 12, 2019
include dotfiles in packages

This PR solves #7183

It changes the behavior of `cargo package` to also include dotfiles by default.

It should be discussed if this is intended or if the implementation should be changed to only include dotfiles which are specified in the `include` section.

From the [existing comment](https://github.com/stefanhoelzl/cargo/blob/40885dfab40a1bf62b22aa03f732ef45163c013f/src/cargo/sources/path.rs#L358) it is a little bit unclear to me, but I supposed it was intended only to exclude directories starting with a dot?
@bors
Copy link
Contributor

bors commented Dec 12, 2019

☀️ Test successful - checks-azure
Approved by: ehuss
Pushing c8595e8 to master...

@bors bors merged commit 5efda6a into rust-lang:master Dec 12, 2019
bors added a commit that referenced this pull request May 1, 2021
Respect Cargo.toml `[package.exclude]` even not in a git repo.

May resolves #9054

This bug (or feature?) has been lingering for a while. #7680 fixed the `cargo package` part but `cargo vendor` is still affected by the heuristic rule of ignoring dotfiles. ~~I propose to drop the rule and include dotfiles by default even if the package is not under git-controlled~~. See below.

## Updated: Changes Summary

`cargo vendor` vendors dependencies without git-controlled but `cargo package` often runs under a VCS like git. [These lines](https://github.com/rust-lang/cargo/blob/1ca930b/src/cargo/sources/path.rs#L161-L168) are where they diverges: `fn list_files_walk_except_dot_files_and_dirs` builds [its own ignore instance], which cannot merge with other filter rules from `[package.exclude]`. This causes some patterns to not work as expected, such as re-including file after ignoring dotfiles `[.*, !negated_file]`.

To make re-include (negate) rule works, this patch adds the excluding dotfiles rule directly into the `package.exclude` ignore instance if **_no include option nor git repo exists_**. Other old behaviors should not change in this patch.

[its own ignore instance]: https://github.com/rust-lang/cargo/blob/1ca930b6/src/cargo/sources/path.rs#L364-L366
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants