Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 sync should be applied filters to destination when "--delete" is specified #778

Closed
jamesls opened this issue May 15, 2014 · 2 comments
Closed

Comments

@jamesls
Copy link
Member

jamesls commented May 15, 2014

If I have:


local           remote
-----           ------

a.txt           a.txt
                b.py

And I run aws s3 sync local remote --exclude '*.py' --delete, we are currently deleting "b.py". This is because we only apply filters to the sending side, which outside of the "--delete" flag, is all that's required. When "--delete" is specified, we need to run all the filters against the sending and receiving side so that we don't inadvertently delete files.

Our current behavior is similar to rsync's --exclude '*.py' --delete-excluded and we should instead be like rsync's --exclude '*.py' --delete.

This is technically a change in behavior, but we will now no longer be deleting files we were previously deleting so it is a non-destructive change. I think this bug is important enough to change the behavior of --exclude rather than adding a new argument. If anyone wants the previous behavior we can add rsync's --delete-excluded argument.

jamesls added a commit to jamesls/aws-cli that referenced this issue May 16, 2014
Fixes the case with --exclude and --delete being specified.
See aws#778 for more info.
@jamesls
Copy link
Member Author

jamesls commented May 16, 2014

So I have an initial fix for this linked above, but after investigating further, I'm wondering whether or not it makes sense to just adopt rsync's semantics for exclude/include filters. Most users are familiar with rsync so it would be easier to explain if we had the same behavior.

I'd need to do an analysis and see what all the changes would be, and exactly what the delta would be from the existing behavior.

jamesls added a commit to jamesls/aws-cli that referenced this issue May 16, 2014
Fixes the case with --exclude and --delete being specified.
See aws#778 for more info.
@jamesls
Copy link
Member Author

jamesls commented May 19, 2014

I spent some time looking into adopting the rsync exclude/include filters and it would be too big of a change in behavior to change exclude/include semantics. We could possible add either mutually exclusive args that have the rsync style exclude/includes but I'm not sure the complexity is worth it at this point. For now, #780 fixes the immediate issue so I'm going to close this issue.

@jamesls jamesls closed this as completed May 19, 2014
jamesls added a commit that referenced this issue May 19, 2014
This is needed to completely fix #778.  #780 added a partial fix,
this is needed so that this fix also works on windows.
thoward-godaddy pushed a commit to thoward-godaddy/aws-cli that referenced this issue Feb 12, 2022
Bumping version to 0.7.0 in preparation for release
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant