Fix incorrect .airflowignore behavior with multiple nested directories #11994
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@j-y-matsubara does this make sense? |
|
@potiuk would you be so kind as to take a look at another bugfix in airflowignore? Thanks! |
github-actions
bot
added
the
full tests needed
|
The PR needs to run all tests because it modifies core of Airflow! Please rebase it to latest master or ask committer to re-run it! |
|
Some temporary blip when pulling image |
|
No need to run full tests bot :) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem:
In
find_path_from_directory(), thepatterns_by_dirmap is incorrectly emptied and reset with patterns from the current directory, effectively forgetting about the patterns added earlier, for directories not yet processed. The modified testcase demonstrates the issue.Solution:
Instead of resetting the map, update it with the new entries. An entry for a directory is added only once, namely when its immediate parent directory is processed, therefore we never overwrite any key in the map.
Note:
The test is currently marked as
heisentest; PR #11993 unmarks it. I imagine that PR should go first, and this PR should be rebased after that.