Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "Redirect old location module imports to standard provider (#4… #43946

Merged
merged 1 commit into from
Nov 13, 2024

Conversation

potiuk
Copy link
Member

@potiuk potiuk commented Nov 13, 2024

…3610)"

There is a very interesting breaking change introduced in Python 3.11 that will likely mean that we should not use MetaPathFinder for old standard provider classes redirection.

There was a change introduced in Python 3.11 that caused that module might not be found in some cases (for example when unit test patches the path) when the module is loaded as a different module (i.e. a.b differs from sys.modules['a.b'])

This is tracked in CPython via:
python/cpython#117860

This causes standard operator's tests fail in Python 3.11 and 3.12 when the providers/tests/standard/test_module_redirect_finder.py is executed before - i.e. the standard modules are loaded as old modules.

This reverts commit 5de2e73.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

…ache#43610)"

There is a very interesting breaking change introduced in Python 3.11
that will likely mean that we should not use MetaPathFinder for old
standard provider classes redirection.

There was a change introduced in Python 3.11 that caused that
module might not be found in some cases (for example when
unit test patches the path) when the module is loaded as
a different module (i.e. a.b differs from sys.modules['a.b'])

This is tracked in CPython via:
python/cpython#117860

This causes standard operator's tests fail in Python 3.11 and
3.12 when the providers/tests/standard/test_module_redirect_finder.py
is executed before - i.e. the standard modules are loaded as
old modules.

This reverts commit 5de2e73.
@gopidesupavan gopidesupavan merged commit 807fd6d into apache:main Nov 13, 2024
59 checks passed
@gopidesupavan
Copy link
Member

really interesting one :)

@potiuk
Copy link
Member Author

potiuk commented Nov 13, 2024

really interesting one :)

Indeed - that is about third time when our test harness (which I am reeally happy about) uncover a bug in CPython.
For me this one case is a single most important reason why we should have all the matrices and canary builds running everything. Once evey half a year or so, they discover an issue in one or two of the combos that would take us weeks of investigation if it would have happened in "production" - and it happens way, way, way before it is even close to being released.

ellisms pushed a commit to ellisms/airflow that referenced this pull request Nov 13, 2024
…ache#43610)" (apache#43946)

There is a very interesting breaking change introduced in Python 3.11
that will likely mean that we should not use MetaPathFinder for old
standard provider classes redirection.

There was a change introduced in Python 3.11 that caused that
module might not be found in some cases (for example when
unit test patches the path) when the module is loaded as
a different module (i.e. a.b differs from sys.modules['a.b'])

This is tracked in CPython via:
python/cpython#117860

This causes standard operator's tests fail in Python 3.11 and
3.12 when the providers/tests/standard/test_module_redirect_finder.py
is executed before - i.e. the standard modules are loaded as
old modules.

This reverts commit 5de2e73.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants