Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bring back hive support for Python 3.11 #32607

Merged
merged 1 commit into from
Aug 24, 2023
Merged

Conversation

potiuk
Copy link
Member

@potiuk potiuk commented Jul 14, 2023


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@potiuk potiuk added the full tests needed We need to run full set of tests for this PR to merge label Jul 14, 2023
@potiuk potiuk closed this Jul 14, 2023
@potiuk potiuk reopened this Jul 14, 2023
@potiuk potiuk marked this pull request as draft July 14, 2023 15:08
@potiuk
Copy link
Member Author

potiuk commented Jul 14, 2023

ah @pankajastro I saw yo ualso looked at this one :)

@mdeshmu
Copy link

mdeshmu commented Jul 14, 2023

It should be pyhive[hive_pure_sasl]>=0.7.1.dev0

@potiuk
Copy link
Member Author

potiuk commented Jul 14, 2023

It should be pyhive[hive_pure_sasl]>=0.7.1.dev0

Yeah. Seems like we have to combine the two :)

@potiuk potiuk force-pushed the test-hive-for-3.11 branch from 239abef to 1dbb95c Compare July 14, 2023 16:32
@potiuk
Copy link
Member Author

potiuk commented Jul 14, 2023

Updated. Also removed the "sasl" direct inclusion. Let's see :)

@pankajastro
Copy link
Member

ah @pankajastro I saw yo ualso looked at this one :)

yeah, actually because apache-hive provider (sasl )only.

@potiuk potiuk changed the title Test 0.7.1dev0 version relased with 3.11 support Bring back hive support for Python 3.11 support Jul 15, 2023
@potiuk potiuk changed the title Bring back hive support for Python 3.11 support Bring back hive support for Python 3.11 Jul 15, 2023
@potiuk potiuk force-pushed the test-hive-for-3.11 branch 2 times, most recently from 6e7cfa3 to 02fa709 Compare July 15, 2023 09:25
@potiuk
Copy link
Member Author

potiuk commented Jul 15, 2023

ok. @mdeshmu - I updated the PR to the state as it should be merged when 0.7.1 final release is made. Let us know when the release happen and we will merge it then.

@potiuk potiuk force-pushed the test-hive-for-3.11 branch 2 times, most recently from bc28545 to 098c566 Compare July 20, 2023 18:48
@potiuk potiuk force-pushed the test-hive-for-3.11 branch from 098c566 to f10c301 Compare August 6, 2023 10:25
@mdeshmu
Copy link

mdeshmu commented Aug 19, 2023

PyHive 0.7.0 is released which includes Python 3.11 and SQLAlchemy 2.0 support.

@potiuk potiuk marked this pull request as ready for review August 19, 2023 06:27
@potiuk potiuk force-pushed the test-hive-for-3.11 branch from ae5ff89 to ded4490 Compare August 19, 2023 06:27
@potiuk
Copy link
Member Author

potiuk commented Aug 19, 2023

Seems that `pyhive 0.7.0 has been released and we can bring back Hive support for Python 3.11 finally

@potiuk potiuk added this to the Airflow 2.7.1 milestone Aug 19, 2023
@potiuk potiuk force-pushed the test-hive-for-3.11 branch from ded4490 to 1f5ea3a Compare August 21, 2023 22:09
Copy link
Contributor

@eladkal eladkal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
No need for 2.7.1 milestone it will be part of provider release cycle

@eladkal eladkal removed this from the Airflow 2.7.1 milestone Aug 22, 2023
@potiuk potiuk added this to the Airflow 2.7.1 milestone Aug 22, 2023
@potiuk
Copy link
Member Author

potiuk commented Aug 22, 2023

We will actually need it in 2.7 branch/when we prepare 2.7.1 release.

We need to sync provider dependencies when we add a new provider (and this will need to be cherry-picked). The thing is that provider_dependencies.json is generated from provider.yaml files and it is used to generate airflow's extras. This is not needed for `airflow' package (because in the generated package, the dependencies are replaced with "apache-airflow-providers-apache-hive" dependency), but it matters for constraints generation. If we do not have "apache.hive" in 2-7 branch for Python 3.11 then Airflow 2.7.1 will not have constraints generated by hive even if the provider package with 3.11 support is released.

This is why I marked it with 2.7.1 - so that it gets cherry-picked there.

@potiuk potiuk force-pushed the test-hive-for-3.11 branch from 1f5ea3a to 8d61724 Compare August 23, 2023 19:54
Update airflow/providers/apache/hive/CHANGELOG.rst

Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>
@potiuk potiuk force-pushed the test-hive-for-3.11 branch from 8d61724 to b08dc5e Compare August 23, 2023 20:20
@potiuk
Copy link
Member Author

potiuk commented Aug 24, 2023

Just flakes.

@potiuk potiuk merged commit 08188f8 into apache:main Aug 24, 2023
@potiuk potiuk deleted the test-hive-for-3.11 branch August 24, 2023 09:52
ephraimbuddy pushed a commit that referenced this pull request Aug 28, 2023
Update airflow/providers/apache/hive/CHANGELOG.rst

Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>
(cherry picked from commit 08188f8)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers full tests needed We need to run full set of tests for this PR to merge provider:apache-hive
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants