Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use pure-sasl in python 3.11 #454

Merged
merged 1 commit into from
Jun 20, 2023

Conversation

mdeshmu
Copy link
Contributor

@mdeshmu mdeshmu commented May 20, 2023

There is a known issue with installing sasl lib in Python 3.11.

Maintainers of the sasl repo haven't contributed to it since 2021.

This PR adds additional extras pyhive[hive_pure_sasl] which uses pure-sasl allowing end user to choose what sasl library they want to use.

Existing pyhive[hive] extras will continue to use sasl library.

With these code changes, pyhive can work with both python-sasl and pure-sasl with sasl being preferred first.

Please refer below links for the CI test logs from apache/superset repository:
3.11 (pure-sasl) -> https://github.com/apache/superset/actions/runs/5302603180/jobs/9597648049
3.10 (pure-sasl) -> https://github.com/apache/superset/actions/runs/5302917493/jobs/9598184738
3.9 (pure-sasl) -> https://github.com/apache/superset/actions/runs/5303030643/jobs/9598385720

3.10 (sasl) -> https://github.com/apache/superset/actions/runs/5302756384/jobs/9597916446
3.9 (sasl) -> https://github.com/apache/superset/actions/runs/5302964104/jobs/9598269981?pr=24157

pyhive/hive.py Outdated Show resolved Hide resolved
@JimNero009
Copy link

Stumbled upon this while doing some general py3.11 upgrades across some projects (transitive dep here via dbt-spark hive extras). Very keen for an installation path to be available, anything I can do to help?

@mdeshmu mdeshmu force-pushed the use-pure-sasl-in-python-3.11 branch 2 times, most recently from 52f9340 to dd62269 Compare May 29, 2023 14:34
@mdeshmu mdeshmu requested a review from bkyryliuk May 29, 2023 16:37
pyhive/hive.py Outdated Show resolved Hide resolved
pyhive/hive.py Outdated Show resolved Hide resolved
pyhive/tests/test_hive.py Outdated Show resolved Hide resolved
pyhive/tests/test_hive.py Outdated Show resolved Hide resolved
pyhive/tests/test_hive.py Outdated Show resolved Hide resolved
setup.py Outdated Show resolved Hide resolved
pyhive/tests/test_sasl_compat.py Show resolved Hide resolved
@mdeshmu mdeshmu force-pushed the use-pure-sasl-in-python-3.11 branch 7 times, most recently from 1e48903 to 36e6364 Compare June 4, 2023 15:08
@mdeshmu mdeshmu closed this Jun 4, 2023
@mdeshmu mdeshmu force-pushed the use-pure-sasl-in-python-3.11 branch from 36e6364 to 0bd6f5b Compare June 4, 2023 15:33
@mdeshmu mdeshmu reopened this Jun 5, 2023
@mdeshmu mdeshmu force-pushed the use-pure-sasl-in-python-3.11 branch from 9539992 to 2c9a6fe Compare June 5, 2023 10:27
@mdeshmu mdeshmu requested a review from bkyryliuk June 5, 2023 12:05
pyhive/hive.py Outdated Show resolved Hide resolved
@mdeshmu mdeshmu force-pushed the use-pure-sasl-in-python-3.11 branch from f6b2dce to e38eb23 Compare June 18, 2023 07:09
@mdeshmu mdeshmu requested a review from bkyryliuk June 18, 2023 09:40
pyhive/hive.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@bkyryliuk bkyryliuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG%nit

@mdeshmu mdeshmu force-pushed the use-pure-sasl-in-python-3.11 branch from 78691ce to 1eef88b Compare June 20, 2023 10:35
@bkyryliuk bkyryliuk merged commit 4367cc5 into dropbox:master Jun 20, 2023
@tyong920
Copy link

Awesome work! Thanks you guys! Looking forward to a release ASAP!

@colin-rogers-dbt
Copy link

Any idea when this is going to be released?

@mdeshmu
Copy link
Contributor Author

mdeshmu commented Jul 14, 2023

@JimNero009 @tyong920 @colin-rogers-dbt @mb-m

This is released in 0.7.1.dev0. You are requested to test with the dev version and report any bugs in the PyHive GitHub repository before 0.7.1 is released in a month or so.

PyHive now supports pure-sasl via additional extras 'pyhive[hive_pure_sasl]' which supports Python 3.11 in addition to previous Python versions. See #454

@colin-rogers-dbt
Copy link

@mdeshmu still planning on releasing in the next week or two?

@mdeshmu
Copy link
Contributor Author

mdeshmu commented Aug 22, 2023

@colin-rogers-dbt this is released in 0.7.0

@mdeshmu mdeshmu deleted the use-pure-sasl-in-python-3.11 branch August 23, 2023 00:35
betodealmeida added a commit to preset-io/PyHive that referenced this pull request Aug 8, 2024
* feat: add HTTP and HTTPS to hive (dropbox#385)

* feat: add https protocol

* support HTTP

* fix: make hive https py2 compat (dropbox#389)

* fix: make hive https py2 compat

* fix lint

* Update README.rst (dropbox#423)

* chore: rename Trino entry point (dropbox#428)

* Support for Presto decimals (dropbox#430)

* Support for Presto decimals

* lower

* Use str type for driver and name in HiveDialect (dropbox#450)

PyHive's HiveDialect usage of bytes for the name and driver fields is not the norm is causing issues upstream: apache/superset#22316
Even other dialects within PyHive use strings. SQLAlchemy does not strictly require a string, but all the stock dialects return a string, so I figure it is heavily implied.

I think the risk of breaking something upstream with this change is low (but it is there ofc). I figure in most cases we just make someone's `str(dialect.driver)` expression redundant.

Examples for some of the other stock sqlalchemy dialects (name and driver fields using str):
https://github.com/zzzeek/sqlalchemy/blob/main/lib/sqlalchemy/dialects/sqlite/pysqlite.py#L501
https://github.com/zzzeek/sqlalchemy/blob/main/lib/sqlalchemy/dialects/sqlite/base.py#L1891
https://github.com/zzzeek/sqlalchemy/blob/main/lib/sqlalchemy/dialects/mysql/base.py#L2383
https://github.com/zzzeek/sqlalchemy/blob/main/lib/sqlalchemy/dialects/mysql/mysqldb.py#L113
https://github.com/zzzeek/sqlalchemy/blob/main/lib/sqlalchemy/dialects/mysql/pymysql.py#L59

* Correcting Iterable import for python 3.10 (dropbox#451)

* changing drivers to support hive, presto and trino with sqlalchemy>=2.0 (dropbox#448)

* Revert "changing drivers to support hive, presto and trino with sqlalchemy>=2.0 (dropbox#448)" (dropbox#452)

This reverts commit b0206d3.

* Update __init__.py (dropbox#453)

dropbox@1c1da8b

dropbox@1f99552

* use pure-sasl with python 3.11 (dropbox#454)

* minimal changes for sqlalchemy 2.0 support (dropbox#457)

* update readme to reflect recent changes (dropbox#459)

* Update README.rst (dropbox#475)

* Update README.rst (dropbox#476)

* feat: JWT support

* Add CI to build package

---------

Co-authored-by: Daniel Vaz Gaspar <danielvazgaspar@gmail.com>
Co-authored-by: Bogdan <b.kyryliuk@gmail.com>
Co-authored-by: serenajiang <serena.jiang@airbnb.com>
Co-authored-by: Usiel Riedl <usiel.riedl@gmail.com>
Co-authored-by: Multazim Deshmukh <57723564+mdeshmu@users.noreply.github.com>
Co-authored-by: nicholas-miles <nicholas.miles6@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants