Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch cloudant provider from cloudant library to ibmcloudant library #41555

Merged
merged 33 commits into from
Sep 6, 2024

Conversation

topherinternational
Copy link
Contributor

closes: #21004, cloudant is no longer developed and ibmcloudant is the active replacement

This is a breaking change to the provider in the following ways:

  • get_conn now returns a CloudantV1 object with different function names than the previous Cloudant object; the mappings from the old library functions to the new are outlined here: https://github.com/cloudant/python-cloudant/blob/master/MIGRATION.md#reference-table
  • get_conn now directly returns a usable object instead of a context manager; with-block syntax is no longer needed
  • The connection is now checked for a mandatory host value that functions as the Cloudant account name (previously the host field was optional, but could cause a runtime failure)

@topherinternational
Copy link
Contributor Author

topherinternational commented Aug 17, 2024

Rats, looks like the joke is on me and this PR is blocked on another third-party...ibmcloudant has a dependency conflict with the Snowflake Python library that nerfs the Airflow build for Python 3.8 and 3.9.

There's an issue for this in the Snowflake repo (two issues actually) with an open PR, so hopefully it will be fixed soon and this Cloudant PR will unblock.

Ironic as this issue stalled for a while waiting for Flask-AppBuilder to fix a dependency pin.

Details:

ibmcloudant brings in ibm-cloud-sdk-core which requires urllib3 2.x, which is fine for core Airflow, but the snowflake provider brings in snowflake-connector-python, which (for Python 3.8. and 3.9 only) requires urllib3 1.x.

@potiuk
Copy link
Member

potiuk commented Aug 18, 2024

Rats, looks like the joke is on me and this PR is blocked on another third-party...ibmcloudant has a dependency conflict with the Snowflake Python library that nerfs the Airflow build for Python 3.8 and 3.9.

There's an issue for this in the Snowflake repo (two issues actually) with an open PR, so hopefully it will be fixed soon and this Cloudant PR will unblock.

Ironic as this issue stalled for a while waiting for Flask-AppBuilder to fix a dependency pin.

Details:

ibmcloudant brings in ibm-cloud-sdk-core which requires urllib3 2.x, which is fine for core Airflow, but the snowflake provider brings in snowflake-connector-python, which (for Python 3.8. and 3.9 only) requires urllib3 1.x.

You could potentially also add "3.8" and "3.9" to "excluded-python-versions" in the new version of cloudant provider (in provider.yaml) with the link to the snowflake issue - and remove the exclusions once this problem is fixed.

@potiuk
Copy link
Member

potiuk commented Aug 18, 2024

See #41548 - where we just removed such exclusion for Papermill for Python 3.12

@potiuk potiuk added the all versions If set, the CI build will be forced to use all versions of Python/K8S/DBs label Aug 18, 2024
@potiuk
Copy link
Member

potiuk commented Aug 18, 2024

I've added "all-versions" tag to the PR of yours - to make sure it's tested on all versions - so once you exclude python versions you should be able to see both CI images and tests to run for cloudant only on 3.10 -> 3.12 images and be skipped for 3.8 and 3.9

@topherinternational
Copy link
Contributor Author

Great explanation, thank you @potiuk

@topherinternational
Copy link
Contributor Author

Ok @potiuk I've added the exclusion and an explanatory comment, so I think this is complete. Please review at your convenience, thanks.

@potiuk
Copy link
Member

potiuk commented Aug 19, 2024

You should also add a note at the top of changelog (see comment at the top of changelog) and bump the version to the new major release in the provider.yaml.

@potiuk
Copy link
Member

potiuk commented Sep 5, 2024

🤞 🤞 🤞 🤞 🤞 🤞 🤞 🤞🤞 🤞 🤞 🤞 🤞 🤞 🤞 🤞🤞 🤞 🤞 🤞 🤞 🤞 🤞 🤞🤞 🤞 🤞 🤞 🤞 🤞 🤞 🤞

@potiuk potiuk merged commit caa94fa into apache:main Sep 6, 2024
108 checks passed
@potiuk
Copy link
Member

potiuk commented Sep 6, 2024

Woooooohoooo!

@topherinternational
Copy link
Contributor Author

Woooooohoooo!

This has been...educational 😉

@topherinternational topherinternational deleted the ibmcloudant branch September 6, 2024 11:10
@potiuk
Copy link
Member

potiuk commented Sep 6, 2024

This has been...educational

Oh, I am sure it was :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
all versions If set, the CI build will be forced to use all versions of Python/K8S/DBs area:providers provider:cloudant
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Switch from 'cloudant' to 'ibmcloudant'
3 participants