SNOW-396922: Snowflake patch change from 1.2.4 to 1.2.5 introduces a hard dependency on sqlalchemy>=1.4.0

[erikselin]

This is a potential breaking change that is not captured in setup.py and should not be part of a patch version. More context in this PR which ideally should not be the solution.

[sfc-gh-mkeller]

We are in the process of releasing the fix you provided in order to get this addressed as quickly as possible. Our team is reviewing how we can avoid this in the future through additional test cases and we will also review if a longer term fix is needed.
We apologize for any inconvenience that this caused!

[sfc-gh-mkeller]

Re-opening this issue to keep it visible to anyone that runs into the following error in the next while:

from sqlalchemy.sql.roles import From Clause Role ModuleNotFoundError: No module named 'sqlalchemy.sql.roles'

Solutions:

1. Manually update to SQLAlchemy to >=1.4.0,<2.0.0
2. Update snowflake-sqlalchemy to version 1.3.1 which has the correct SQLAlchemy dependency pinned

[jimkim-tc]

Re-opening this issue to keep it visible to anyone that runs into the following error in the next while:

from sqlalchemy.sql.roles import From Clause Role ModuleNotFoundError: No module named 'sqlalchemy.sql.roles'

Solutions:

1. Manually update to SQLAlchemy to >=1.4.0,<2.0.0
2. Update snowflake-sqlalchemy to version 1.3.0 which has the correct SQLAlchemy dependency pinned

Is the second option available? Appears the latest version is 1.2.5?

[sfc-gh-abhatnagar]

@jimkim-tc I will update here once the next version is released , likely Monday.

[sfc-gh-abhatnagar]

Released new version 1.3.1 for Snowflake SqlAlchemy which has the fix. Let us know for any issues.

[sfc-gh-kbregula]

@sfc-gh-abhatnagar This is still a problem. In the Apache Airflow project, we don't support SQLAlchemy 1.4, as Flask-AppBuilder dependency doesn't support SQlAlchemy 1.4 yet. Flask-App-Builder has a corresponding entry in setup.py.
https://github.com/dpgaspar/Flask-AppBuilder/blob/baddd7c73e68c347abe74973c4a122003393fd71/setup.py#L56-L57

        # SQLAlchemy 1.4.0 breaks flask-sqlalchemy and sqlalchemy-utils
        "SQLAlchemy<1.4.0",

Here is definition requirement of flask-appbuilder in apache-airflow project:
https://github.com/apache/airflow/blob/98514cc1599751d7611b3180c60887da0a25ff5e/setup.cfg#L107

    flask-appbuilder>=3.3.2, <4.0.0

In other words, Apache Airflow requires SQLAlchemy 1.3+, <1.4.
apache-airflow requires snowflake-sqlalchemy, but we use a very narrow API. Practically, we just need to create a connection, so we have very loose constraints, so we expect all problem should be handled by pip dependency resolver.
https://github.com/apache/airflow/blob/98514cc1599751d7611b3180c60887da0a25ff5e/setup.py#L470-L473

snowflake = [
    'snowflake-connector-python>=2.4.1',
    'snowflake-sqlalchemy>=1.1.0',
]

Unfortunately, these constraints is not sufficient as pip installs snowflake-sqlalchemy==1.2.5 after dependency resolution.
We must explicitly exclude these versions in order for the correct version to install.

snowflake = [
    'snowflake-connector-python>=2.4.1',
    'snowflake-sqlalchemy>=1.1.0',
    # The snowflake-alchemy 1.2.5 introduces a hard dependency on sqlalchemy>=1.4.0, but they didn't define
    # this requirements in setup.py, so pip cannot figure out the correct set of dependencies.
    # See: https://github.com/snowflakedb/snowflake-sqlalchemy/issues/234
    'snowflake-sqlalchemy>=1.1.0,!=1.2.5',
]

Could you mark the 1.2.5 release as yanked so that pip doesn't try to install this version that have a known bug in the dependency definition?

What's a "yanked" release?
A yanked release is a release that is always ignored by an installer, unless it is the only release that matches a version specifier (using either == or ===). See PEP 592 for more information.

(source: https://pypi.org/help/#yanked)

[sfc-gh-kbregula]

For reference: SNOW-518532 - Ticket about marking one release as yanked.

[sfc-gh-mkeller]

We have marked 1.2.5 as yanked