-
Notifications
You must be signed in to change notification settings - Fork 470
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SNOW-395801: MemoryError when connecting on OSX using PyOpenSSL #799
Comments
This issue is currently blocking users from installing dbt-snowflake on standard Apple M1 architectures. While it's not a blocker to using dbt + Snowflake at all—there are valid workarounds via Rosetta—it makes installation much more difficult than it needs to be. My understanding is that the operative issue here is the monkey-patch of It looks like there are two places where the Snowflake python connector turns on the monkey patch: The actual
My questions:
|
This is actually done by
Yes, unfortunately
We require |
@sfc-gh-mkeller Really appreciate the quick response. You're right, it looks like the most recent version of From pyca/pyopenssl#873 (comment), it sounds like
Do you have any plans, or see a way forward, to supporting native installations of this package on M1 Macbooks / ARM-based architectures? |
|
For what it's worth, I unsuccessfully tried a rosetta approach to install dbt. I also tried other recommended methods such as pip and virtualenv. So, while it's not blocking everyone, it's causing me a headache. |
We also have a bunch of users on M1 macs who we want to get set up on dbt; and this issue is causing us a fair amount of pain in our dbt installation |
As a temporary fix: This is a working solution using rosetta. I'm using it right now on a M1 MBP. |
@jtcohen6 - Can you share an update on when this might get fixed? So far we can't get any of these workarounds to work with our new analysts' M1. |
Instead of the linked workaround above (installing Homebrew via Rosetta), I customized a docker image based on this image https://hub.docker.com/r/xemuliam/dbt. Docker Desktop now natively supports M1 architectures, and this image can build arm64 containers. |
In case it helps anyone, after many unsuccessful attempts, what worked for me was bumping python version to 3.10. |
I installed Python 3.9 through
and then installed the current I read that Is anyone still able to reproduce this error? |
@sfc-gh-mkeller I can still reproduce this error with the following env:
Env was created via miniforge with these commands: conda create -n test-env python=3.9.12
conda activate test-env
conda install -c conda-forge snowflake-connector-python==2.7.6 Excerpts from
And here is my minimal test function (with creds stored in env vars) import snowflake.connector
import os
con = snowflake.connector.connect(
user=os.getenv('SNOWFLAKE_USER'),
password=os.getenv('SNOWFLAKE_PASSWORD'),
account=os.getenv('SNOWFLAKE_ACCOUNT'),
role=os.getenv('SNOWFLAKE_ROLE'),
warehouse=os.getenv('SNOWFLAKE_WH'),
)
c = con.cursor().execute("select current_user()")
print(c.fetchone()) EDIT: The only thing that's worked for me is to install everything under Rosetta emulation osx-64. However, Rosetta is not ideal for some other ML packages I want in the same env. |
@JasonTam I wonder if this is because of the conda compiled Python binary. However; I don't know anything about Anaconda. |
@sfc-gh-mkeller You're right, it's the
I had to do this since there were also a whole bunch of things that required Otherwise, more simply, installing |
Since nobody seems to be having issues recently I'm going to close this issue, please don't hesitate to reach out if you are having issues! |
I still have issues. I got the error when trying to run "pip install cffi" following the suggestion mentioned above -
My environment
Any ideas? |
While trying to establish a connection, I receive a
MemoryError
which appears to be connected to some low level permission error on OSX which is triggered by using PyOpenSSL, at least on M1 mac books
I believe this is caused by pyca/pyopenssl#873
Python 3.9.6
macOS-11.4-arm64-arm-64bit
Conda list
What did you do?
If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
What did you expect to see?
What did you see instead?
Logs
The text was updated successfully, but these errors were encountered: