Unable to install dbt 0.19.1 on M1 Apple Silicon and python 3.8.6

[jtalmi]

Describe the bug

While on a new apple silicon mac, the user can only install dbt 0.18.2. Attempting to upgrade using pip yields the following error message:

ERROR: Could not find a version that satisfies the requirement snowflake-connector-python[secure-local-storage]==2.3.6 (from dbt-snowflake)
ERROR: No matching distribution found for snowflake-connector-python[secure-local-storage]==2.3.6

Steps To Reproduce

pip install dbt installs 0.18.2.

Expected behavior

Install the latest version (0.19.1) of dbt

Screenshots and log output

See above

System information

Which database are you using dbt with?

• postgres
• redshift
• bigquery
• [x ] snowflake
• other (specify: ____________)

The output of dbt --version:

installed version: 0.18.2
   latest version: 0.19.1
Your version of dbt is out of date! You can find instructions for upgrading here:
https://docs.getdbt.com/docs/installation
Plugins:
  - bigquery: 0.18.2
  - snowflake: 0.18.2
  - redshift: 0.18.2
  - postgres: 0.18.2

The operating system you're using:
Mac OS Big Sur

The output of python --version:
Python 3.8.6
pip 21.0.1

[jtcohen6]

@jtalmi I believe this is because of the version of pyarrow required by snowflake-connector-python==2.3.6. I think the best workaround for the time being is to instal Rosetta, which should then let you install via pip or Homebrew in a py38 as if using an Intel chip.

This is definitely something we're aware of (e.g. over in #3162):

• Anyone not using Snowflake can install the latest version of their dbt adapter plugin on an M1 chip or in a py39 environment by specifying the adapter: pip install dbt-[adapter]
• Anyone using Snowflake: We've bumped the version of snowflake-connector-python in the develop branch, which should now be M1/py39 compatible. This issue will be resolved in the next version of dbt (v0.20.0). Sorry for the pain in the meantime!

[jtalmi]

Thanks @jtcohen6, I think they'll stay on 0.18.2 for the meantime if they can't get Rosetta working.

[drewbanin]

I had a tricky time getting dbt installed in my dev env using:

• Python 3.9
• macOS Big Sur
• an Apple silicon Mac

I tried running

pip install -r editable_requirements.txt

But i was seeing errors about the grpcio lib failing to install with an error about clang. I found this comment grpc/grpc#24677 (comment) and ran:

GRPC_BUILD_WITH_BORING_SSL_ASM="" GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=true GRPC_PYTHON_BUILD_SYSTEM_ZLIB=true pip install grpcio

And after that I was able to get dbt installed! Sharing in case it's useful to anyone else :)

[joshuataylor]

I purchased a Macbook Air with a M1 today, after being a stoic Linux user (well still am on Desktop) for many years.

Here's what I had to do to get dbt & snowflake working for me, and natively!:

1. Upgrade to python 3.9 (using asdf, this is super easy)
2. Upgrade to dbt-core 0.20.0-rc1, and remove dbt (as this references postgres, which I didn't need)
3. Bump cffi to latest version (1.14.5)
4. Bump hologram to 0.0.14
5. Bump jinja to 2.11.3
6. Bump numpy to 1.21.0
7. Bump pandas to 1.2.5
8. Bump snowflake-connector-python to 2.4.6

If you use any packages that aren't part of dbt, I highly recommend upgrading to latest as they probably support m1 by now.

The nice thing about getting all this to work is that future ARM64 processors will be so much easier (like AWS Graviton, which is also great).

[thomasaarholt]

I had a tricky time getting dbt installed in my dev env using:

• Python 3.9
• macOS Big Sur
• an Apple silicon Mac

I tried running

pip install -r editable_requirements.txt

But i was seeing errors about the grpcio lib failing to install with an error about clang. I found this comment grpc/grpc#24677 (comment) and ran:

GRPC_BUILD_WITH_BORING_SSL_ASM="" GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=true GRPC_PYTHON_BUILD_SYSTEM_ZLIB=true pip install grpcio

And after that I was able to get dbt installed! Sharing in case it's useful to anyone else :)

This did not work for me. Running dbt deps yields a cffi wrong architecture error.

[jeremyyeo]

@thomasaarholt keen to give this (https://github.com/jeremyyeo/dbt-docker-m1) a go? In docker though.

[thomasaarholt]

That'll run dbt in a Intel-architechture docker image? Or an arm one? I can't tell from the docker image. I can give it a try, sure.

[jeremyyeo]

That'll run dbt in a Intel-architechture docker image? Or an arm one? I can't tell from the docker image. I can give it a try, sure.

It will run dbt in an intel-arch docker image on a host arm machine - though performance may be less than desirable due to emulation. Only for those truly desperate I suppose. 🤣

[thomasaarholt]

Please see my post on snowflake-connector-python. I believe the problems are fixed when using latest 1.0 of dbt and current master (or in the future, version > 2.7.1) of snowflake-connector-python.

I have successfully updated my snowflake table using dbt natively on my M1 Macbook Pro.

[joshuataylor]

We've been running 0.20.1 and beyond on M1 fine.

[danielcarletti]

I'm on an M1 Mac, I could only finish the installation by specifying the package:
pip install dbt-redshift==0.20.1

Did not work otherwise.

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.

[github-actions]

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest; add a comment to notify the maintainers.

[HuangLiPang]

I'm on an M1 Mac, I could only finish the installation by specifying the package: pip install dbt-redshift==0.20.1

Did not work otherwise.

@danielcarletti Any workaround for this? Having the same issue and could not fix