Cannot install in macOS BigSur (M1 chip)

[martriay]

I'm trying to install fastecdsa on macOS BigSur (M1 chip) by running

(venv) $ pip3 install fastecdsa

and even though I previously installed gmp:

$ brew install gmp

it cant find the lib, no matter what I do

    src/curve.h:4:10: fatal error: 'gmp.h' file not found
    #include "gmp.h"

Although the error changed when I created a symlink

ln -s /opt/homebrew/include/gmp.h /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include

and now I get:

    ld: library not found for -lgmp
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    error: command '/usr/bin/gcc' failed with exit code 1

I also tried:

• passing the path via env CFLAGS, LDFLAGS and both via global env export (export CFLAGS=...)

LDFLAGS=-L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib pip3 install fastecdsa

CFLAGS=-I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include pip3 install fastecdsa

• xcode-select --install
• endlessly brew uninstall gmp and brew install gmp, even brew reinstall gmp and brew unlink gmp
• installing rosetta2
• turning it off and on again

I can't put my finger on it :(

[milancermak]

This worked for me:

CFLAGS=-I/opt/homebrew/opt/gmp/include LDFLAGS=-L/opt/homebrew/opt/gmp/lib pip install fastecdsa

[AntonKueltz]

@martriay did the above work for you? If you have installed gmp via brew you should be able to point the compiler to the correct headers and libs via CFLAGS and LDFLAGs. You can use brew --prefix to determine the root path of your brew installed packages if they are different from the above comment.

[martriay]

@AntonKueltz no, but I compiled a list of solutions to my specific problem:
OpenZeppelin/nile#22 (comment)

[justinba1010]

Hi @martriay, I think this should work. I just tested it on my M1 mac.
python3 -m pip install --no-binary :all: --no-use-pep517 fastecdsa
I've been using this as a pip alias for awhile now.
Give this a shot and let me know.

[martriay]

Hi @martriay, I think this should work. I just tested it on my M1 mac. python3 -m pip install --no-binary :all: --no-use-pep517 fastecdsa I've been using this as a pip alias for awhile now. Give this a shot and let me know.

Even with gmp installed:

(cairo_venv) ➜  ~ brew reinstall gmp
==> Downloading https://ghcr.io/v2/homebrew/core/gmp/manifests/6.2.1_1
Already downloaded: /Users/martriay/Library/Caches/Homebrew/downloads/a1986a80dafb4e42b8372268a00d8d14aa8294154c876ba82e7ce48db06885cb--gmp-6.2.1_1.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/gmp/blobs/sha256:491220f1ff2c662b96295d931a80702523eeaee681d7305fb02b561e527dcbb8
Already downloaded: /Users/martriay/Library/Caches/Homebrew/downloads/3e74dcc39d8faf31f1e226060beea4bda63ec6888f7fe71d3513fd2b977df55e--gmp--6.2.1_1.arm64_big_sur.bottle.tar.gz
==> Reinstalling gmp
==> Pouring gmp--6.2.1_1.arm64_big_sur.bottle.tar.gz
🍺  /opt/homebrew/Cellar/gmp/6.2.1_1: 21 files, 3.3MB

gcc -bundle -undefined dynamic_lookup -arch x86_64 -g build/temp.macosx-10.9-x86_64-3.9/src/curve.o build/temp.macosx-10.9-x86_64-3.9/src/curveMath.o build/temp.macosx-10.9-x86_64-3.9/src/point.o -lgmp -o build/lib.macosx-10.9-x86_64-3.9/fastecdsa/curvemath.cpython-39-darwin.so
    ld: library not found for -lgmp
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    error: command '/usr/bin/gcc' failed with exit code 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/martriay/cairo_venv/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/m3/c_rs0zls4bg6k2d328r293kr0000gn/T/pip-install-5v_4xzlj/fastecdsa_c2411a6e2b5c4ee6af8fde06bf939490/setup.py'"'"'; __file__='"'"'/private/var/folders/m3/c_rs0zls4bg6k2d328r293kr0000gn/T/pip-install-5v_4xzlj/fastecdsa_c2411a6e2b5c4ee6af8fde06bf939490/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/m3/c_rs0zls4bg6k2d328r293kr0000gn/T/pip-record-flm9jcii/install-record.txt --single-version-externally-managed --compile --install-headers /Users/martriay/cairo_venv/include/site/python3.9/fastecdsa Check the logs for full command output.

I'm honestly a bit tired of trying. I have it running on a dockerized ubuntu, it's not perfect but at least I'm not wasting more time with this.

[AntonKueltz]

Sorry to hear that this is still broken for you on the M1 chips. I have no way to repro so I'm afraid I can't help much here. The only thing I can think of is that GMP appears to be installing for the ARM architecture (Pouring gmp--6.2.1_1.arm64_big_sur.bottle.tar.gz), whereas GCC is compiling against the x86 architecture (-arch x86_64). This seems bound to cause issues.

[martriay]

@AntonKueltz thanks for trying! I have a good-enough solution at the moment (spinning up a docker), so I'm good. Let's hope for this to work in the future.

[justinba1010]

Hi @martriay,

Ahh, I think I fixed that for another library. So the fix for that gmp issue is,

arch -arm64 brew install gmp

Hope that helps!

[justinba1010]

I would go ahead and alias brew install to arch -arm64 brew install, I've made the mistake of forgetting too many times to count.

Sorry for the delay as well, I don't really use github too much as I no longer use it for work.

[martriay]

Thanks. I did a bit of everything proposed here by using arm64, compiler flags, etc:

arch -arm64 brew install gmp

CFLAGS=-I/opt/homebrew/opt/gmp/include LDFLAGS=-L/opt/homebrew/opt/gmp/lib python3 -m pip install --no-binary :all: --no-use-pep517 fastecdsa

And although installation succeeded, execution fails:

  File "/Users/martriay/dev/openzeppelin/cairo-contracts/env/lib/python3.9/site-packages/fastecdsa/point.py", line 1, in <module>
    from fastecdsa import curvemath
ImportError: dlopen(/Users/martriay/dev/openzeppelin/cairo-contracts/env/lib/python3.9/site-packages/fastecdsa/curvemath.cpython-39-darwin.so, 2): Symbol not found: ___gmpz_add
  Referenced from: /Users/martriay/dev/openzeppelin/cairo-contracts/env/lib/python3.9/site-packages/fastecdsa/curvemath.cpython-39-darwin.so
  Expected in: flat namespace
 in /Users/martriay/dev/openzeppelin/cairo-contracts/env/lib/python3.9/site-packages/fastecdsa/curvemath.cpython-39-darwin.so

[justinba1010]

Can you try without the CFLAGS and LDFLAGS? That and if that fails try reinstalling gmp and it's dependents. And trying without the flags. I think pip is smart enough to know how to compile these things if you install gmp with arm64 arch, and the flags may be overriding(also it looks like those flags are not necessarily where the gmp lib goes as for me it is:

/opt/homebrew/include/gmp.h
/opt/homebrew/Cellar/gmp/6.2.1_1/include/gmp.h

)
So I would just get rid of those flags and try again.

[martriay]

Yes, it fails without them. Tried uninstalling gmp and creating a whole new venv before too.

    /usr/local/include/stdlib.h:359:13: note: insert '_Nonnull' if the pointer should never be null
    extern char *suboptarg;         /* getsubopt(3) external variable */
                ^
                  _Nonnull
    180 warnings generated.
    gcc -bundle -undefined dynamic_lookup -arch x86_64 -g build/temp.macosx-10.9-x86_64-3.9/src/curve.o build/temp.macosx-10.9-x86_64-3.9/src/curveMath.o build/temp.macosx-10.9-x86_64-3.9/src/point.o -lgmp -o build/lib.macosx-10.9-x86_64-3.9/fastecdsa/curvemath.cpython-39-darwin.so
    ld: library not found for -lgmp
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    error: command '/usr/bin/gcc' failed with exit code 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/martriay/dev/openzeppelin/cairo-contracts/env/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/m3/c_rs0zls4bg6k2d328r293kr0000gn/T/pip-install-212875o5/fastecdsa_e6c1c9f4740848e79325926bc03d88c2/setup.py'"'"'; __file__='"'"'/private/var/folders/m3/c_rs0zls4bg6k2d328r293kr0000gn/T/pip-install-212875o5/fastecdsa_e6c1c9f4740848e79325926bc03d88c2/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/m3/c_rs0zls4bg6k2d328r293kr0000gn/T/pip-record-ov2rq6lk/install-record.txt --single-version-externally-managed --compile --install-headers /Users/martriay/dev/openzeppelin/cairo-contracts/env/include/site/python3.9/fastecdsa Check the logs for full command output.

I'm impressed with your perseverance.

[jw122]

This worked for me:

CFLAGS=-I/opt/homebrew/opt/gmp/include LDFLAGS=-L/opt/homebrew/opt/gmp/lib pip install fastecdsa

This also worked for me, thank you @milancermak!
I'm on macOS big sur 11.2.3

[Mrtrl]

This worked for me:

CFLAGS=-I/opt/homebrew/opt/gmp/include LDFLAGS=-L/opt/homebrew/opt/gmp/lib pip install fastecdsa

This also worked for me!! Thanks

[Rohith-sreedharan]

This worked Dude, thanks for that

[justinba1010]

Can you add more details @Rohith-sreedharan? Might be helpful for those stumbling into this.

[neilmi]

hey guys,
i had this exact same problem on mac monterey m1
the way i solved it:

1. uninstalled python completely
2. installed python3.x exactly like as this article says: https://diewland.medium.com/how-to-install-python-3-7-on-macbook-m1-87c5b0fcb3b5
3. ibrew install gmp
4. then it should work :)

@martriay

[dhvanipa]

This worked for me:

CFLAGS=-I/opt/homebrew/opt/gmp/include LDFLAGS=-L/opt/homebrew/opt/gmp/lib pip install fastecdsa

This didn't work for me on an Intel chip. Instead I had to change the path of gmp to where it actually is (ie where homebrew installs it)

CFLAGS=-I/usr/local/Cellar/gmp/6.2.1_1/include LDFLAGS=-L/usr/local/Cellar/gmp/6.2.1_1/lib pip install fastecdsa

[AntonKueltz]

Correct, the thread was specific to M1 architecture Macs, which installs to a different path. I recommend using brew --prefix to abstract away the install directory. For example -

$ echo "$(brew --prefix)/lib"
/opt/homebrew/lib

[dolores-park]

was able to make this work on M1, by

1. ibrew install gmp as suggested by @neilmi
2. python3 -m pip install --no-binary :all: fastecdsa as suggested by @justinba1010

[b1aafulei]

In case anyone gets a similar error on an Intel mac

      In file included from src/curve.c:1:
      src/curve.h:4:10: fatal error: 'gmp.h' file not found
      #include "gmp.h"
               ^~~~~~~
      1 error generated.
      error: command '/usr/local/bin/clang' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for fastecdsa

Here is how I solved it -

brew install gmp
# just to make sure gmp is linked
brew unlink gmp && brew link gmp
CFLAGS=-I/usr/local/Cellar/gmp/6.3.0/include LDFLAGS=-L/usr/local/Cellar/gmp/6.3.0/lib pip install fastecdsa

[MarcoDotIO]

For context, this was apart of my issue with installing bitcoinlib. I made sure the CFlag and LDFlag variables were set when install fastecdsa. However, when attempting to use fastecdsa, I get this error:

Traceback (most recent call last):
  File "/Volumes/OpenDive/mastering-bitcoin/HelloWorld.py", line 1, in <module>
    from bitcoinlib.wallets import Wallet
  File "/Users/marcodotio/Library/Python/3.12/lib/python/site-packages/bitcoinlib/__init__.py", line 22, in <module>
    import bitcoinlib.keys
  File "/Users/marcodotio/Library/Python/3.12/lib/python/site-packages/bitcoinlib/keys.py", line 33, in <module>
    from fastecdsa import _ecdsa
ImportError: dlopen(/Users/marcodotio/Library/Python/3.12/lib/python/site-packages/fastecdsa/_ecdsa.cpython-312-darwin.so, 0x0002): symbol not found in flat namespace '___gmpz_add'

[AntonKueltz]

Do you have a minimal example you can share, e.g. HelloWorld.py of how you're using bicoinlib (and by extension this library)? In general the _ecdsa package should not be used directly by external packages, the usage by the importing library may be causing issues given the way this library is currently set up.