Skip to content
This repository has been archived by the owner on Dec 11, 2024. It is now read-only.

error: building the downstream libh3 C library using the Solana SDK via cmake #30

Closed
johhnsmmith198 opened this issue Jan 19, 2022 · 23 comments

Comments

@johhnsmmith198
Copy link

Hi,
solana has this helloworld program that you can build (and deploy on a validator): https://github.com/solana-labs/example-helloworld

It works well before adding the h3ron library.

adding this dependency:

h3ron = "0.13.0"

and running:

 npm run build:program-rust

would generate this error

Error: Function _ZN15crossbeam_epoch8internal6Global7collect17hdc7ca3f69bd0b12bE Stack offset of -4144 exceeded max offset of -4096 by 48 bytes, please minimize large stack variables
*** stack smashing detected ***: terminated
error: could not compile `crossbeam-epoch`

Caused by:
  process didn't exit successfully: `rustc --crate-name crossbeam_epoch --edition=2018 /home/pc/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/crossbeam-epoch-0.9.5/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg 'feature="alloc"' --cfg 'feature="lazy_static"' --cfg 'feature="std"' -C metadata=1bb560ac1e04b0d7 -C extra-filename=-1bb560ac1e04b0d7 --out-dir /home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/deps --target bpfel-unknown-unknown -L dependency=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/deps -L dependency=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/release/deps --extern cfg_if=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/deps/libcfg_if-1643e4a2c6efb510.rmeta --extern crossbeam_utils=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/deps/libcrossbeam_utils-c2f18398aedce3c4.rmeta --extern lazy_static=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/deps/liblazy_static-327a97971696967c.rmeta --extern memoffset=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/deps/libmemoffset-1762d3769b1ca8cc.rmeta --extern scopeguard=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/deps/libscopeguard-83203493741a9def.rmeta --cap-lints allow -C lto=no` (signal: 6, SIGABRT: process abort signal)
warning: build failed, waiting for other jobs to finish...
error: build failed

Could you please help?

@nmandery
Copy link
Owner

Hi,

Seems this is caused by the crossbeam-epoch crate. h3ron does not use that library directly but only indirectly via rayon. There was a bug until recently (see #29) where h3ron pulled in rayon regardless if the use-rayon feature was activated. This should now be fixed in the master branch, but has not been released yet.

Try switching your dependency from the crates.io version "0.13" to latest master from git and see if the problem persists.

@johhnsmmith198
Copy link
Author

Thanks for your help
I gave it a try by changing this on the Cargo.toml

h3ron = { git = "https://github.com/nmandery/h3ron.git"}

I get this error now


   Compiling h3ron-h3-sys v0.12.0 (https://github.com/nmandery/h3ron.git#ab3a27f3)
error: failed to run custom build command for `h3ron-h3-sys v0.12.0 (https://github.com/nmandery/h3ron.git#ab3a27f3)`

Caused by:
  process didn't exit successfully: `/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/release/build/h3ron-h3-sys-1ab6cd26d7e5abe9/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-changed=libh3
  CMAKE_TOOLCHAIN_FILE_bpfel-unknown-unknown = None
  CMAKE_TOOLCHAIN_FILE_bpfel_unknown_unknown = None
  TARGET_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_bpfel-unknown-unknown = None
  CMAKE_GENERATOR_bpfel_unknown_unknown = None
  TARGET_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_bpfel-unknown-unknown = None
  CMAKE_PREFIX_PATH_bpfel_unknown_unknown = None
  TARGET_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_bpfel-unknown-unknown = None
  CMAKE_bpfel_unknown_unknown = None
  TARGET_CMAKE = None
  CMAKE = None
  running: "cmake" "/home/pc/.cargo/git/checkouts/h3ron-31217b227e8cd3a6/ab3a27f/h3ron-h3-sys/libh3" "-DBUILD_BENCHMARKS=OFF" "-DBUILD_FILTERS=OFF" "-DBUILD_GENERATORS=OFF" "-DBUILD_TESTING=OFF" "-DENABLE_COVERAGE=OFF" "-DENABLE_DOCS=OFF" "-DENABLE_FORMAT=OFF" "-DENABLE_LINTING=OFF" "-DCMAKE_INSTALL_PREFIX=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown" "-DCMAKE_C_COMPILER=/home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC" "-DCMAKE_CXX_COMPILER=/usr/bin/c++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown" "-DCMAKE_ASM_COMPILER=/home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang" "-DCMAKE_BUILD_TYPE=Release"
  -- The C compiler identification is Clang 13.0.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - failed
  -- Check for working C compiler: /home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang
  -- Check for working C compiler: /home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang - broken
  -- Configuring incomplete, errors occurred!
  See also "/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeOutput.log".
  See also "/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeError.log".

  --- stderr
  CMake Error at /usr/local/share/cmake-3.22/Modules/CMakeTestCCompiler.cmake:69 (message):
    The C compiler

      "/home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang"

    is not able to compile a simple test program.

    It fails with the following output:

      Change Dir: /home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp

      Run Build Command(s):/usr/bin/make -f Makefile cmTC_006a3/fast && /usr/bin/make  -f CMakeFiles/cmTC_006a3.dir/build.make CMakeFiles/cmTC_006a3.dir/build
      make[1]: Entering directory '/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp'
      Building C object CMakeFiles/cmTC_006a3.dir/testCCompiler.c.o
      /home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang   -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown  -std=gnu99 -MD -MT CMakeFiles/cmTC_006a3.dir/testCCompiler.c.o -MF CMakeFiles/cmTC_006a3.dir/testCCompiler.c.o.d -o CMakeFiles/cmTC_006a3.dir/testCCompiler.c.o -c /home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp/testCCompiler.c
      Linking C executable cmTC_006a3
      /usr/local/bin/cmake -E cmake_link_script CMakeFiles/cmTC_006a3.dir/link.txt --verbose=1
      /home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang  -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown  -rdynamic CMakeFiles/cmTC_006a3.dir/testCCompiler.c.o -o cmTC_006a3
      /usr/bin/ld: unknown architecture of input file `CMakeFiles/cmTC_006a3.dir/testCCompiler.c.o' is incompatible with i386:x86-64 output
      collect2: error: ld returned 1 exit status
      clang-13: error: linker (via gcc) command failed with exit code 1 (use -v to see invocation)
      make[1]: *** [CMakeFiles/cmTC_006a3.dir/build.make:100: cmTC_006a3] Error 1
      make[1]: Leaving directory '/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp'
      make: *** [Makefile:127: cmTC_006a3/fast] Error 2





    CMake will not be able to correctly generate this project.
  Call Stack (most recent call first):
    CMakeLists.txt:59 (project)


  thread 'main' panicked at '
  command did not execute successfully, got: exit status: 1

  build script failed, must exit now', /home/pc/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/cmake-0.1.46/src/lib.rs:974:5
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed

@johhnsmmith198
Copy link
Author

johhnsmmith198 commented Jan 19, 2022

Just so you know this is what the npm command that I am running is doing (cargo build-bpf):

npm run build:program-rust

> helloworld@0.0.1 build:program-rust /home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld
> cargo build-bpf --manifest-path=./src/program-rust/Cargo.toml --bpf-out-dir=dist/program

BPF SDK: /home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf
cargo-build-bpf child: rustup toolchain list -v
cargo-build-bpf child: cargo +bpf build --target bpfel-unknown-unknown --release
   Compiling proc-macro2 v1.0.27
   Compiling unicode-xid v0.2.2
   Compiling syn v1.0.73
   Compiling version_check v0.9.3
   Compiling serde v1.0.126
   ...

@nmandery
Copy link
Owner

Seems the crossbeam issue is solved - great.

Now the problem is building the downstream libh3 C library using the Solana SDK via cmake. It appears cmake correctly detects the C compiler of the SDK, but is not able to find a suitable linker. So it falls back to your systems default linker, but that one does not support the bpfel architecture. I suppose there should be a suitable linker with in the SDK in the same directory as the compiler found by cmake.

Messing with cmake can be tricky, but https://stackoverflow.com/questions/1867745/cmake-use-a-custom-linker gives some hints. The -D flags can also be set using environment variables like https://cmake.org/cmake/help/latest/envvar/CFLAGS.html

@johhnsmmith198
Copy link
Author

@nmandery I think I am getting onto something.
I was trying to step back and build and run the original h3 C library.
I was getting into some issues while running the example and created an issue for that: uber/h3#566
Someone was able to find that the linker needed -lh3 but also -lm for the math library.
Could this be related to my other linker issue above ?

@johhnsmmith198
Copy link
Author

Also I am seeing that I sudo make install the library called h3 but it seems h3ron is calling it libh3 (that I didn't install actually).

@nmandery
Copy link
Owner

nmandery commented Jan 21, 2022

The h3ron-h3-sys crate compiles its own copy of the h3 library when the crate is build. This happens here https://github.com/nmandery/h3ron/blob/master/h3ron-h3-sys/build.rs#L15 and is done using cmake. The reason for this is to be able to do a static linking of the rust code to the C library and to be independent of h3 being installed with the system libraries.

That means all information about the environment we are compiling for must go there. That works great for the Solana-specfic compiler as this is set in the CC environment variable. So cmake finds this and uses this. But there is not such thing for the linker.

Please have a look in /home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/ - there should be something called *ld*, thats the linker we need here. As this is LLVM, its certainly called lld.

cmake supports the CFLAGS environment variable, so we can use this to pass this information to cmake.

First run cargo clean in your project as cmake does lots of caching and will probably not use the new environment variable as long as the caches exist.

Try to build your project with something like

cargo clean
export CFLAGS="-fuse-ld=/home/pc/.local/share/solana/install/releases/1.9.2/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/lld"
npm run build:program-rust

Thats a quite long chain of build tools calling build tools - so hopefully this is of help to you.

libm should be contained in the LLVM of the SDK, I suppose.

Some related links:

@johhnsmmith198
Copy link
Author

johhnsmmith198 commented Jan 21, 2022

I have switch to use 1.9.4 I hope its ok.
I double checked the paths

=> ls /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin
clang  clang++  clang-13  clang-cl  clang-cpp  ld64.lld  ld.lld  llc  lld  lld-link  llvm-ar  llvm-objcopy  llvm-objdump  llvm-readelf  llvm-readobj

all good

npm run clean
 export CFLAGS="-fuse-ld=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/lld"
npm run build:program-rust

the output

 Run Build Command(s):/usr/bin/make -f Makefile cmTC_8b0c0/fast && /usr/bin/make  -f CMakeFiles/cmTC_8b0c0.dir/build.make CMakeFiles/cmTC_8b0c0.dir/build
      make[1]: Entering directory '/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp'
      Building C object CMakeFiles/cmTC_8b0c0.dir/testCCompiler.c.o
      /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang   -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown -fuse-ld=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/lld  -std=gnu99 -MD -MT CMakeFiles/cmTC_8b0c0.dir/testCCompiler.c.o -MF CMakeFiles/cmTC_8b0c0.dir/testCCompiler.c.o.d -o CMakeFiles/cmTC_8b0c0.dir/testCCompiler.c.o -c /home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp/testCCompiler.c
      clang-13: warning: argument unused during compilation: '-fuse-ld=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/lld' [-Wunused-command-line-argument]
      Linking C executable cmTC_8b0c0
      /usr/local/bin/cmake -E cmake_link_script CMakeFiles/cmTC_8b0c0.dir/link.txt --verbose=1
      /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang  -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown -fuse-ld=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/lld  -rdynamic CMakeFiles/cmTC_8b0c0.dir/testCCompiler.c.o -o cmTC_8b0c0
      gcc: error: unrecognized command line option ‘-fuse-ld=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/lld’
      clang-13: error: linker (via gcc) command failed with exit code 1 (use -v to see invocation)
      make[1]: *** [CMakeFiles/cmTC_8b0c0.dir/build.make:100: cmTC_8b0c0] Error 1
      make[1]: Leaving directory '/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp'
      make: *** [Makefile:127: cmTC_8b0c0/fast] Error 2

it says: argument unused during compilation

@nmandery
Copy link
Owner

nmandery commented Jan 21, 2022

Nice that clang gives us this warning.

Try to use the LDFLAGS (https://cmake.org/cmake/help/latest/envvar/LDFLAGS.html) environment variable instead of CFLAGS. Dont forget the cargo clean

@johhnsmmith198
Copy link
Author

johhnsmmith198 commented Jan 21, 2022

Thanks all good I am cleaning each time with npm run clean
I see it's clean because it restart downloading everything from scratch

> cargo build-bpf --manifest-path=./src/program-rust/Cargo.toml --bpf-out-dir=dist/program

BPF SDK: /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf
cargo-build-bpf child: rustup toolchain list -v
cargo-build-bpf child: cargo +bpf build --target bpfel-unknown-unknown --release
   Compiling proc-macro2 v1.0.27
   Compiling unicode-xid v0.2.2
   Compiling syn v1.0.73
...

I have open a new window to get rid of the CFLAGS and only have the LDFLAGS defined
same

 gcc: error: unrecognized command line option ‘-fuse-ld=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/lld’
      clang-13: error: linker (via gcc) command failed with exit code 1 (use -v to see invocation)
    

also this is the beginning of the trace. I see clang - broken


   Compiling h3ron-h3-sys v0.12.0 (https://github.com/nmandery/h3ron.git#ab3a27f3)
error: failed to run custom build command for h3ron-h3-sys v0.12.0 (https://github.com/nmandery/h3ron.git#ab3a27f3)

Caused by:
 process didn't exit successfully: /home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/release/build/h3ron-h3-sys-1ab6cd26d7e5abe9/build-script-build (exit status: 101)
 --- stdout
 cargo:rerun-if-changed=libh3
 CMAKE_TOOLCHAIN_FILE_bpfel-unknown-unknown = None
 CMAKE_TOOLCHAIN_FILE_bpfel_unknown_unknown = None
 TARGET_CMAKE_TOOLCHAIN_FILE = None
 CMAKE_TOOLCHAIN_FILE = None
 CMAKE_GENERATOR_bpfel-unknown-unknown = None
 CMAKE_GENERATOR_bpfel_unknown_unknown = None
 TARGET_CMAKE_GENERATOR = None
 CMAKE_GENERATOR = None
 CMAKE_PREFIX_PATH_bpfel-unknown-unknown = None
 CMAKE_PREFIX_PATH_bpfel_unknown_unknown = None
 TARGET_CMAKE_PREFIX_PATH = None
 CMAKE_PREFIX_PATH = None
 CMAKE_bpfel-unknown-unknown = None
 CMAKE_bpfel_unknown_unknown = None
 TARGET_CMAKE = None
 CMAKE = None
 running: "cmake" "/home/pc/.cargo/git/checkouts/h3ron-31217b227e8cd3a6/ab3a27f/h3ron-h3-sys/libh3" "-DBUILD_BENCHMARKS=OFF" "-DBUILD_FILTERS=OFF" "-DBUILD_GENERATORS=OFF" "-DBUILD_TESTING=OFF" "-DENABLE_COVERAGE=OFF" "-DENABLE_DOCS=OFF" "-DENABLE_FORMAT=OFF" "-DENABLE_LINTING=OFF" "-DCMAKE_INSTALL_PREFIX=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown" "-DCMAKE_C_COMPILER=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC" "-DCMAKE_CXX_COMPILER=/usr/bin/c++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown" "-DCMAKE_ASM_COMPILER=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang" "-DCMAKE_BUILD_TYPE=Release"
 -- The C compiler identification is Clang 13.0.0
 -- Detecting C compiler ABI info
 -- Detecting C compiler ABI info - failed
 -- Check for working C compiler: /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang
 -- Check for working C compiler: /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang - broken
 -- Configuring incomplete, errors occurred!

@nmandery
Copy link
Owner

gcc calling clang? Things are getting confusing ;) Also interesting that the flag is not understood.

One more attempt, if this does not work, I am slowly running out of ideas:

unset CFLAGS
unset LDFLAGS

# search in the SDK first
export PATH="/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/:$PATH"

cargo clean

# ... then build

@nmandery
Copy link
Owner

That bpf.mk file here could give more hints on expected environment settings for C/C++ builds to succeed: https://docs.solana.com/developing/on-chain-programs/developing-c#project-layout

@johhnsmmith198
Copy link
Author

well back to the beginning I think. Thanks for your help, really appreciate it

 Run Build Command(s):/usr/bin/make -f Makefile cmTC_b4485/fast && /usr/bin/make  -f CMakeFiles/cmTC_b4485.dir/build.make CMakeFiles/cmTC_b4485.dir/build
      make[1]: Entering directory '/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp'
      Building C object CMakeFiles/cmTC_b4485.dir/testCCompiler.c.o
      /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang   -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown  -std=gnu99 -MD -MT CMakeFiles/cmTC_b4485.dir/testCCompiler.c.o -MF CMakeFiles/cmTC_b4485.dir/testCCompiler.c.o.d -o CMakeFiles/cmTC_b4485.dir/testCCompiler.c.o -c /home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp/testCCompiler.c
      Linking C executable cmTC_b4485
      /usr/local/bin/cmake -E cmake_link_script CMakeFiles/cmTC_b4485.dir/link.txt --verbose=1
      /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang  -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown  -rdynamic CMakeFiles/cmTC_b4485.dir/testCCompiler.c.o -o cmTC_b4485
      /usr/bin/ld: unknown architecture of input file `CMakeFiles/cmTC_b4485.dir/testCCompiler.c.o' is incompatible with i386:x86-64 output
      collect2: error: ld returned 1 exit status
      clang-13: error: linker (via gcc) command failed with exit code 1 (use -v to see invocation)
      make[1]: *** [CMakeFiles/cmTC_b4485.dir/build.make:100: cmTC_b4485] Error 1
      make[1]: Leaving directory '/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out/build/CMakeFiles/CMakeTmp'
      make: *** [Makefile:127: cmTC_b4485/fast] Error 2

@johhnsmmith198
Copy link
Author

Solana allows you to build C program or Rust program. This would be for the C program.
https://github.com/solana-labs/example-helloworld/blob/master/src/program-c/makefile

When I say program I mean the solana smart contract

@nmandery
Copy link
Owner

No worries.

I have to confess I do not understand much of the blockchain terminology ;)

Looking at https://github.com/solana-labs/solana/blob/master/sdk/bpf/c/bpf.mk it seems to contain the flags needed to compile C code for the pbf architecture. Maybe the compilation here works when you extract flags from the files BPF_C_FLAGS and BPF_LLD_FLAGS and put them in CFLAGS and LDFLAGS.

In case that gives conflicts with the rust compilation, maybe putting them directly into h3ron-h3-sys build.rs allows to circumvent that.

Thats the command to compile C for solana: https://github.com/solana-labs/solana/blob/master/sdk/bpf/c/bpf.mk#L159
and thats how it appears to be linked: https://github.com/solana-labs/solana/blob/master/sdk/bpf/c/bpf.mk#L187

In the example you linked that bpf.mk file gets included and applied to compile the example. So the solution is somewhere in there.

@johhnsmmith198 johhnsmmith198 changed the title error: could not compile crossbeam-epoch while including this lib in solana helloworld example error: building the downstream libh3 C library using the Solana SDK via cmake Jan 24, 2022
@johhnsmmith198
Copy link
Author

johhnsmmith198 commented Jan 24, 2022

@nmandery
I decided to step back a little. I cloned h3ron and then ran those commands

export PATH="/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/:$PATH"
➜  funspace cd h3ron
➜  h3ron git:(master) ls
Cargo.toml  CHANGES.md  CONTRIBUTING.md  data  h3ron  h3ron-graph  h3ron-h3-sys  h3ron-ndarray  justfile  LICENSE-MIT  README.md
➜  h3ron git:(master) mkdir build
➜  h3ron git:(master) cd build
➜  build git:(master) "cmake" "/home/pc/.cargo/git/checkouts/h3ron-31217b227e8cd3a6/ab3a27f/h3ron-h3-sys/libh3" \
"-DBUILD_BENCHMARKS=OFF" \
"-DBUILD_FILTERS=OFF" \
"-DBUILD_GENERATORS=OFF" \
"-DBUILD_TESTING=OFF" \
"-DENABLE_COVERAGE=OFF" \
"-DENABLE_DOCS=OFF" \
"-DENABLE_FORMAT=OFF" \
"-DENABLE_LINTING=OFF" \
"-DCMAKE_INSTALL_PREFIX=/home/pc/Documents/funspace/crypto-world/lab/solanaworkspace/example-helloworld/src/program-rust/target/bpfel-unknown-unknown/release/build/h3ron-h3-sys-df478ba73015eab9/out" \
"-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-none -v -fuse-ld=lld" \
"-DCMAKE_C_COMPILER=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang" \
"-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC " \
"-DCMAKE_CXX_COMPILER=/usr/bin/c++" \
"-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown" \
"-DCMAKE_ASM_COMPILER=/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang" \
"-DCMAKE_BUILD_TYPE=Release"

-- The C compiler identification is Clang 13.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - failed
-- Check for working C compiler: /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang
-- Check for working C compiler: /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang - broken

CMake Error at /usr/local/share/cmake-3.22/Modules/CMakeTestCCompiler.cmake:69 (message):
  The C compiler

    "/home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /home/pc/Documents/funspace/h3ron/CMakeFiles/CMakeTmp

    Run Build Command(s):/usr/bin/make -f Makefile cmTC_dab46/fast && /usr/bin/make  -f CMakeFiles/cmTC_dab46.dir/build.make CMakeFiles/cmTC_dab46.dir/build
    make[1]: Entering directory '/home/pc/Documents/funspace/h3ron/CMakeFiles/CMakeTmp'
    Building C object CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o
    /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang   -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-none -v -fuse-ld=lld  -std=gnu99 -MD -MT CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o -MF CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o.d -o CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o -c /home/pc/Documents/funspace/h3ron/CMakeFiles/CMakeTmp/testCCompiler.c
    clang version 13.0.0 (https://github.com/solana-labs/llvm-project.git 9a51f4037220994af74488ba064efc4e96ee08c0)
    Target: bpfel-unknown-none
    Thread model: posix
    InstalledDir: /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin
    clang-13: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
     (in-process)
     "/home/pc/.cache/solana/v1.21/bpf-tools/llvm/bin/clang-13" -cc1 -triple bpfel-unknown-none -emit-obj -mrelax-all --mrelax-relocations -disable-free -disable-llvm-verifier -discard-value-names -main-file-name testCCompiler.c -mrelocation-model pic -pic-level 2 -fhalf-no-semantic-interposition -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -debugger-tuning=gdb -v -ffunction-sections -fdata-sections -fcoverage-compilation-dir=/home/pc/Documents/funspace/h3ron/CMakeFiles/CMakeTmp -resource-dir /home/pc/.cache/solana/v1.21/bpf-tools/llvm/lib/clang/13.0.0 -dependency-file CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o.d -MT CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o -sys-header-deps -std=gnu99 -fdebug-compilation-dir=/home/pc/Documents/funspace/h3ron/CMakeFiles/CMakeTmp -ferror-limit 19 -fgnuc-version=4.2.1 -faddrsig -o CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o -x c /home/pc/Documents/funspace/h3ron/CMakeFiles/CMakeTmp/testCCompiler.c
    clang -cc1 version 13.0.0 based upon LLVM 13.0.0-rust-dev default target x86_64-unknown-linux-gnu
    #include "..." search starts here:
    #include <...> search starts here:
     /usr/local/include
     /home/pc/.cache/solana/v1.21/bpf-tools/llvm/lib/clang/13.0.0/include
     /usr/include
    End of search list.
    Linking C executable cmTC_dab46
    /usr/local/bin/cmake -E cmake_link_script CMakeFiles/cmTC_dab46.dir/link.txt --verbose=1
    /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin/clang  -ffunction-sections -fdata-sections -fPIC --target=bpfel-unknown-unknown -v -fuse-ld=lld  -rdynamic CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o -o cmTC_dab46
    clang version 13.0.0 (https://github.com/solana-labs/llvm-project.git 9a51f4037220994af74488ba064efc4e96ee08c0)
    Target: bpfel-unknown-none
    Thread model: posix
    InstalledDir: /home/pc/.local/share/solana/install/releases/1.9.4/solana-release/bin/sdk/bpf/dependencies/bpf-tools/llvm/bin
     "/usr/bin/gcc" -fuse-ld=lld -rdynamic -o cmTC_dab46 CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o
    ld.lld: error: CMakeFiles/cmTC_dab46.dir/testCCompiler.c.o is incompatible with elf64-x86-64
    collect2: error: ld returned 1 exit status
    clang-13: error: linker (via gcc) command failed with exit code 1 (use -v to see invocation)
    make[1]: *** [CMakeFiles/cmTC_dab46.dir/build.make:100: cmTC_dab46] Error 1
    make[1]: Leaving directory '/home/pc/Documents/funspace/h3ron/CMakeFiles/CMakeTmp'
    make: *** [Makefile:127: cmTC_dab46/fast] Error 2





  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:59 (project)


-- Configuring incomplete, errors occurred!
See also "/home/pc/Documents/funspace/h3ron/build/CMakeFiles/CMakeOutput.log".
See also "/home/pc/Documents/funspace/h3ron/build/CMakeFiles/CMakeError.log".

running the same cmake command that solana is doing is triggering the error.
meaning its not really solana or rust the problem, it is really this cmake command that doesn't work with h3ron
Note I added -v -fuse-ld=lld to the DCMAKE_C_FLAGS

@nmandery
Copy link
Owner

Would it help when we implement a feature-flag to compile libh3 without cmake using the cc crate?

@johhnsmmith198
Copy link
Author

johhnsmmith198 commented Jan 24, 2022

@nmandery I would like to ask a few questions.
When a rust program add h3ron as a dependency. h3ron is being downloaded and a command is ran to build the underlying libh3 C library.
Where in your code is this building happening?
How can the parent program (in this case example-helloworld ) can pass their own variables like -DCMAKE_C_COMPILER -DCMAKE_ASM_FLAGS etc... to build the underlying c bindings library?
In this case, it seems that solana want the underlying c library to be built with BPF right ?

@nmandery
Copy link
Owner

The libh3 C-sources are actually bundled in the h3ron-h3-sys crate when its downloaded from crates.io. The build of the C sources is done by the build.rs of the h3ron-h3-sys crate.

There is no possibility to pass custom flags to the cmake command - I am not aware of any way to do such things using cargo. The reason cmake is often used is that it figures out the correct settings for the host-system itself. Thats also the reason why cmake compiles the "simple test program" (see the cmake output) - it does so simply to validate its able to build. The basis for cmake being able to find the compiler etc are environment variables - these here are the supported ones. These can be used to pass information to cmake in a way which should also work with cargo being in between. On your system for example CC seems to be set so cmake finds the clang of the solana SDK. The environment-variable for the ASM_FLAGS seems to be this one: https://cmake.org/cmake/help/latest/envvar/ASM_DIALECTFLAGS.html

The tricky part here is that cmake is not supposed to build for the host system, but instead should be cross compile for solana. CMake has documentation how to use it for cross compilation: https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html#cross-compiling-toolchain - setting CMAKE_CROSSCOMPILING to disable the compiler checks could be a step in the right direction.

In case this can not be resolved using environment variables, perhaps forking the repo and modifying the build.rs file directly to make it work with solana would be an appropriate solution.

@johhnsmmith198
Copy link
Author

johhnsmmith198 commented Jan 27, 2022

@nmandery thanks for your help.
What would the build.rs look like if I was using the system h3 that I built and installed previously, instead of rebuilding it?

@nmandery
Copy link
Owner

You could leave out lines 12 and 15 - everything else you will still need though. You also got to change the dst_path to point to your manually build library and h3api.h header. The println!("cargo:rus.... statements give instructions to rustc on how to link the crate - see the documentation here.

You can choose between two kinds of linkage, static and dynamic. I am using static linking to produce a single binary with libh3 being contained within. I am also not sure if solana and bpf support dynamic linking.

@nmandery
Copy link
Owner

#47 drops the requirement for cmake and compiles h3 with the cc crate

@nmandery
Copy link
Owner

Closing as there is now h3o which provides a rust implementation of H3.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants