Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

macos: ld: library not found for -lstdc++ when compiling no_deps example #247

Open
nikclayton-dfinity opened this issue Oct 13, 2020 · 11 comments

Comments

@nikclayton-dfinity
Copy link

Building a fresh checkout of examples/remote/no_deps with Bazel 3.6.0 fails on MacOS with the error:

% uname -a
Darwin Niks-MBP 19.6.0 Darwin Kernel Version 19.6.0: Thu Jun 18 20:49:00 PDT 2020; root:xnu-6153.141.1~1/RELEASE_X86_64 x86_64

% bazel build :all
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: SHA256 (https://static.rust-lang.org/dist/rustfmt-1.4.18-x86_64-apple-darwin.tar.gz) = 8c5ec426f2b55cd5de1a9cd47e98c5eeb7f24b7e8d83ef60673f1718952f706c
INFO: SHA256 (https://static.rust-lang.org/dist/rust-std-1.44.0-wasm32-unknown-unknown.tar.gz) = 8e12796a0c2fb083953042218f832bdeb78da1bfaf67b9dfe3d719920084d755
INFO: Analyzed target //remote/no_deps:no_deps (25 packages loaded, 171 targets configured).
INFO: Found 1 target...
INFO: From Compiling Rust bin no_deps (1 files):
error: linking with `external/local_config_cc/cc_wrapper.sh` failed: exit code: 1
  |
  = note: "external/local_config_cc/cc_wrapper.sh" "-m64" "-L" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.0.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.1.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.2.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.3.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.4.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.5.rcgu.o" "-o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.20nzc483t9mjzk0n.rcgu.o" "-Wl,-dead_strip" "-nodefaultlibs" "-L" "external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib" "-L" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libstd-5aac390c25a3be91.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libpanic_unwind-8d95e919acff6358.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libhashbrown-b74724130e02bca1.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_alloc-0e0d29142d1c9272.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libbacktrace-f1652c422fbf7fc7.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libbacktrace_sys-3d74ad27c9a94aec.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/librustc_demangle-1255ec28ac7c09c4.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libunwind-8753dbda32b8facf.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libcfg_if-a00fd94477b6f959.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/liblibc-764e6791ca18d144.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/liballoc-a494dce1f057654b.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_core-2c78f494769c69d7.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libcore-47f37590dc106077.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-edfd79f9969978bb.rlib" "-lSystem" "-lresolv" "-lc" "-lm" "-undefined" "dynamic_lookup" "-headerpad_max_install_names" "-lstdc++" "-lm"
  = note: ld: library not found for -lstdc++
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

Reproduction recipe:

  1. Install Bazel from Homebrew, following the instructions at https://docs.bazel.build/versions/3.6.0/install-os-x.html
% brew install bazel
...
==> bazel
Bash completion has been installed to:
  /usr/local/etc/bash_completion.d

zsh completions have been installed to:
  /usr/local/share/zsh/site-functions

% bazel --version
bazel 3.6.0-homebrew
  1. Install cargo-raze
% cargo --version
cargo 1.47.0 (f3c7e066a 2020-08-28)

% cargo install cargo-raze
...
   Installed package `cargo-raze v0.5.0` (executable `cargo-raze`)
  1. Check out the cargo-raze repository.
% git clone https://github.com/google/cargo-raze.git
Cloning into 'cargo-raze'...
remote: Enumerating objects: 6, done.
remote: Counting objects: 100% (6/6), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 10825 (delta 0), reused 0 (delta 0), pack-reused 10819
Receiving objects: 100% (10825/10825), 10.23 MiB | 1.60 MiB/s, done.
Resolving deltas: 100% (7543/7543), done.

% git rev-parse HEAD
253a853d01d493bfaa35176510190aa09ec00128
  1. Attempt to build.
% cd cargo-raze/examples/remote/no_deps

% cargo raze
info: syncing channel updates for 'stable-x86_64-apple-darwin'
info: latest update on 2020-10-08, rust version 1.47.0 (18bf6b4f0 2020-10-07)
info: downloading component 'cargo'
info: downloading component 'clippy'
info: downloading component 'rust-docs'
info: downloading component 'rust-std'
info: downloading component 'rustc'
info: downloading component 'rustfmt'
info: installing component 'cargo'
info: Defaulting to 500.0 MiB unpack ram
info: installing component 'clippy'
info: installing component 'rust-docs'
info: installing component 'rust-std'
info: installing component 'rustc'
info: installing component 'rustfmt'
Loaded override settings: RazeSettings {
    workspace_path: "//remote/no_deps/cargo",
    incompatible_relative_workspace_path: true,
    target: "x86_64-unknown-linux-gnu",
    crates: {},
    gen_workspace_prefix: "remote_no_deps",
    genmode: Remote,
    output_buildfile_suffix: "BUILD.bazel",
    default_gen_buildrs: false,
}
Generated /Users/nik/github/cargo-raze/examples/remote/no_deps/cargo/remote/BUILD.bazel successfully
Generated /Users/nik/github/cargo-raze/examples/remote/no_deps/cargo/BUILD.bazel successfully
Generated /Users/nik/github/cargo-raze/examples/remote/no_deps/cargo/crates.bzl successfully

% bazel build :all
INFO: Analyzed target //remote/no_deps:no_deps (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
INFO: From Compiling Rust bin no_deps (1 files):
error: linking with `external/local_config_cc/cc_wrapper.sh` failed: exit code: 1
  |
  = note: "external/local_config_cc/cc_wrapper.sh" "-m64" "-L" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.0.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.1.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.2.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.3.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.4.rcgu.o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.no_deps.7rcbfp3g-cgu.5.rcgu.o" "-o" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps" "bazel-out/darwin-fastbuild/bin/remote/no_deps/no_deps.20nzc483t9mjzk0n.rcgu.o" "-Wl,-dead_strip" "-nodefaultlibs" "-L" "external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib" "-L" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libstd-5aac390c25a3be91.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libpanic_unwind-8d95e919acff6358.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libhashbrown-b74724130e02bca1.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_alloc-0e0d29142d1c9272.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libbacktrace-f1652c422fbf7fc7.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libbacktrace_sys-3d74ad27c9a94aec.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/librustc_demangle-1255ec28ac7c09c4.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libunwind-8753dbda32b8facf.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libcfg_if-a00fd94477b6f959.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/liblibc-764e6791ca18d144.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/liballoc-a494dce1f057654b.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_core-2c78f494769c69d7.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libcore-47f37590dc106077.rlib" "/private/var/tmp/_bazel_nik/ce05af2a155f281c5802d92be9764620/external/rust_darwin_x86_64/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-edfd79f9969978bb.rlib" "-lSystem" "-lresolv" "-lc" "-lm" "-undefined" "dynamic_lookup" "-headerpad_max_install_names" "-lstdc++" "-lm"
  = note: ld: library not found for -lstdc++
          clang: error: linker command failed with exit code 1 (use -v to see invocation)


error: aborting due to previous error

ERROR: /Users/nik/github/cargo-raze/examples/remote/no_deps/BUILD:5:12: output 'remote/no_deps/no_deps' was not created
ERROR: /Users/nik/github/cargo-raze/examples/remote/no_deps/BUILD:5:12: not all outputs were created or valid
Target //remote/no_deps:no_deps failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.596s, Critical Path: 0.48s
INFO: 2 processes: 1 internal, 1 darwin-sandbox.
FAILED: Build did NOT complete successfully
@nikclayton-dfinity
Copy link
Author

pandas-dev/pandas#23424 (comment) may be relevant.

@nikclayton-dfinity
Copy link
Author

https://developer.apple.com/documentation/xcode-release-notes/xcode-10-release-notes?preferredLanguage=occ suggests that on recent OSX libstd++ is no longer supported and libc++ should be used instead.

@nikclayton-dfinity
Copy link
Author

bazelbuild/rules_rust#226 is probably the real bug here.

@nikclayton-dfinity
Copy link
Author

Add this to the bottom of the WORKSPACE file:

# Everything below here is key to building on OSX. Load a custom Clang
# toolchain rather than whatever happens to be installed on the machine.
#
# The entry in .bazelrc automatically selects this toolchain for building.
http_archive(
    name = "com_grail_bazel_toolchain",
    url = "https://github.com/grailbio/bazel-toolchain/archive/0.5.6.zip",
    sha256 = "759e6c10e7883b5969db01b5bdb01c5518bcf7ae637972d5a109eb994a669487",
    strip_prefix = "bazel-toolchain-0.5.6",
)

load("@com_grail_bazel_toolchain//toolchain:deps.bzl", "bazel_toolchain_dependencies")
bazel_toolchain_dependencies()

load("@com_grail_bazel_toolchain//toolchain:rules.bzl", "llvm_toolchain")
llvm_toolchain(
    name = "llvm_toolchain",
    absolute_paths = True,
    llvm_version = "9.0.0",
)

load("@llvm_toolchain//:toolchains.bzl", "llvm_register_toolchains")
llvm_register_toolchains()

and create .bazelrc in the same directory as the WORKSPACE file with the following contents:

build --crosstool_top=@llvm_toolchain//:toolchain

Then everything works for the no_deps example.

@nikclayton-dfinity
Copy link
Author

complicated_cargo_library also works.

@nikclayton-dfinity
Copy link
Author

non_cratesio doesn't.

It builds, but running it produces no output.

% cd examples/remote/non_cratesio

% cargo raze

% bazel build :non_cratesio_remote
INFO: Analyzed target //remote/non_cratesio:non_cratesio_remote (44 packages loaded, 563 targets configured).
INFO: Found 1 target...
Target //remote/non_cratesio:non_cratesio_remote up-to-date:
  bazel-bin/remote/non_cratesio/non_cratesio_remote
INFO: Elapsed time: 8.716s, Critical Path: 8.37s
INFO: 28 processes: 5 internal, 23 darwin-sandbox.
INFO: Build completed successfully, 28 total actions

% RUST_LOG=info bazel run :non_cratesio_remote
INFO: Analyzed target //remote/non_cratesio:non_cratesio_remote (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //remote/non_cratesio:non_cratesio_remote up-to-date:
  bazel-bin/remote/non_cratesio/non_cratesio_remote
INFO: Elapsed time: 0.091s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action

However, running it with Cargo has the same problem:

% RUST_LOG=info cargo run
warning: unused manifest key: raze
   Compiling compile_with_bazel v0.1.0 (/Users/nik/github/cargo-raze/examples/remote/non_cratesio)
    Finished dev [unoptimized + debuginfo] target(s) in 0.73s
     Running `target/debug/non_cratesio`

so I'm not sure what's going on there.

@UebelAndre
Copy link
Collaborator

Without digging too deeply, is this a bug in cargo-raze or in rules_rust? (as per #247 (comment))

@nikclayton-dfinity
Copy link
Author

The primary bug is in rules_rust.

I think there's probably two bugs in the cargo-raze examples.

  1. They don't work out of the box on macOS, you need to do bazel build //... fails on OSX bazelbuild/rules_rust#226 (comment) in the WORKSPACE file.

The most version I have of those additions is:

http_archive(
    name = "com_grail_bazel_toolchain",
    url = "https://github.com/grailbio/bazel-toolchain/archive/0.5.6.zip",
    sha256 = "759e6c10e7883b5969db01b5bdb01c5518bcf7ae637972d5a109eb994a669487",
    strip_prefix = "bazel-toolchain-0.5.6",
)

load("@com_grail_bazel_toolchain//toolchain:deps.bzl", "bazel_toolchain_dependencies")
bazel_toolchain_dependencies()

load("@com_grail_bazel_toolchain//toolchain:rules.bzl", "llvm_toolchain")
llvm_toolchain(
    name = "llvm_toolchain",
    absolute_paths = True,
    llvm_version = "10.0.0",
)

load("@llvm_toolchain//:toolchains.bzl", "llvm_register_toolchains")
llvm_register_toolchains()
  1. non_cratesio producing no output irrespective of how it's built (macos: ld: library not found for -lstdc++ when compiling no_deps example #247 (comment)).

@UebelAndre
Copy link
Collaborator

1. They don't work out of the box on macOS, you need to do [bazelbuild/rules_rust#226 (comment)](https://github.com/bazelbuild/rules_rust/issues/226#issuecomment-626378800) in the `WORKSPACE` file.

Odd, I'm able to build the examples on MacOS just fine. Perhaps the rust rules are relying on some specific machine setup. It might be good to open a pull request to rules_rust making it's tooling more explicit so users see consistent behavior regardless of how their machines are configured (if that is indeed the case here).

2. `non_cratesio` producing no output irrespective of how it's built ([#247 (comment)](https://github.com/google/cargo-raze/issues/247#issuecomment-708230957)).

Interesting, seems to me like env_logger wasn't setup correctly? It does seem to be though. From some quick tests it doesn't seem like the logger is initializing to log Info level logs. Adding the following always panics

if !log_enabled!(log::Level::Info) {
  panic!("uh-oh");
}

This should definitely be looked at and updated. I may take a stab at it when I can find the time.

Sorry this isn't much help. But it's good to know where the issue is. You should chime in on bazelbuild/rules_rust#226 and let the owners know users are still running into that issue 😄

@keith
Copy link
Contributor

keith commented Jan 11, 2022

For folks experiencing ld: library not found for -lstdc++, this is a bug in bazel I've submitted a patch for, for more details see bazelbuild/bazel#14395 (comment)

bazel-io pushed a commit to bazelbuild/bazel that referenced this issue Jan 27, 2022
This flag has been invalid seemingly since macOS 10.9 from 2013. By
default clang remaps this back to -lc++, but if your linkopts pass
-nodefaultlibs this is not the case, in which case this invalid flag
fails the link. This happened for rust builds where the rust driver
passes this flag intentionally.

For more info see #14395 (comment)

Fixes google/cargo-raze#247 #14395 bazelbuild/rules_rust#226

Closes #14542.

PiperOrigin-RevId: 424588734
brentleyjones pushed a commit to brentleyjones/bazel that referenced this issue Feb 8, 2022
This flag has been invalid seemingly since macOS 10.9 from 2013. By
default clang remaps this back to -lc++, but if your linkopts pass
-nodefaultlibs this is not the case, in which case this invalid flag
fails the link. This happened for rust builds where the rust driver
passes this flag intentionally.

For more info see bazelbuild#14395 (comment)

Fixes google/cargo-raze#247 bazelbuild#14395 bazelbuild/rules_rust#226

Closes bazelbuild#14542.

PiperOrigin-RevId: 424588734
(cherry picked from commit 8b60c90)
Wyverald pushed a commit to bazelbuild/bazel that referenced this issue Feb 9, 2022
This flag has been invalid seemingly since macOS 10.9 from 2013. By
default clang remaps this back to -lc++, but if your linkopts pass
-nodefaultlibs this is not the case, in which case this invalid flag
fails the link. This happened for rust builds where the rust driver
passes this flag intentionally.

For more info see #14395 (comment)

Fixes google/cargo-raze#247 #14395 bazelbuild/rules_rust#226

Closes #14542.

PiperOrigin-RevId: 424588734
(cherry picked from commit 8b60c90)

Co-authored-by: Keith Smiley <keithbsmiley@gmail.com>
@cdmistman
Copy link

with the landing of bazelbuild/bazel#14750 I believe this issue can be closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants