C++ cc_binary segfaults on macos-12

[avdv]

Description of the problem / feature request:

The most basic cc hello world example produces a segfaulting binary on macos-12, when using rules_cc > 0.0.4.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

WORKSPACE:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_cc",
    sha256 = "2037875b9a4456dce4a79d112a8ae885bbc4aad968e6587dca6e64f3a0900cdf",
    strip_prefix = "rules_cc-0.0.9",
    urls = [
        "https://github.com/bazelbuild/rules_cc/releases/download/0.0.9/rules_cc-0.0.9.tar.gz",
    ],
)

load("@rules_cc//cc:repositories.bzl", "rules_cc_dependencies", "rules_cc_toolchains")

rules_cc_dependencies()

rules_cc_toolchains()

BUILD:

load("@rules_cc//cc:defs.bzl", "cc_binary")

cc_binary(
    name = "hello",
    srcs = ["hello.cc"],
)

hello.cc:

#include <iostream>

int main()
{
	std::cout << "Hello world!\n";
}

What operating system are you running Bazel on?

macos-12 (on Github Actions)

What's the output of bazel info release?

release 6.5.0

What version of rules_cc do you use? Can you paste the workspace rule used to fetch rules_cc? What other relevant dependencies does your project have?

I am using rules_cc 0.0.9 (See WORKSPACE above), also tried 0.0.5 - 0.0.8 -- all of them fail in the same way. Version 0.0.1 works (that was the version we were using, but wanted to upgrade to the latest. 0.0.4 is the last version that works)

What Bazel options do you use to trigger the issue? What C++ toolchain do you use?

No added options (just -s to see the commands).

The project is using the local_config_cc toolchain created by rules_cc.

Have you found anything relevant by searching the web?

No.

Any other information, logs, or outputs that you want to share?

Here is a CI run: https://github.com/avdv/testproject/actions/runs/9612425632/job/26513036166

The build succeeds on Ubuntu, but fails on macos with:

   Loading: 0 packages loaded
  Analyzing: target //:hello (1 packages loaded, 0 targets configured)
  Analyzing: target //:hello (34 packages loaded, 127 targets configured)
  Analyzing: target //:hello (34 packages loaded, 127 targets configured)
  INFO: Analyzed target //:hello (38 packages loaded, 164 targets configured).
  INFO: Found 1 target...
  [0 / 5] [Scann] Compiling hello.cc
  SUBCOMMAND: # //:hello [action 'Compiling hello.cc', configuration: 4d251272501a09258ab8679e379bc5cb22adc694d852a280468de7618e5b7017, execution platform: @local_config_platform//:host]
  (cd /private/var/tmp/_bazel_runner/ea77c70f2ed17460d1dbacea4c3f82b2/execroot/__main__ && \
    exec env - \
      PATH=/Users/runner/Library/Caches/bazelisk/downloads/sha256/bbf9c2c03bac48e0514f46db0295027935535d91f6d8dcd960c53393559eab29/bin:/usr/local/lib/ruby/gems/3.0.0/bin:/usr/local/opt/ruby@3.0/bin:/usr/local/opt/pipx_bin:/Users/runner/.cargo/bin:/usr/local/opt/curl/bin:/usr/local/bin:/usr/local/sbin:/Users/runner/bin:/Users/runner/.yarn/bin:/Users/runner/Library/Android/sdk/tools:/Users/runner/Library/Android/sdk/platform-tools:/Library/Frameworks/Python.framework/Versions/Current/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/usr/bin:/bin:/usr/sbin:/sbin:/Users/runner/.dotnet/tools:/Users/runner/.ghcup/bin \
      *** \
    external/local_config_cc/cc_wrapper.sh -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -Wunused-but-set-parameter -Wno-free-nonheap-object -fcolor-diagnostics -fno-omit-frame-pointer '-std=c++0x' -MD -MF bazel-out/darwin-fastbuild/bin/_objs/hello/hello.pic.d '-frandom-seed=bazel-out/darwin-fastbuild/bin/_objs/hello/hello.pic.o' -fPIC '-DBAZEL_CURRENT_REPOSITORY=""' -iquote . -iquote bazel-out/darwin-fastbuild/bin -iquote external/rules_cc -iquote bazel-out/darwin-fastbuild/bin/external/rules_cc -iquote external/bazel_tools -iquote bazel-out/darwin-fastbuild/bin/external/bazel_tools -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c hello.cc -o bazel-out/darwin-fastbuild/bin/_objs/hello/hello.pic.o)
  # Configuration: 4d251272501a09258ab8679e379bc5cb22adc694d852a280468de7618e5b7017
  # Execution platform: @local_config_platform//:host
  [4 / 5] [Prepa] Linking hello
  SUBCOMMAND: # //:hello [action 'Linking hello', configuration: 4d251272501a09258ab8679e379bc5cb22adc694d852a280468de7618e5b7017, execution platform: @local_config_platform//:host]
  (cd /private/var/tmp/_bazel_runner/ea77c70f2ed17460d1dbacea4c3f82b2/execroot/__main__ && \
    exec env - \
      PATH=/Users/runner/Library/Caches/bazelisk/downloads/sha256/bbf9c2c03bac48e0514f46db0295027935535d91f6d8dcd960c53393559eab29/bin:/usr/local/lib/ruby/gems/3.0.0/bin:/usr/local/opt/ruby@3.0/bin:/usr/local/opt/pipx_bin:/Users/runner/.cargo/bin:/usr/local/opt/curl/bin:/usr/local/bin:/usr/local/sbin:/Users/runner/bin:/Users/runner/.yarn/bin:/Users/runner/Library/Android/sdk/tools:/Users/runner/Library/Android/sdk/platform-tools:/Library/Frameworks/Python.framework/Versions/Current/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/usr/bin:/bin:/usr/sbin:/sbin:/Users/runner/.dotnet/tools:/Users/runner/.ghcup/bin \
      *** \
    external/local_config_cc/cc_wrapper.sh -o bazel-out/darwin-fastbuild/bin/hello bazel-out/darwin-fastbuild/bin/_objs/hello/hello.pic.o -Wl,-S -undefined dynamic_lookup -headerpad_max_install_names)
  # Configuration: 4d251272501a09258ab8679e379bc5cb22adc694d852a280468de7618e5b7017
  # Execution platform: @local_config_platform//:host
  Target //:hello up-to-date:
    bazel-bin/hello
  INFO: Elapsed time: 8.605s, Critical Path: 1.45s
  INFO: 5 processes: 3 internal, 2 darwin-sandbox.
  INFO: Build completed successfully, 5 total actions
  /Users/runner/work/_temp/61ab77a2-d8cd-4d8b-96be-5586bfcc805f.sh: line 4:  4156 Segmentation fault: 11  bazel-bin/hello

Inside a tmate session I could see that the acutal command that it used to link was this:

/usr/bin/gcc -o bazel-out/darwin-fastbuild/bin/hello bazel-out/darwin-fastbuild/bin/_objs/hello/hello.pic.o -Wl,-S -undefined dynamic_lookup -headerpad_max_install_names

So, this does not specify any C++ libraries at all, resulting in dynamic resolution at runtime:

(lldb) target create "bazel-bin/hello"
Current executable set to '/Users/runner/work/rules_nixpkgs/rules_nixpkgs/examples/toolchains/cc/bazel-bin/hello' (x86_64).
(lldb) run
Process 7097 launched: '/Users/runner/work/rules_nixpkgs/rules_nixpkgs/examples/toolchains/cc/bazel-bin/hello' (x86_64)
Process 7097 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xffffffffffffffe8)
    frame #0: 0x00007ff81ae39e80 libc++.1.dylib`std::__1::basic_ostream<char, std::__1::char_traits<char> >::sentry::sentry(std::__1::basic_ostream<char, std::__1::char_traits<char> >&) + 16
libc++.1.dylib`std::__1::basic_ostream<char, std::__1::char_traits<char> >::sentry::sentry:
->  0x7ff81ae39e80 <+16>: movq   -0x18(%rax), %rax
    0x7ff81ae39e84 <+20>: cmpl   $0x0, 0x20(%rsi,%rax)
    0x7ff81ae39e89 <+25>: jne    0x7ff81ae39ea3            ; <+51>
    0x7ff81ae39e8b <+27>: movq   %rdi, %rbx
Target 0: (hello) stopped.

The cc toolchain's config has link_libs = ["-lstdc++", "-lm"] but that is not passed to the linker command....

After adding linkopts = ["-lstdc++"] to the cc_binary rule, the binary works.

[keith]

You likely need to pass macos_minimum_os to make sure the binary is built for your version and not a newer one. You can check the binary with vtool to see what version it's targeting

[avdv]

You likely need to pass macos_minimum_os to make sure the binary is built for your version and not a newer one. You can check the binary with vtool to see what version it's targeting

You mean it builds for something other than my current os? By default? 🤯

vtool -show-build bazel-bin/hello shows:

  bazel-bin/hello:
  Load command 10
        cmd LC_BUILD_VERSION
    cmdsize 32
   platform MACOS
      minos 12.0
        sdk 13.1
     ntools 1
       tool LD
    version 820.1

But even when using bazel build -s --macos_minimum_os=12.0 :hello the binary still fails with a segfault and the vtool command reports the same info:

bazel-bin/hello:
  Load command 10
        cmd LC_BUILD_VERSION
    cmdsize 32
   platform MACOS
      minos 12.0
        sdk 13.1
     ntools 1
       tool LD
    version 820.1

because there was no change for Bazel:

  INFO: Build option --macos_minimum_os has changed, discarding analysis cache.
  Analyzing: target //:hello (0 packages loaded, 0 targets configured)
  INFO: Analyzed target //:hello (0 packages loaded, 165 targets configured).
  INFO: Found 1 target...
  [0 / 6] [Prepa] BazelWorkspaceStatusAction stable-status.txt
  Target //:hello up-to-date:
    bazel-bin/hello
  INFO: Elapsed time: 0.391s, Critical Path: 0.00s
  INFO: 1 process: 1 internal.
  INFO: Build completed successfully, 1 total action

[keith]

probably worth verifying the compile commands do correctly have the minimal OS version baked into them. is the crash report the same as the original post?

[avdv]

probably worth verifying the compile commands do correctly have the minimal OS version baked into them.

What exactly is the expected effect of passing --macos_minimum_os=12.0 to Bazel? Should this result in -mmacos-version-min=12.0 on the command line when compiling or linking?

The commands run for compiling and linking are this:

/usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -Wunused-but-set-parameter -Wno-free-nonheap-object -fcolor-diagnostics -fno-omit-frame-pointer -std=c++0x -MD -MF bazel-out/darwin-fastbuild/bin/_objs/hello/hello.pic.d -frandom-seed=bazel-out/darwin-fastbuild/bin/_objs/hello/hello.pic.o -fPIC '-DBAZEL_CURRENT_REPOSITORY=""' -iquote . -iquote bazel-out/darwin-fastbuild/bin -iquote external/rules_cc -iquote bazel-out/darwin-fastbuild/bin/external/rules_cc -iquote external/bazel_tools -iquote bazel-out/darwin-fastbuild/bin/external/bazel_tools -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c hello.cc -o bazel-out/darwin-fastbuild/bin/_objs/hello/hello.pic.o

/usr/bin/gcc -o bazel-out/darwin-fastbuild/bin/hello bazel-out/darwin-fastbuild/bin/_objs/hello/hello.pic.o -Wl,-S -undefined dynamic_lookup -headerpad_max_install_names

But even when I add -mmacos-version-min=12.0 manually to these commands, the binary still crashes:

runner@Mac-1719294883569 testproject % /usr/bin/gcc -mmacos-version-min=12.0  -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -Wunused-but-set-parameter -Wno-free-nonheap-object -fcolor-diagnostics -fno-omit-frame-pointer -std=c++0x -frandom-seed=bazel-out/darwin-fastbuild/bin/_objs/hello/hello.pic.o -fPIC '-DBAZEL_CURRENT_REPOSITORY=""' -iquote . -iquote bazel-out/darwin-fastbuild/bin -iquote external/rules_cc -iquote bazel-out/darwin-fastbuild/bin/external/rules_cc -iquote external/bazel_tools -iquote bazel-out/darwin-fastbuild/bin/external/bazel_tools -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c hello.cc -o hello.o
runner@Mac-1719294883569 testproject % /usr/bin/gcc -o hello hello.o -Wl,-S -undefined dynamic_lookup -headerpad_max_install_names
runner@Mac-1719294883569 testproject % ./hello
zsh: segmentation fault  ./hello
runner@Mac-1719294883569 testproject % /usr/bin/gcc -o hello hello.o -Wl,-S -undefined dynamic_lookup -headerpad_max_install_names -mmacos-version-min=12.0
runner@Mac-1719294883569 testproject % ./hello                                                                                                             
zsh: segmentation fault  ./hello
runner@Mac-1719294883569 testproject % lldb ./hello                                                                                                        
(lldb) target create "./hello"
Current executable set to '/Users/runner/work/testproject/testproject/hello' (x86_64).
(lldb) run
Process 7275 launched: '/Users/runner/work/testproject/testproject/hello' (x86_64)
Process 7275 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xffffffffffffffe8)
    frame #0: 0x00007ff80d6a4e80 libc++.1.dylib`std::__1::basic_ostream<char, std::__1::char_traits<char> >::sentry::sentry(std::__1::basic_ostream<char, std::__1::char_traits<char> >&) + 16
libc++.1.dylib`std::__1::basic_ostream<char, std::__1::char_traits<char> >::sentry::sentry:
->  0x7ff80d6a4e80 <+16>: movq   -0x18(%rax), %rax
    0x7ff80d6a4e84 <+20>: cmpl   $0x0, 0x20(%rsi,%rax)
    0x7ff80d6a4e89 <+25>: jne    0x7ff80d6a4ea3            ; <+51>
    0x7ff80d6a4e8b <+27>: movq   %rdi, %rbx
Target 0: (hello) stopped.

Why isn't the C++ standard library just added to the linker command line as it is done for Linux?

[keith]

I missed that adding the C++ stdlib explicitly fixes things. I don't know why that's generally not passed by default. I think it might have to do with the cc toolchain doesn't know if it's linking a C++ or a C binary right now I don't think. with the apple toolchain from the apple_support repo this isn't a problem since we use clang++ to link C++ binaries, where the default toolchain uses the non ++ version