Debugging C++ on MacOS does not work

[Teivaz]

Description of the problem / feature request:

Debugging C++ on MacOS does not work. The built binary file contains debug hints which point to the location in the sandbox (which is removed after compilation).

Feature requests: what underlying problem are you trying to solve with this feature?

Replace this line with your answer.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

BUILD file

cc_binary(
    name = "sample",
    srcs = ["main.cpp"],
    visibility = ["//main:__pkg__"],
)

main.cpp file

#include <iostream>

int main() {
    std::cout << "tests" << std::endl;
    return 0;
}

Build with bazel build --compilation_mode=dbg sample
And debug

lldb bazel-bin/sample
(lldb) breakpoint set -n main
Breakpoint 1: 13 locations.
(lldb) r
Process 24391 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
    frame #0: 0x00000001000021b0 sample`main
sample`main:
->  0x1000021b0 <+0>: pushq  %rbp
    0x1000021b1 <+1>: movq   %rsp, %rbp
    0x1000021b4 <+4>: subq   $0x20, %rsp
    0x1000021b8 <+8>: movq   0xe51(%rip), %rdi         ; (void *)0x00007fff9c93a660: std::__1::cout
Target 0: (sample) stopped.

Debug hints in the executable:

nm -pa bazel-bin/sample | grep OSO
000000005bb7c96b - 03 0001   OSO /private/var/tmp/_bazel_teivaz/1e731ee5ae5a3ce6177976984318ec76/bazel-sandbox/1688134534644271312/execroot/__main__/bazel-out/darwin-dbg/bin/_objs/sample/main.o

While file built with clang++ -g main.cpp -o sample can be debugged normally:

lldb sample
(lldb) breakpoint set -n main
Breakpoint 1: where = sample`main + 29 at main.cpp:4, address = 0x0000000100000f9d
(lldb) r
Process 24410 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
    frame #0: 0x0000000100000f9d sample`main at main.cpp:4
   1    #include <iostream>
   2    
   3    int main() {
-> 4        std::cout << "tests" << std::endl;
   5        return 0;
   6    }
Target 0: (sample) stopped.

What operating system are you running Bazel on?

MacOS High Sierra 10.13.2

What's the output of bazel info release?

release 0.17.2-homebrew

Have you found anything relevant by searching the web?

I have found related issue 5545 but the proposed solution was already merged to the version of Bazel I am using.

Any other information, logs, or outputs that you want to share?

I have also asked a question on Stackoverflow

[steeve]

I think we can add #1000 to the list of related issues

[steeve]

And #5031

[kastiglione]

As a potential temporary work around, does --sandbox_debug (which leaves the sandbox directory intact) get around the problem?

[Teivaz]

@kastiglione Yes it is another workaround though I would like not to use it. I have found another workaround - disabling sandbox with --spawn_strategy=standalone. But none of them solves the issue.

[hlopko]

Yup this never worked, and I don't think we'll have time to look into this in 2018. Bringing apple expertise @sergiocampama just in case the fix is simpler than it sounds :).

[sergiocampama]

I would defer to the C++ language team, I don't have much experience with debugging C++ apps.

[jwnimmer-tri]

Also related to #2537 perhaps.

[ghost]

I have the same problem on Windows 10, I get

No symbol table is loaded. error message in the console when I try to debug

https://stackoverflow.com/questions/53840959/how-to-build-and-debug-a-c-executable-using-bazel-on-windows-10-x64

[steeve]

For the record, it's working for apple targets via --apple_generate_dsym flag. Perhaps cc_binary could consume it too ?

[slonka]

@steeve for me --apple_generate_dsym does not work. I'm trying to debug https://github.com/envoyproxy/envoy. I have the version of the bazyl clion extension linked here: bazelbuild/intellij#494 (comment) .

My setup is:
CLion 2019.1.3 Build #CL-191.7141.37, built on May 6, 2019
macOS 10.14.4
Build label: 0.26.0-homebrew
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue May 28 14:16:39 2019 (1559052999)
Build timestamp: 1559052999
Build timestamp as int: 1559052999

I'm putting a breakpoint the log line you see in the output and it does not catch it.

My run configuration:

When I pause the debugger it seems to know where it's paused (it can read the names of the functions, for example: Envoy::MainCommonBase::run() 0x000000010081e52e, but it does not seem to be able to map it in the editor - it jumps to disassembly)

Can you help with that or show me what to do so I can fix it myself?

[steeve]

@slonka can you try to disable the sandbox? --spawn_strategy=standalone
The reason is that paths in DWARF can reference the sandbox, not the actual path.
You can either disable sandboxing, or use a sourcemap.

Also, apple_generate_dsym works only for macos_application, but not cc_binary (which was my comment).

As for CLion, I don't know how to make it work, but I managed to do it in VSCode.

[slonka]

@steeve I tried with "--spawn_strategy=standalone" and "--sandbox_debug" they did not work in clion. Do you think it's an issue for bazel or the clion plugin?

[bofeiw]

Also experiencing this issue, I got this error when I debug the C++ program:

com.jetbrains.cidr.execution.debugger.backend.gdb.GDBDriver$GDBCommandException: "/private/var/tmp/_bazel_bofeiwang/856f12db26bfbe04ae52bb24ae3db2e3/execroot/__main__/bazel-out/darwin-dbg/bin/assignments/wl/main": not in executable format: File format not recognized

[depp]

While --spawn_strategy=standalone worked for me, there are some drawbacks to running all rules standalone. I added the following line to my bazelrc:

build --strategy_regexp=^Linking=local

This means that only the linking steps will be run without the sandbox.

[depp]

If changing the spawn strategy doesn’t work, try running bazel clean and then running bazel build --strategy_regexp=^Linking=local. Bazel will not rebuild targets just because the spawn strategy changes!

[TheButlah]

@depp using local for the linking strategy doesn't seem to work for me. I did the following to test my executable:

bazel clean
bazel run -c dbg @spybot//realsense/programs:server --run_under=lldb --strategy_regexp=^Linking=local

Now while in lldb:

break set -n main
run

This gives something like this:

Process 33280 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
    frame #0: 0x000000010000432b server`main(argc=1, argv=0x00007ffeefbff228) at server.cpp:60:15
Target 0: (server) stopped.

server.cpp is my file that contains the main() function. I wanted to see the location that the debug symbols claim this file is located at, so I did:

source info

This gives:

Lines found in module `server
[0x000000010000432b-0x0000000100004337): /private/var/tmp/_bazel_ryan.butler/14e00cfc95af169a4eb8b5156cec6517/sandbox/darwin-sandbox/369/execroot/saicny_robotics/external/spybot/realsense/programs/server.cpp:60:15

Note that the path is under the sandbox. And as feared, that file doesn't exist.

Also, it really would be good to get this fixed, its been a year and a half :(

[kjteske]

+1 - would love to see this fixed

[philwo]

I just tried to debug an issue in Bazel's own C++ code using lldb on my Mac and ... I couldn't get the line numbers or code to show up. 🤦

I'll try to find someone who can help fix this. Thanks everyone for documenting the workaround with disabling sandboxing!

Edit: Yay, it works with --spawn_strategy=local and my usual load of debug mode flags ( -c dbg --javacopt="-g" --copt="-g" --strip="never") :)

[keith]

As of today I think most of the issues are resolved, but unfortunately not enabled by default (and realistically probably can't be since some might require extra lldb configuration).

Note that I'm testing with bazel HEAD (although I think the current 5.x release has the same fixes)

For the original repro case here, using --features=oso_prefix_is_pwd is enough to get this to work. This results in the binary containing relative paths to the object files, so they are no longer affected by the temporary sandboxing paths.

A while back I also made debug_prefix_map_pwd_is_dot default to true, which likely covered another case here. Note that in some cases you likely need to set some LLDB configuration because of this, something like (note you have to replace $PWD with your project's actual PWD):

platform settings -w $PWD
settings set target.source-map ./ $PWD

You can set this in your ~/.lldbinit, or you can create a project local .lldbinit, but if you do the latter you either have to launch lldb with --local-lldbinit or you have to set settings set target.load-cwd-lldbinit true in your ~/.lldbinit for security.

If folks have more cases that don't work please post them here.

[StrongerXi]

MacOS with M1 chip.

For those experiencing the same issue, this is my minimal setting to get line numbers in lldb:

bazel build -c dbg --spawn_strategy=local ...

[jfirebaugh]

I found that platform settings -w need to be provided the Bazel execution root (i.e. the output of bazel info execution_root), not the workspace root or working directory.

[keith]

Please reopen new issues if the mentioned fixes don't work for you!