Provide means for opting out of having "_virtual_includes" mentioned in C/C++ binaries

[nacl]

Description of the problem / feature request:

Whenever include_prefix or strip_include_prefix are provided to cc_library, bazel creates a hierarchy in the execution root that starts with something like this:

path/to/$PACKAGE/_virtual_includes/$TARGET

Which are passed in to the C/C++ compiler via -I. This results in the debuginfo values in the compiled files specifying that said headers can be found entirely under execution root, which is only valid if Bazel has been run before. Further, the _virtual_includes path hides exactly where the files in question are located, and requires additional steps to be performed to determine the location the actual header file for editing purposes.

We have a workflow in which engineers do debugging of coredumps provided by customers and they must (at least initially) be able to operate in environments where a build was not necessarily done previously. It is understood and expected that debugging generated code or external repositories will require interaction with the build system.

Our current thoughts on this is that it should be possible to create a toolchain feature that can create this mapping using compiler flags like gcc's -fdebug-prefix-map or -ffile-prefix-map.

See also our post on this on bazel-discuss: https://groups.google.com/forum/#!topic/bazel-discuss/prVSDuifmHY

Feature requests: what underlying problem are you trying to solve with this feature?

Debuginfos generated by gcc in Bazel may specify headers relative to a _virtual_includes path instead of where they actually may be found in the source repository. An option should be available to allow users to opt-out of this feature.

This problem, in one form or another, impacts things like emitting coverage instrumentation (bazel build --collect-code-coverage ...) and potentially compilation databases.

What operating system are you running Bazel on?

CentOS Linux release 7.7.1908 (Core)

What's the output of bazel info release?

release 3.3.0-vmware

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

N/A

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

N/A

Have you found anything relevant by searching the web?

Similar issues/commits found:

• Commit 3f46dd0 adds a feature for the macOS CROSSTOOL that uses -fdebug-prefix-map. It, however, is only really useful for macOS builds.

• Absolute paths are embedded in produced binaries #1000 is similar to this issue, but is mostly related to macOS and the embedding of absolute paths within the binaries. This ticket is more interested in remapping paths from within the macOS sandbox; the Linux sandboxes all seem to use paths relative to the WORKSPACE root.

• Output the actual path of virtual include headers in the coverage report #6254 fixes a similar problem was encountered by coverage builds (as run via bazel coverage). Unfortunately, while we could use the dataset that is assembled there, it, unfortunately, is far too large to be used on compiler command lines, nearing 6000+ entries in some cases.

Any other information, logs, or outputs that you want to share?

We have written a internal change to bazel that creates a CcToolchainConfigInfo build variable containing the necessary mapping information that can be used to write a toolchain feature that can solve this problem by using gcc's -fdebug-prefix-map. We can look into publishing it here, but it needs a little more refinement before it would be ready.

[burkpojken]

For our company it is also a big problem that even if you have all source code you cannot use the debug information without actually building the binaries your self. We have very much the same use case as stated in the request with engineers that do debugging of coredumps provided by customers.
So for us it would be a very valuable improvement if this was solved.

[github-actions]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 2+ years. It will be closed in the next 14 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

[github-actions]

This issue has been automatically closed due to inactivity. If you're still interested in pursuing this, please reach out to the triage team (@bazelbuild/triage). Thanks!

[kolrami]

Hi @nacl and @burkpojken, I jus discovered this issue because I have the same situation for me. Is there some solution for this?

[burkpojken]

We stopped using strip_include_prefix and use includes instead, https://bazel.build/reference/be/c-cpp#cc_library.includes

[eguiraud]

This should really be reopened: afaict there is still no good way to generate a compile_commands.json that does not contain non-existing virtual_include paths to third-party dependency headers.

[eguiraud]

@bazelbuild/triage gentle ping

[jwnimmer-tri]

... afaict there is still no good way ...

To be clear, the immediately prior message shows a solution for strip_include_prefix: use includes = ["foo"] instead of strip_include_prefix = "foo". It would help steer the discussion if you could explain what about that solution is "no good". That would help sharpen what the remaining problem(s) are here.

[eguiraud]

Sorry for the high latency -- the problem I was seeing turns out to be a bit more convoluted (more details here) and it is not strictly the fault of virtual_includes -- it's that when generating the compilation database the execroot directory in which those includes live is deleted/overwritten.

About using includes instead of strip_include_prefix: in my case the problem is with external dependencies and I am not sure what to put in the includes in that case. For example, what's the path to a googletest include directory when it is pulled in as a dependency?

[cpsauer]

^ indeed, not every _virtual_includes has an includes equivalent, sadly. Consider the case with just an include_prefix.

[ghost]

One drawback with includes vs. {,strip_}include_prefix is that, with includes, headers are included with -isystem. GCC suppresses warnings when processing system headers.

For another, suppose the following:

• Header is foo/bar/baz.h.
• Bazel package is //foo/bar.
• Canonical include statement is #include "bar/baz.h".

I can either use include_prefix = "bar" or includes = [".."]. The latter form is (a) unintuitive and (b) given past experience, is a little risky. (Misc Bazel impl details tend to assume the absence of, if not outright forbid, stray ..s.)

Given that, @nacl whipped up a patch for this that we've been using for a few years, and I'm currently rebasing it against Bazel 7. I'd be happy to turn this into a PR.

[bgianfo]

@bazelbuild/triage does it make sense to re-open this, as there is an active PR trying to fix it?