Sandbox hermetic tmp folder causes gcc switches back and forth

[daivinhtran]

Description of the bug:

This bug is only in Bazel 7.0.* .

Sandbox hermetic tmp folder causes gcc switches back and forth between /tmp/ and the declared include paths when resolving the headers.

Which category does this issue belong to?

No response

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

1. Checkout daivinhtran/examples@98b049d in the examples repo
2. USE_BAZEL_VERSION=7.0.2 bazelisk build //main:hello-world

#include <...> search starts here:
 external/gcc_toolchain/x86_64-buildroot-linux-gnu/include/c++/12.3.0
 external/gcc_toolchain/x86_64-buildroot-linux-gnu/include/c++/12.3.0/x86_64-buildroot-linux-gnu
 external/gcc_toolchain/x86_64-buildroot-linux-gnu/include
 external/gcc_toolchain/x86_64-buildroot-linux-gnu/sysroot/usr/include
 external/gcc_toolchain/lib/gcc/x86_64-buildroot-linux-gnu/12.3.0/include
 /tmp/bazel-source-roots/0/bin/../lib/gcc/x86_64-buildroot-linux-gnu/12.3.0/../../../../x86_64-buildroot-linux-gnu/include/c++/12.3.0
 /tmp/bazel-source-roots/0/bin/../lib/gcc/x86_64-buildroot-linux-gnu/12.3.0/../../../../x86_64-buildroot-linux-gnu/include/c++/12.3.0/x86_64-buildroot-linux-gnu
 /tmp/bazel-source-roots/0/bin/../lib/gcc/x86_64-buildroot-linux-gnu/12.3.0/../../../../x86_64-buildroot-linux-gnu/include/c++/12.3.0/backward
 /tmp/bazel-source-roots/0/bin/../lib/gcc/x86_64-buildroot-linux-gnu/12.3.0/include
 /tmp/bazel-source-roots/0/bin/../lib/gcc/x86_64-buildroot-linux-gnu/12.3.0/include-fixed
 /tmp/bazel-source-roots/0/bin/../lib/gcc/x86_64-buildroot-linux-gnu/12.3.0/../../../../x86_64-buildroot-linux-gnu/include
End of search list.

In file included from external/gcc_toolchain/x86_64-buildroot-linux-gnu/include/c++/12.3.0/ext/string_conversions.h:41,
                 from external/gcc_toolchain/x86_64-buildroot-linux-gnu/include/c++/12.3.0/bits/basic_string.h:3968,
                 from /tmp/bazel-source-roots/0/x86_64-buildroot-linux-gnu/include/c++/12.3.0/string:53,
                 from external/gcc_toolchain/x86_64-buildroot-linux-gnu/include/c++/12.3.0/bits/locale_classes.h:40,
                 from external/gcc_toolchain/x86_64-buildroot-linux-gnu/include/c++/12.3.0/bits/ios_base.h:41,
                 from /tmp/bazel-source-roots/0/x86_64-buildroot-linux-gnu/include/c++/12.3.0/ios:42,
                 from /tmp/bazel-source-roots/0/x86_64-buildroot-linux-gnu/include/c++/12.3.0/ostream:38,
                 from /tmp/bazel-source-roots/0/x86_64-buildroot-linux-gnu/include/c++/12.3.0/iostream:39,
                 from main/hello-world.cc:1:
/tmp/bazel-source-roots/0/x86_64-buildroot-linux-gnu/include/c++/12.3.0/cstdlib:75:15: fatal error: stdlib.h: No such file or directory
   75 | #include_next <stdlib.h>
      |               ^~~~~~~~~~

Even though the /tmp/* paths are not specified to the compiler, the compiler detected them and included in the path list to search for headers.

This is probably fine because /tmp/* paths are listed after the ones specified on the actions. However, as you can see in the error, the /tmp is preferred

from /tmp/bazel-source-roots/0/x86_64-buildroot-linux-gnu/include/c++/12.3.0/iostream:39,
from main/hello-world.cc:1:

and then switch back and forth between external/ and /tmp/ causing stdlib.h to eventually be unfound.

Building the same target + same gcc toolchain with Bazel 6.3.0 works.

Which operating system are you running Bazel on?

Linux

What is the output of bazel info release?

release 7.0.2

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

I think #20718 attempted to fix a related bug.

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[daivinhtran]

cc: @Wyverald

[fmeum]

Could you try adding -fno-canonical-system-headers and/or -no-canonical-prefixes? It's surprising that gcc would follow symlinks to an absolute path under /tmp if all headers are available under the provided relative paths.

[daivinhtran]

@fmeum Thanks! Using -fno-canonical-system-headers enforces me to specify all the missing headers.

[lberki]

For the record, it's kind of weird that gcc chooses to resolve symlinks in some entries of the include path but not in some others, but AFAIU it's perfectly within its rights to do so, so ¯_(ツ)_/¯