'__config_site' file not found

[alexirion]

Using a self-build clang 14.0.6, setup for cross compiling for arm-linux-gnueabihf, leads to the compilation error:

/usr/local/bin/../include/c++/v1/__config:13:10: fatal error: '__config_site' file not found

I build clang 14.0.6 by:

cmake -G Ninja -S llvm -B build -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS="clang" -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi" -DLLVM_RUNTIME_TARGETS="arm-linux-gnueabihf" \
-DLLVM_TARGETS_TO_BUILD="ARM"

When I use -I/usr/local/include/arm-linux-gnueabihf/c++/v1 the compilation works, however I would expect that clang could find that target non-agnostic file without explicitly setting the include directory, by using the value of the -target option.

CXXFLAGS is set to --target=arm-linux-gnueabihf -march=armv8-a -mcpu=cortex-a72 -mfpu=neon-fp-armv8 -mfloat-abi=hard -stdlib=libc++.

Here the full verbose compilation output, which fails:

/usr/local/bin/clang++   --target=arm-linux-gnueabihf -march=armv8-a -mcpu=cortex-a72 -mfpu=neon-fp-armv8 -mfloat-abi=hard -stdlib=libc++ -v -fcolor-diagnostics -Wall -Wpedantic -Werror -Wconversion -std=gnu++20 -MD -MT homenet/test/helloworld/CMakeFiles/helloworld.dir/main.cpp.o -MF homenet/test/helloworld/CMakeFiles/helloworld.dir/main.cpp.o.d -o homenet/test/helloworld/CMakeFiles/helloworld.dir/main.cpp.o -c /home/airion/projects/homenet/myhomenet/homenet/test/helloworld/main.cpp
clang version 14.0.6 (https://github.com/llvm/llvm-project.git f28c006a5895fc0e329fe15fead81e37457cb1d1)
Target: arm-unknown-linux-gnueabihf
Thread model: posix
InstalledDir: /usr/local/bin
Found candidate GCC installation: /usr/lib/gcc-cross/arm-linux-gnueabihf/8
Found candidate GCC installation: /usr/lib/gcc/arm-linux-gnueabihf/8
Selected GCC installation: /usr/lib/gcc-cross/arm-linux-gnueabihf/8
Candidate multilib: .;@m32
Selected multilib: .;@m32
 (in-process)
 "/usr/local/bin/clang-14" -cc1 -triple armv8-unknown-linux-gnueabihf -emit-obj -mrelax-all --mrelax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name main.cpp -mrelocation-model static -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -target-cpu cortex-a72 -target-feature +crc -target-feature -dotprod -target-feature +dsp -target-feature -mve -target-feature -mve.fp -target-feature -ras -target-feature -bf16 -target-feature -sb -target-feature -i8mm -target-feature -lob -target-feature -cdecp0 -target-feature -cdecp1 -target-feature -cdecp2 -target-feature -cdecp3 -target-feature -cdecp4 -target-feature -cdecp5 -target-feature -cdecp6 -target-feature -cdecp7 -target-feature -pacbti -target-feature +hwdiv-arm -target-feature +hwdiv -target-feature +vfp2 -target-feature +vfp2sp -target-feature +vfp3 -target-feature +vfp3d16 -target-feature +vfp3d16sp -target-feature +vfp3sp -target-feature +fp16 -target-feature +vfp4 -target-feature +vfp4d16 -target-feature +vfp4d16sp -target-feature +vfp4sp -target-feature +fp-armv8 -target-feature +fp-armv8d16 -target-feature +fp-armv8d16sp -target-feature +fp-armv8sp -target-feature -fullfp16 -target-feature +fp64 -target-feature +d32 -target-feature +neon -target-feature -fp16fml -target-feature -crypto -target-feature -sha2 -target-feature -aes -target-abi aapcs-linux -mfloat-abi hard -fallow-half-arguments-and-returns -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=gdb -v -fcoverage-compilation-dir=/home/airion/projects/homenet/products/rpi4 -resource-dir /usr/local/lib/clang/14.0.6 -dependency-file homenet/test/helloworld/CMakeFiles/helloworld.dir/main.cpp.o.d -MT homenet/test/helloworld/CMakeFiles/helloworld.dir/main.cpp.o -sys-header-deps -internal-isystem /usr/local/bin/../include/c++/v1 -internal-isystem /usr/local/lib/clang/14.0.6/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc-cross/arm-linux-gnueabihf/8/../../../../arm-linux-gnueabihf/include -internal-externc-isystem /usr/include/arm-linux-gnueabihf -internal-externc-isystem /include -internal-externc-isystem /usr/include -Wall -Wpedantic -Werror -Wconversion -std=gnu++20 -fdeprecated-macro -fdebug-compilation-dir=/home/airion/projects/homenet/products/rpi4 -ferror-limit 19 -fno-signed-char -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o homenet/test/helloworld/CMakeFiles/helloworld.dir/main.cpp.o -x c++ /home/airion/projects/homenet/myhomenet/homenet/test/helloworld/main.cpp
clang -cc1 version 14.0.6 based upon LLVM 14.0.6 default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/bin/../include/c++/v1
 /usr/local/lib/clang/14.0.6/include
 /usr/local/include
 /usr/lib/gcc-cross/arm-linux-gnueabihf/8/../../../../arm-linux-gnueabihf/include
 /usr/include/arm-linux-gnueabihf
 /usr/include
End of search list.
In file included from /home/airion/projects/homenet/myhomenet/homenet/test/helloworld/main.cpp:1:
In file included from /usr/local/bin/../include/c++/v1/cstdio:98:
/usr/local/bin/../include/c++/v1/__config:13:10: fatal error: '__config_site' file not found
#include <__config_site>

[victorstewart]

this still isn't fixed in 15.0.2?

[mizvekov]

I see this problem when running lldb tests from the test suite on a linux system.

Copying __config_site into the main include dir helps getting everything to work (Horrible workaround, but indicates this is the only problem affecting the system).

[ldionne]

I think this is a Clang problem -- the Clang driver for that target isn't looking into the right directory.

@petrhosek Any clue about this? Shouldn't the driver for all targets now be looking into the per-target include dir?

[petrhosek]

Clang should add that include path automatically, see

llvm-project/clang/lib/Driver/ToolChains/Gnu.cpp

Lines 3073 to 3076 in 13b8362

	// First add the per-target include path if it exists.
	std::string TargetDir = Path + "/" + Target + "/c++/" + Version;
	if (D.getVFS().exists(TargetDir))
	addSystemInclude(DriverArgs, CC1Args, TargetDir);

I think the issue here is the triple. You pass --target=arm-linux-gnueabihf (that is 32-bit ARM Linux) but also -march=armv8-a -mcpu=cortex-a72 which overrides the triple, so the effective normalized triple is going to be aarch64-unknown-linux-gnu (that is 64-bit ARM Linux) and Clang will end up looking in /usr/local/include/aarch64-unknown-linux-gnu/c++/v1 which doesn't exist since you've built libc++ for arm-linux-gnueabihf.

I'd also point out that you're only building the 32-bit ARM backend because you pass -DLLVM_TARGETS_TO_BUILD="ARM", so I'd expect -march=armv8-a -mcpu=cortex-a72 to cause an error during later stages since there's no 64-bit backend.

I'm not sure what's the right behavior in this case, maybe Clang should complain that the --target, -march and -mcpu are incompatible?

[alexirion]

Same issue, when I only pass --target=arm-linux-gnueabihf without -march and -cpu. The /usr/local/include/arm-linux-gnueabihf/c++/v1 is not in the search path.
It makes now arm-unknown-linux-gnueabihf from --target=arm-linux-gnueabihf.

/usr/local/bin/clang++ --target=arm-linux-gnueabihf -D_FILE_OFFSET_BITS=64 -v -stdlib=libc++ -fcolor-diagnostics -Wall -Wpedantic -Werror -Wconversion -std=gnu++20 -MD -MT homenet/test/helloworld/CMakeFiles/helloworld.dir/main.cpp.o -MF homenet/test/helloworld/CMakeFiles/helloworld.dir/main.cpp.o.d -o homenet/test/helloworld/CMakeFiles/helloworld.dir/main.cpp.o -c /home/airion/projects/homenet/myhomenet/homenet/test/helloworld/main.cpp
clang version 15.0.0 (https://github.com/llvm/llvm-project.git 4ba6a9c)
Target: arm-unknown-linux-gnueabihf
Thread model: posix
InstalledDir: /usr/local/bin
Found candidate GCC installation: /usr/lib/gcc/arm-linux-gnueabihf/8
Selected GCC installation: /usr/lib/gcc/arm-linux-gnueabihf/8
Candidate multilib: .;@m32
Selected multilib: .;@m32
(in-process)
"/usr/local/bin/clang-15" -cc1 -triple armv6kz-unknown-linux-gnueabihf -emit-obj -mrelax-all --mrelax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name main.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -target-cpu arm1176jzf-s -target-feature +vfp2 -target-feature +vfp2sp -target-feature -vfp3 -target-feature -vfp3d16 -target-feature -vfp3d16sp -target-feature -vfp3sp -target-feature -fp16 -target-feature -vfp4 -target-feature -vfp4d16 -target-feature -vfp4d16sp -target-feature -vfp4sp -target-feature -fp-armv8 -target-feature -fp-armv8d16 -target-feature -fp-armv8d16sp -target-feature -fp-armv8sp -target-feature -fullfp16 -target-feature +fp64 -target-feature -d32 -target-feature -neon -target-feature -sha2 -target-feature -aes -target-feature -fp16fml -target-feature +strict-align -target-abi aapcs-linux -mfloat-abi hard -Wunaligned-access -fallow-half-arguments-and-returns -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=gdb -v -fcoverage-compilation-dir=/home/airion/projects/homenet/products/rpi4 -resource-dir /usr/local/lib/clang/15.0.0 -dependency-file homenet/test/helloworld/CMakeFiles/helloworld.dir/main.cpp.o.d -MT homenet/test/helloworld/CMakeFiles/helloworld.dir/main.cpp.o -sys-header-deps -D _FILE_OFFSET_BITS=64 -internal-isystem /usr/local/bin/../include/c++/v1 -internal-isystem /usr/local/lib/clang/15.0.0/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/arm-linux-gnueabihf/8/../../../../arm-linux-gnueabihf/include -internal-externc-isystem /usr/include/arm-linux-gnueabihf -internal-externc-isystem /include -internal-externc-isystem /usr/include -Wall -Wpedantic -Werror -Wconversion -std=gnu++20 -fdeprecated-macro -fdebug-compilation-dir=/home/airion/projects/homenet/products/rpi4 -ferror-limit 19 -fno-signed-char -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o homenet/test/helloworld/CMakeFiles/helloworld.dir/main.cpp.o -x c++ /home/airion/projects/homenet/myhomenet/homenet/test/helloworld/main.cpp
clang -cc1 version 15.0.0 based upon LLVM 15.0.0 default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/usr/lib/gcc/arm-linux-gnueabihf/8/../../../../arm-linux-gnueabihf/include"
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
/usr/local/bin/../include/c++/v1
/usr/local/lib/clang/15.0.0/include
/usr/local/include
/usr/include/arm-linux-gnueabihf
/usr/include
End of search list.
In file included from /home/airion/projects/homenet/myhomenet/homenet/test/helloworld/main.cpp:1:
In file included from /usr/local/bin/../include/c++/v1/cstdio:98:
In file included from /usr/local/bin/../include/c++/v1/__assert:13:
/usr/local/bin/../include/c++/v1/__config:13:10: fatal error: '__config_site' file not found
#include <__config_site>
^~~~~~~~~~~~~~~
1 error generated.

[GavinRay97]

Also having with problem, with LLVM 16 dev and Fedora Rawhide on x86_x64-linux-gnu

I built from source using:

$ cmake -S llvm -B build -G Ninja \
	-DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra" \
 	-DLLVM_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi" \
	-DLLVM_USE_LINKER=mold \
	-DCMAKE_BUILD_TYPE=Release \
	-DCMAKE_C_COMPILER=clang \
	-DCMAKE_CXX_COMPILER=clang++
$ cd build
$ cmake --build . --target install

[alexirion]

With setting -DLLVM_RUNTIME_TARGETS="arm-unknown-linux-gnueabihf" it has finally worked! :-) Here the unknown is important! The __config_site file is then created in /usr/local/include/arm-unknown-linux-gnueabihf/c++/v1 and is later on found when compiling a program with --target=arm-unknown-linux-gnueabihf.
Might be better when the value of LLVM_RUNTIME_TARGETS would also be normalized, then leaving out the "unknown" would work in the same way?

[petrhosek]

Yes, normalizing triples in LLVM_RUNTIME_TARGETS is planned, but there's some groundwork we need to do first.

[luporl]

I was also seeing this issue when running ninja check-all, from the build directory, when using release/16.x.

The following patch fixed the issue for me:

commit 67992e487b87a1e3f4dba9dcbb25c652597799ff
Author: Leandro Lupori <leandro.lupori@linaro.org>
Date:   Fri May 19 17:42:46 2023 +0000

    [libcxx] Fix __config_site location on ARM

diff --git a/libcxx/include/CMakeLists.txt b/libcxx/include/CMakeLists.txt
index a12aa1de1356..51dc7fccf377 100644
--- a/libcxx/include/CMakeLists.txt
+++ b/libcxx/include/CMakeLists.txt
@@ -877,10 +877,18 @@ foreach(feature LIBCXX_ENABLE_FILESYSTEM LIBCXX_ENABLE_LOCALIZATION LIBCXX_ENABL
   endif()
 endforeach()

-configure_file("__config_site.in" "${LIBCXX_GENERATED_INCLUDE_TARGET_DIR}/__config_site" @ONLY)
+if(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR AND NOT APPLE)
+  set(LIBCXX_CONFIG_SITE "${LIBCXX_GENERATED_INCLUDE_TARGET_DIR}/__config_site")
+  set(LIBCXX_CONFIG_SITE_INSTALL_DIR "${LIBCXX_INSTALL_INCLUDE_TARGET_DIR}")
+else()
+  set(LIBCXX_CONFIG_SITE "${LIBCXX_GENERATED_INCLUDE_DIR}/__config_site")
+  set(LIBCXX_CONFIG_SITE_INSTALL_DIR "${LIBCXX_INSTALL_INCLUDE_DIR}")
+endif()
+
+configure_file("__config_site.in" "${LIBCXX_CONFIG_SITE}" @ONLY)
 configure_file("module.modulemap.in" "${LIBCXX_GENERATED_INCLUDE_DIR}/module.modulemap" @ONLY)

-set(_all_includes "${LIBCXX_GENERATED_INCLUDE_TARGET_DIR}/__config_site"
+set(_all_includes "${LIBCXX_CONFIG_SITE}"
                   "${LIBCXX_GENERATED_INCLUDE_DIR}/module.modulemap")
 foreach(f ${files})
   set(src "${CMAKE_CURRENT_SOURCE_DIR}/${f}")
@@ -910,9 +918,9 @@ if (LIBCXX_INSTALL_HEADERS)
     )
   endforeach()

-  # Install the generated __config_site file to the per-target include dir.
-  install(FILES "${LIBCXX_GENERATED_INCLUDE_TARGET_DIR}/__config_site"
-    DESTINATION "${LIBCXX_INSTALL_INCLUDE_TARGET_DIR}"
+  # Install the generated __config_site file.
+  install(FILES "${LIBCXX_CONFIG_SITE}"
+    DESTINATION "${LIBCXX_CONFIG_SITE_INSTALL_DIR}"
     PERMISSIONS OWNER_READ OWNER_WRITE GROUP_READ WORLD_READ
     COMPONENT cxx-headers)

diff --git a/llvm/runtimes/CMakeLists.txt b/llvm/runtimes/CMakeLists.txt
index 694f346ee8e9..c16dec309d01 100644
--- a/llvm/runtimes/CMakeLists.txt
+++ b/llvm/runtimes/CMakeLists.txt
@@ -65,7 +65,8 @@ function(builtin_default_target compiler_rt_path)

   set(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR_default ON)
   # AIX should fold 32-bit & 64-bit arch libraries into a single archive.
-  if (LLVM_TARGET_TRIPLE MATCHES "aix")
+  # Per target dir not yet supported on Arm 32 bit due to arm vs armhf handling
+  if (LLVM_TARGET_TRIPLE MATCHES "aix" OR LLVM_TARGET_TRIPLE MATCHES "^arm")
     set(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR_default OFF)
   endif()

@@ -219,7 +220,8 @@ function(runtime_default_target)

   set(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR_default ON)
   # AIX should fold 32-bit & 64-bit arch libraries into a single archive.
-  if (LLVM_TARGET_TRIPLE MATCHES "aix")
+  # Per target dir not yet supported on Arm 32 bit due to arm vs armhf handling
+  if (LLVM_TARGET_TRIPLE MATCHES "aix" OR LLVM_TARGET_TRIPLE MATCHES "^arm")
     set(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR_default OFF)
   endif()

I don't know if this is the right way to fix this, but there are some mismatches regarding LLVM_ENABLE_PER_TARGET_RUNTIME_DIR on ARM:

• It's turned off in llvm/CMakeLists.txt, because it's not supported. But it was ON for runtimes.
• LLDB tests expect __config_site to be on ${LLVM_BINARY_DIR}/include/c++/v1 but libcxx/include/CMakeLists.txt was always generating it on ${LLVM_BINARY_DIR}/include/${LLVM_DEFAULT_TARGET_TRIPLE}/c++/v1.
• clang doesn't look for any target specific include directory in my install location, /usr/local, but only searches /usr/local/include. It does search /usr/include/arm-linux-gnueabihf, however.

[abrisco]

I'm running into this trying to build a release artifact for 14.0.6 on a Redhat-8.4 machine. Anyone have a suggested work around?

./llvm/utils/release/test-release.sh -no-test-suite -release 14.0.6 -triple x86_64-linux-gnu-rhel-8.4 -final -j 12

...
...
In file included from /llvm-build/llvm-project/final/llvm-project/bolt/runtime/instr.cpp:44:
In file included from /llvm-build/llvm-project/final/llvm-project/bolt/runtime/common.h:38:
In file included from /usr/include/elf.h:28:
In file included from /llvm-build/llvm-project/final/Phase2/Release/llvmCore-14.0.6-final.install/usr/local/bin/../include/c++/v1/stdint.h:106:
/llvm-build/llvm-project/final/Phase2/Release/llvmCore-14.0.6-final.install/usr/local/bin/../include/c++/v1/__config:13:10: fatal error: '__config_site' file not found
#include <__config_site>
         ^~~~~~~~~~~~~~~

[DimitryAndric]

I'm running into this trying to build a release artifact for 14.0.6 on a Redhat-8.4 machine.
...
In file included from /llvm-build/llvm-project/final/Phase2/Release/llvmCore-14.0.6-final.install/usr/local/bin/../include/c++/v1/stdint.h:106:
/llvm-build/llvm-project/final/Phase2/Release/llvmCore-14.0.6-final.install/usr/local/bin/../include/c++/v1/__config:13:10: fatal error: '__config_site' file not found
#include <__config_site>
^~~~~~~~~~~~~~~

I cannot reproduce this on a Rocky 8.8 machine (as I don't have access to old versions such as 8.4), but for me it errors in a different place:

cd /home/jenkins/llvm/14.0.6/final/Phase1/Release/llvmCore-14.0.6-final.obj/projects/libcxxabi/src && /bin/c++ -DHAVE___CXA_THREAD_ATEXIT_IMPL -D_GNU_SOURCE -D_LIBCPP_BUILDING_LIBRARY -D_LIBCPP_DISABLE_EXTERN_TEMPLATE -D_LIBCXXABI_BUILDI>
In file included from /home/jenkins/llvm/14.0.6/final/Phase1/Release/llvmCore-14.0.6-final.obj/include/c++/v1/exception:84,
                 from /home/jenkins/llvm/14.0.6/final/Phase1/Release/llvmCore-14.0.6-final.obj/include/c++/v1/new:93,
                 from /home/jenkins/llvm/14.0.6/final/llvm-project/libcxxabi/src/cxa_aux_runtime.cpp:13:
/home/jenkins/llvm/14.0.6/final/Phase1/Release/llvmCore-14.0.6-final.obj/include/c++/v1/type_traits:554:53: error: there are no arguments to '__is_same' that depend on a template parameter, so a declaration of '__is_same' must be availab>
 struct _LIBCPP_TEMPLATE_VIS is_same : _BoolConstant<__is_same(_Tp, _Up)> { };
                                                     ^~~~~~~~~
/home/jenkins/llvm/14.0.6/final/Phase1/Release/llvmCore-14.0.6-final.obj/include/c++/v1/type_traits:554:53: note: (if you use '-fpermissive', G++ will accept your code, but allowing the use of an undeclared name is deprecated)
/home/jenkins/llvm/14.0.6/final/Phase1/Release/llvmCore-14.0.6-final.obj/include/c++/v1/type_traits:554:72: error: template argument 1 is invalid

That said, you will have to investigate where __config_site is generated: do you see it anywhere in your object tree? If so, in which directory is it? And if it was not generated, can you check the CMake logs to see why?

[abrisco]

I don't know how or why I was able to build the artifact but by only installing the following from a clean RHEL-8.4 VM I was able to build an artifact

sudo dnf install git chrpath cmake clang gcc gcc-c++

[pkeir]

Adding -I $MY_LLVM_INSTALL/include/x86_64-unknown-linux-gnu/c++/v1 to my clang++ invocation worked for me - with 16.0.6 on Crostini (Debian):

clang++ -std=c++2b -stdlib=libc++ -nostdinc++ -I $MY_LLVM_INSTALL/include/c++/v1 -I $MY_LLVM_INSTALL/include/x86_64-unknown-linux-gnu/c++/v1 -L $MY_LLVM_INSTALL/lib -Wl,-rpath,$MY_LLVM_INSTALL/lib test.cpp

[llvmbot]

@llvm/issue-subscribers-clang-driver

[momchil-velikov]

I've got include/x86_64-unknown-linux-gnu/c++/v1/__config_site in my installation, which works fine.
Until you try to invoke clang as, e.g. clang++ -target x86_64-pc-linux ... or any other kind of an equivalent, but not identical target triple.

[rustyx]

I've got include/x86_64-unknown-linux-gnu/c++/v1/__config_site in my installation, which works fine. Until you try to invoke clang as, e.g. clang++ -target x86_64-pc-linux ... or any other kind of an equivalent, but not identical target triple.

If you specify a -target that doesn't match the installed clang target, you'll have more issues, the missing __config_site is just the first one reported.

Question is, why is the source build of LLVM result in the x86_64-unknown-linux-gnu triple on Linux, while e.g. a Debian package build produces x86_64-pc-linux (which happens to be what Boost is using in its clang toolchain).