It would be nice to be able to build with clang rather than gcc

[ianamason]

I added a target file called target-llvm.cmake for clang and clang++ (from llvm 10), the relevant lines being:

SET(CMAKE_C_COMPILER            "/usr/lib/llvm-10/bin/clang")
SET(CMAKE_CXX_COMPILER          "/usr/lib/llvm-10/bin/clang++")

But alas after

make SIMULATION=llvm prep

followed by

make

I get

vagrant@ubuntu-bionic:~/cFS$ make
make --no-print-directory -C "build" mission-all
Scanning dependencies of target elf2cfetbl
[ 50%] Building C object tools/elf2cfetbl/CMakeFiles/elf2cfetbl.dir/elf2cfetbl.c.o
[100%] Linking C executable elf2cfetbl
[100%] Built target elf2cfetbl
Scanning dependencies of target mission-version
[100%] Built target mission-version
Scanning dependencies of target mission-prebuild
[100%] Built target mission-prebuild
Scanning dependencies of target llvm_default_cpu1-all
Scanning dependencies of target osal_posix_impl
[  1%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/src/os-impl-binsem.c.o
[  2%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/src/os-impl-common.c.o
[  2%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/src/os-impl-console.c.o
[  3%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/src/os-impl-countsem.c.o
[  4%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/src/os-impl-dirs.c.o
[  5%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/src/os-impl-errors.c.o
[  5%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/src/os-impl-files.c.o
[  6%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/src/os-impl-filesys.c.o
[  7%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/src/os-impl-heap.c.o
[  7%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/src/os-impl-idmap.c.o
[  8%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/src/os-impl-mutex.c.o
[  9%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/src/os-impl-queues.c.o
[  9%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/src/os-impl-tasks.c.o
[ 10%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/src/os-impl-timebase.c.o
[ 12%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/src/os-impl-fpu.c.o
[ 13%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/src/os-impl-interrupts.c.o
[ 13%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/__/portable/os-impl-posix-gettime.c.o
[ 14%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/__/portable/os-impl-console-bsp.c.o
[ 15%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/__/portable/os-impl-bsd-select.c.o
[ 15%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/__/portable/os-impl-posix-io.c.o
[ 16%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/__/portable/os-impl-posix-files.c.o
[ 17%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/__/portable/os-impl-posix-dirs.c.o
[ 18%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/__/portable/os-impl-no-shell.c.o
[ 18%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/src/os-impl-loader.c.o
[ 19%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/__/portable/os-impl-posix-dl-loader.c.o
[ 20%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/__/portable/os-impl-posix-dl-symtab.c.o
[ 20%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/__/portable/os-impl-bsd-sockets.c.o
[ 21%] Building C object osal/posix_impl/CMakeFiles/osal_posix_impl.dir/__/portable/os-impl-posix-network.c.o
[ 21%] Built target osal_posix_impl
Scanning dependencies of target osal_generic-linux_impl
[ 23%] Building C object osal/generic-linux_impl/CMakeFiles/osal_generic-linux_impl.dir/src/bsp_start.c.o
[ 23%] Building C object osal/generic-linux_impl/CMakeFiles/osal_generic-linux_impl.dir/src/bsp_console.c.o
[ 23%] Built target osal_generic-linux_impl
Scanning dependencies of target osal_bsp
[ 23%] Building C object osal/CMakeFiles/osal_bsp.dir/src/bsp/shared/src/osapi-bsp.c.o
[ 24%] Building C object osal/CMakeFiles/osal_bsp.dir/src/bsp/shared/src/bsp_default_app_run.c.o
[ 25%] Building C object osal/CMakeFiles/osal_bsp.dir/src/bsp/shared/src/bsp_default_app_startup.c.o
[ 26%] Building C object osal/CMakeFiles/osal_bsp.dir/src/bsp/shared/src/bsp_default_symtab.c.o
[ 26%] Building C object osal/CMakeFiles/osal_bsp.dir/src/bsp/shared/src/bsp_default_voltab.c.o
[ 27%] Linking C static library libosal_bsp.a
[ 27%] Built target osal_bsp
Scanning dependencies of target osal
[ 28%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-binsem.c.o
[ 28%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-clock.c.o
[ 29%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-common.c.o
[ 30%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-countsem.c.o
[ 31%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-dir.c.o
[ 31%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-errors.c.o
[ 32%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-file.c.o
[ 34%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-filesys.c.o
[ 34%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-heap.c.o
[ 35%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-idmap.c.o
[ 36%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-module.c.o
[ 37%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-mutex.c.o
[ 37%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-network.c.o
[ 38%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-printf.c.o
[ 39%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-queue.c.o
[ 39%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-select.c.o
[ 40%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-shell.c.o
[ 41%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-sockets.c.o
[ 41%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-task.c.o
[ 42%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-timebase.c.o
[ 43%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-time.c.o
[ 45%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-debug.c.o
[ 45%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-fpu.c.o
[ 46%] Building C object osal/CMakeFiles/osal.dir/src/os/shared/src/osapi-interrupts.c.o
[ 47%] Linking C static library libosal.a
[ 47%] Built target osal
Scanning dependencies of target cfe-core
[ 48%] Building C object cfe-core/CMakeFiles/cfe-core.dir/src/es/cfe_es_api.c.o
[ 49%] Building C object cfe-core/CMakeFiles/cfe-core.dir/src/es/cfe_es_apps.c.o
[ 49%] Building C object cfe-core/CMakeFiles/cfe-core.dir/src/es/cfe_es_backgroundtask.c.o
[ 50%] Building C object cfe-core/CMakeFiles/cfe-core.dir/src/es/cfe_es_cds.c.o
[ 51%] Building C object cfe-core/CMakeFiles/cfe-core.dir/src/es/cfe_es_cds_mempool.c.o
[ 51%] Building C object cfe-core/CMakeFiles/cfe-core.dir/src/es/cfe_es_erlog.c.o
[ 52%] Building C object cfe-core/CMakeFiles/cfe-core.dir/src/es/cfe_es_objtab.c.o
[ 53%] Building C object cfe-core/CMakeFiles/cfe-core.dir/src/es/cfe_es_perf.c.o
[ 54%] Building C object cfe-core/CMakeFiles/cfe-core.dir/src/es/cfe_es_shell.c.o
[ 54%] Building C object cfe-core/CMakeFiles/cfe-core.dir/src/es/cfe_es_start.c.o
[ 56%] Building C object cfe-core/CMakeFiles/cfe-core.dir/src/es/cfe_es_syslog.c.o
[ 57%] Building C object cfe-core/CMakeFiles/cfe-core.dir/src/es/cfe_es_task.c.o
/home/vagrant/cFS/cfe/fsw/cfe-core/src/es/cfe_es_task.c:635:52: error: cast from 'CFE_SB_MsgPtr_t' (aka 'CFE_SB_Msg_t *') to 'CFE_ES_SendMemPoolStats_t *' increases required alignment from 4 to 8 [-Werror,-Wcast-align]
                        CFE_ES_SendMemPoolStatsCmd((CFE_ES_SendMemPoolStats_t*)Msg);
                                                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
cfe-core/CMakeFiles/cfe-core.dir/build.make:326: recipe for target 'cfe-core/CMakeFiles/cfe-core.dir/src/es/cfe_es_task.c.o' failed
make[7]: *** [cfe-core/CMakeFiles/cfe-core.dir/src/es/cfe_es_task.c.o] Error 1
CMakeFiles/Makefile2:381: recipe for target 'cfe-core/CMakeFiles/cfe-core.dir/all' failed
make[6]: *** [cfe-core/CMakeFiles/cfe-core.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make[5]: *** [all] Error 2
CMakeFiles/llvm_default_cpu1-all.dir/build.make:57: recipe for target 'CMakeFiles/llvm_default_cpu1-all' failed
make[4]: *** [CMakeFiles/llvm_default_cpu1-all] Error 2
CMakeFiles/Makefile2:356: recipe for target 'CMakeFiles/llvm_default_cpu1-all.dir/all' failed
make[3]: *** [CMakeFiles/llvm_default_cpu1-all.dir/all] Error 2
CMakeFiles/Makefile2:138: recipe for target 'CMakeFiles/mission-all.dir/rule' failed
make[2]: *** [CMakeFiles/mission-all.dir/rule] Error 2
Makefile:201: recipe for target 'mission-all' failed
make[1]: *** [mission-all] Error 2
Makefile:120: recipe for target 'all' failed
make: *** [all] Error 2
vagrant@ubuntu-bionic:~/cFS$

[jphickey]

What version/branch did you build? Is this CFE 6.7 (official) or CFE 6.8 (main)?

There were a number of alignment fixes that went into the last release cycle. I would expect this error on CFE 6.7, but not in CFE 6.8.

[ianamason]

I just did

git clone https://github.com/nasa/cFS.git
cd cFS
git submodule init
git submodule update

So I guess I am on master?

[jphickey]

Also note - if your target architecture does not have strict alignment requirements (e.g. x86) you can also remove -Wcast-align from the example compile options (in mission_build_custom.c) or perhaps replace with -Wno-cast-align if needed. This will squelch the warning and let the build proceed, at least.

[ianamason]

Perfect, I will try that. Thanks.

[jphickey]

I took a closer look at this one -- this is actually for the CFE_ES_SEND_MEM_POOL_STATS_CC and this is related to the fact that it has a direct memory pointer for the pool address and the message are only aligned to 32-bit values.

So yes -- this is still an isssue in 6.8, but it will be fixed by nasa/cFE#651

[ianamason]

Getting close!

[100%] Linking C executable core-cpu1
../psp/libpsp-pc-linux.a(cfe_psp_timer.c.o): In function `CFE_PSP_GetTime':
/home/vagrant/cFS/psp/fsw/pc-linux/src/cfe_psp_timer.c:83: undefined reference to `OS_GetLocalTime'
../psp/libpsp-pc-linux.a(cfe_psp_timer.c.o): In function `CFE_PSP_Get_Timebase':
/home/vagrant/cFS/psp/fsw/pc-linux/src/cfe_psp_timer.c:166: undefined reference to `OS_GetLocalTime'
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I also had to futz with arch_build_custom.cmake and for some strange reason add a newline at the end of
sch_lab_version.h. Any hints for getting over the final hurdle? @jphickey

[jphickey]

These functions should be part of libosal.a -- can you confirm? They would be in the os-impl-posix-gettime.o unit.
Can you share the full link line (i.e. build with make VERBOSE=1 which shows the full command line)

Does clang support the --whole-archive switch?

[ianamason]

[100%] Linking C executable core-cpu1
cd /home/vagrant/cFS/build/llvm/default_cpu1/cpu1 && /usr/bin/cmake -E cmake_link_script CMakeFiles/core-cpu1.dir/link.txt --verbose=1
/usr/lib/llvm-10/bin/clang -g  -Wl,--export-dynamic -rdynamic CMakeFiles/core-cpu1.dir/src/target_config.c.o  -o core-cpu1 ../cfe-core/libcfe-core.a ../osal/libosal.a ../psp/libpsp-pc-linux.a ../osal/libosal_bsp.a -lpthread -ldl -lrt

and

vagrant@ubuntu-bionic:~/cFS$ clang --whole-archive
clang: error: unsupported option '--whole-archive'

[ianamason]

Followed by

vagrant@ubuntu-bionic:~/cFS/build/llvm/default_cpu1/osal$ nm libosal.a | grep OS_GetLocalTime
0000000000000000 T OS_GetLocalTime
                 U OS_GetLocalTime_Impl
0000000000000000 T OS_GetLocalTime_Impl
vagrant@ubuntu-bionic:~/cFS/build/llvm/default_cpu1/osal$

[ianamason]

Finally

/usr/lib/llvm-10/bin/clang -g  -Wl,--export-dynamic -rdynamic CMakeFiles/core-cpu1.dir/src/target_config.c.o  -o core-cpu1 ../cfe-core/libcfe-core.a ../psp/libpsp-pc-linux.a ../osal/libosal.a ../osal/libosal_bsp.a  -lpthread -ldl -lrt

does not complain.

[ianamason]

Thanks a lot for your help @jphickey

[jphickey]

OK -- this explains it, as the current code only puts in the --whole-archive switch if using a gcc compiler. So your final link line doesn't have this, which means the unit that had this function wasn't getting pulled in, as it is referenced by PSP rather than CFE.

Looks like you swapped the order - which is fine - but be advised there could be other other symbols which are not referenced at all in either CFE nor PSP - but should still be linked into the final EXE so they'd be available for apps. That was the reason for using --whole-archive.

There is also an existing issue nasa/cFE#722 to implement this in a better way that doesn't depend on toolchain-specific feature.

[ianamason]

Thanks. Actually I am using using gllvm so I can get all the llvm bitcode associated with the build, so the
link is largely moot.

[skliper]

See also nasa/cFE#2204