Add -ccbin/--compiler-bindir flag for nvcc compiler

[OpportunityLiu]

See https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#file-and-path-specifications-compiler-bindir

-ccbin flag should be given as xmake configured cxx compiler

If not given

the default host compiler executable (gcc and g++ on Linux, clang and clang++ on Mac OS X, and cl.exe on Windows) found in the current execution search path will be used

See https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#supported-host-compilers

[waruqi]

You can try xmake f --nvcc=/xxx/bin/nvcc --cxx=/xxx/bin/gcc

[OpportunityLiu]

跟这有啥关系啊，就算是workaround也应该是
add_cuflags("-ccbin $(cxx)")
这种吧，我现在就是这么搞的

设了cxx也拦不住nvcc用啥啊

[waruqi]

我以为你要改nvcc的配置，如果只是加flags，你现在的add_cuflags不是已经解决了么，还有什么问题呢？

[OpportunityLiu]

意思就是我觉得这句应该是默认的配置，

xmake f --cxx=clang++

之后我的预期就是已经带这个flag了，而不需要在配置文件里再写什么

[waruqi]

就是说 当前cxx的配置 默认通过-ccbin透传给nvcc? 但是-ccbin是设置bin目录，而如果cxx=gcc 并没有给定全路径，-ccbin应该不支持吧，而且cc/cxx分别指定了c/c++编译器，两者也许并不相同，如何同时透传给nvcc呢？

[OpportunityLiu]

所以才需要更复杂的脚本来把cxx展开成全路径，上面那个addcuflags能用只是因为clang在path里，甚至可以认为是巧合（虽然一般就是这么巧）

只有编译cu才用nvcc，我也没见过什么cupp之类的扩展名，应该是只给cxx就好了，我明天再看看文档吧

[waruqi]

如果只是 *.c + *.cu 的混合编译工程，是不设置cxx，只设置cc，如果是 *.c + *.cpp + *.cu的混编工程，会同时设置cc/cxx，那应该传哪个给 nvcc呢？ cc? cxx?

[OpportunityLiu]

cu 实际是cpp文件

指定clang

指定clang++

可以看到是通过 -x c++ flag 强行调c++编译器的，所以只要给cxx就可以，只设了cc的时候给cc也没关系

[waruqi]

ok, 我回头看下。

[OpportunityLiu]

而且说是bin dir，只给dir就会这样

所以还是要给到实际的可执行文件

[waruqi]

这样其实比传递dir更好处理些

[OpportunityLiu]

又踩到一个坑，在windows上只支持cl.exe，设别的它也会试图用cl的调用方式调用，即使clang-cl也会报

nvcc fatal   : Host compiler targets unsupported OS.

[waruqi]

那win上就没必要传ccbin了吧，仅对linux/macos 下设置下ccbin 绑定cxx

[OpportunityLiu]

mac上标的也是只支持clang啊，可惜我没机器测

[waruqi]

我在linux/macos上加上支持了，你更新下dev分支试试，默认会透传cxx的配置，你也可以手动通过xmake f --cu-cxx=gcc 来修改传入nvcc的cxx编译器

[OpportunityLiu]

并没有自动用cxx的配置啊，我搞错了什么吗？

➜  PowerFlow git:(master) ✗ xmake f -c                                    
checking for the architecture ... x86_64
checking for the Cuda SDK directory ... /usr/local/cuda
➜  PowerFlow git:(master) ✗ xmake f --cc=clang --cxx=clang++ --ld=clang++ -v
checking for the architecture ... x86_64
checking for the Cuda SDK directory ... /usr/local/cuda
checking for the gcc ... /usr/bin/gcc
checking for the cuda host c++ compiler (cu-cxx) ... gcc
configure
{
    ld = clang++
    cxx = clang++
    mode = release
    ccache = true
    host = linux
    cu-cxx = /usr/bin/gcc
    plat = linux
    kind = static
    buildir = build
    cuda = /usr/local/cuda
    cc = clang
    arch = x86_64
}

然后就直接-ccbin /usr/bin/gcc了

[waruqi]

嗯 如果不手动改cxx的话 ，cxx 和ccbin 是按相同规则检测的，完全同步，如果手动改写cxx的话，需要手动配置xmake f --cu-cxx=gcc --cxx=gcc

[OpportunityLiu]

好的

[OpportunityLiu]

还需要个 -cu-ld 来指定 nvcc 当连接器用的时候给的 -ccbin 参数

[OpportunityLiu]

未指定

➜  /usr/local/cuda/bin/nvcc -o build/linux/x86_64/release/main build/.objs/main/linux/x86_64/release/./main.cu.o -m64 -Xcompiler -m64 -L/usr/local/cuda/lib -Xlinker -rpath=/usr/local/cuda/lib -dryrun
#$ _SPACE_= 
#$ _CUDART_=cudart
#$ _HERE_=/usr/local/cuda/bin
#$ _THERE_=/usr/local/cuda/bin
#$ _TARGET_SIZE_=
#$ _TARGET_DIR_=
#$ _TARGET_DIR_=targets/x86_64-linux
#$ TOP=/usr/local/cuda/bin/..
#$ NVVMIR_LIBRARY_DIR=/usr/local/cuda/bin/../nvvm/libdevice
#$ LD_LIBRARY_PATH=/usr/local/cuda/bin/../lib:
#$ PATH=/usr/local/cuda/bin/../nvvm/bin:/usr/local/cuda/bin:/home/dps/.local/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
#$ INCLUDES="-I/usr/local/cuda/bin/../targets/x86_64-linux/include"  
#$ LIBRARIES=  "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib/stubs" "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib"
#$ CUDAFE_FLAGS=
#$ PTXAS_FLAGS=
#$ nvlink --arch=sm_30 --register-link-binaries="/tmp/tmpxft_00003ac2_00000000-2_main_dlink.reg.c"  -m64 -L"/usr/local/cuda/lib"   "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib/stubs" "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib" -cpu-arch=X86_64 "build/.objs/main/linux/x86_64/release/./main.cu.o"  -o "/tmp/tmpxft_00003ac2_00000000-4_main_dlink.sm_30.cubin"
#$ fatbinary --create="/tmp/tmpxft_00003ac2_00000000-3_main_dlink.fatbin" -64 -link "--image=profile=sm_30,file=/tmp/tmpxft_00003ac2_00000000-4_main_dlink.sm_30.cubin" --embedded-fatbin="/tmp/tmpxft_00003ac2_00000000-3_main_dlink.fatbin.c" 
#$ rm /tmp/tmpxft_00003ac2_00000000-3_main_dlink.fatbin
#$ gcc -c -x c++ -DFATBINFILE="\"/tmp/tmpxft_00003ac2_00000000-3_main_dlink.fatbin.c\"" -DREGISTERLINKBINARYFILE="\"/tmp/tmpxft_00003ac2_00000000-2_main_dlink.reg.c\"" -I. -D__NV_EXTRA_INITIALIZATION= -D__NV_EXTRA_FINALIZATION= -D__CUDA_INCLUDE_COMPILER_INTERNAL_HEADERS__  -m64 "-I/usr/local/cuda/bin/../targets/x86_64-linux/include"    -D__CUDACC_VER_MAJOR__=10 -D__CUDACC_VER_MINOR__=1 -D__CUDACC_VER_BUILD__=168 -m64 -o "/tmp/tmpxft_00003ac2_00000000-5_main_dlink.o" "/usr/local/cuda/bin/crt/link.stub" 
#$ g++ -m64 -m64 -o "build/linux/x86_64/release/main" -Wl,--start-group "/tmp/tmpxft_00003ac2_00000000-5_main_dlink.o" "build/.objs/main/linux/x86_64/release/./main.cu.o" -Xlinker -rpath=/usr/local/cuda/lib -L"/usr/local/cuda/lib"   "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib/stubs" "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib" -lcudadevrt  -lcudart_static  -lrt -lpthread  -ldl  -Wl,--end-group 

指定 clang

➜  /usr/local/cuda/bin/nvcc -o build/linux/x86_64/release/main build/.objs/main/linux/x86_64/release/./main.cu.o -m64 -Xcompiler -m64 -L/usr/local/cuda/lib -Xlinker -rpath=/usr/local/cuda/lib -ccbin clang -dryrun
#$ _SPACE_= 
#$ _CUDART_=cudart
#$ _HERE_=/usr/local/cuda/bin
#$ _THERE_=/usr/local/cuda/bin
#$ _TARGET_SIZE_=
#$ _TARGET_DIR_=
#$ _TARGET_DIR_=targets/x86_64-linux
#$ TOP=/usr/local/cuda/bin/..
#$ NVVMIR_LIBRARY_DIR=/usr/local/cuda/bin/../nvvm/libdevice
#$ LD_LIBRARY_PATH=/usr/local/cuda/bin/../lib:
#$ PATH=/usr/local/cuda/bin/../nvvm/bin:/usr/local/cuda/bin:/home/dps/.local/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
#$ INCLUDES="-I/usr/local/cuda/bin/../targets/x86_64-linux/include"  
#$ LIBRARIES=  "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib/stubs" "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib"
#$ CUDAFE_FLAGS=
#$ PTXAS_FLAGS=
#$ nvlink --arch=sm_30 --register-link-binaries="/tmp/tmpxft_00003ad5_00000000-2_main_dlink.reg.c"  -m64 -L"/usr/local/cuda/lib"   "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib/stubs" "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib" -cpu-arch=X86_64 "build/.objs/main/linux/x86_64/release/./main.cu.o"  -o "/tmp/tmpxft_00003ad5_00000000-4_main_dlink.sm_30.cubin"
#$ fatbinary --create="/tmp/tmpxft_00003ad5_00000000-3_main_dlink.fatbin" -64 -link "--image=profile=sm_30,file=/tmp/tmpxft_00003ad5_00000000-4_main_dlink.sm_30.cubin" --embedded-fatbin="/tmp/tmpxft_00003ad5_00000000-3_main_dlink.fatbin.c" 
#$ rm /tmp/tmpxft_00003ad5_00000000-3_main_dlink.fatbin
#$ clang -c -x c++ -DFATBINFILE="\"/tmp/tmpxft_00003ad5_00000000-3_main_dlink.fatbin.c\"" -DREGISTERLINKBINARYFILE="\"/tmp/tmpxft_00003ad5_00000000-2_main_dlink.reg.c\"" -I. -D__NV_EXTRA_INITIALIZATION= -D__NV_EXTRA_FINALIZATION= -D__CUDA_INCLUDE_COMPILER_INTERNAL_HEADERS__  -m64 "-I/usr/local/cuda/bin/../targets/x86_64-linux/include"    -D__CUDACC_VER_MAJOR__=10 -D__CUDACC_VER_MINOR__=1 -D__CUDACC_VER_BUILD__=168 -m64 -o "/tmp/tmpxft_00003ad5_00000000-5_main_dlink.o" "/usr/local/cuda/bin/crt/link.stub" 
#$ clang++ -m64 -m64 -o "build/linux/x86_64/release/main" -Wl,--start-group "/tmp/tmpxft_00003ad5_00000000-5_main_dlink.o" "build/.objs/main/linux/x86_64/release/./main.cu.o" -Xlinker -rpath=/usr/local/cuda/lib -L"/usr/local/cuda/lib"   "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib/stubs" "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib" -lcudadevrt  -lcudart_static  -lrt -lpthread  -ldl  -Wl,--end-group 

指定 clang++

➜  /usr/local/cuda/bin/nvcc -o build/linux/x86_64/release/main build/.objs/main/linux/x86_64/release/./main.cu.o -m64 -Xcompiler -m64 -L/usr/local/cuda/lib -Xlinker -rpath=/usr/local/cuda/lib -ccbin clang++ -dryrun
#$ _SPACE_= 
#$ _CUDART_=cudart
#$ _HERE_=/usr/local/cuda/bin
#$ _THERE_=/usr/local/cuda/bin
#$ _TARGET_SIZE_=
#$ _TARGET_DIR_=
#$ _TARGET_DIR_=targets/x86_64-linux
#$ TOP=/usr/local/cuda/bin/..
#$ NVVMIR_LIBRARY_DIR=/usr/local/cuda/bin/../nvvm/libdevice
#$ LD_LIBRARY_PATH=/usr/local/cuda/bin/../lib:
#$ PATH=/usr/local/cuda/bin/../nvvm/bin:/usr/local/cuda/bin:/home/dps/.local/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
#$ INCLUDES="-I/usr/local/cuda/bin/../targets/x86_64-linux/include"  
#$ LIBRARIES=  "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib/stubs" "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib"
#$ CUDAFE_FLAGS=
#$ PTXAS_FLAGS=
#$ nvlink --arch=sm_30 --register-link-binaries="/tmp/tmpxft_00003ae8_00000000-2_main_dlink.reg.c"  -m64 -L"/usr/local/cuda/lib"   "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib/stubs" "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib" -cpu-arch=X86_64 "build/.objs/main/linux/x86_64/release/./main.cu.o"  -o "/tmp/tmpxft_00003ae8_00000000-4_main_dlink.sm_30.cubin"
#$ fatbinary --create="/tmp/tmpxft_00003ae8_00000000-3_main_dlink.fatbin" -64 -link "--image=profile=sm_30,file=/tmp/tmpxft_00003ae8_00000000-4_main_dlink.sm_30.cubin" --embedded-fatbin="/tmp/tmpxft_00003ae8_00000000-3_main_dlink.fatbin.c" 
#$ rm /tmp/tmpxft_00003ae8_00000000-3_main_dlink.fatbin
#$ clang++ -c -x c++ -DFATBINFILE="\"/tmp/tmpxft_00003ae8_00000000-3_main_dlink.fatbin.c\"" -DREGISTERLINKBINARYFILE="\"/tmp/tmpxft_00003ae8_00000000-2_main_dlink.reg.c\"" -I. -D__NV_EXTRA_INITIALIZATION= -D__NV_EXTRA_FINALIZATION= -D__CUDA_INCLUDE_COMPILER_INTERNAL_HEADERS__  -m64 "-I/usr/local/cuda/bin/../targets/x86_64-linux/include"    -D__CUDACC_VER_MAJOR__=10 -D__CUDACC_VER_MINOR__=1 -D__CUDACC_VER_BUILD__=168 -m64 -o "/tmp/tmpxft_00003ae8_00000000-5_main_dlink.o" "/usr/local/cuda/bin/crt/link.stub" 
#$ clang++ -m64 -m64 -o "build/linux/x86_64/release/main" -Wl,--start-group "/tmp/tmpxft_00003ae8_00000000-5_main_dlink.o" "build/.objs/main/linux/x86_64/release/./main.cu.o" -Xlinker -rpath=/usr/local/cuda/lib -L"/usr/local/cuda/lib"   "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib/stubs" "-L/usr/local/cuda/bin/../targets/x86_64-linux/lib" -lcudadevrt  -lcudart_static  -lrt -lpthread  -ldl  -Wl,--end-group 

[OpportunityLiu]

不过现在已经有一个 cu-ld 了？貌似是可以链接的时候不用nvcc?

[waruqi]

默认设置的cu-ld=nvcc去链接，你也可以指定 cu-ld=gcc/g++ 去直接链接试试，应该也是可以的。。

[OpportunityLiu]

是可以链接，但是看上面的 -dryrun的结果，会漏掉nvlink生成的步骤（灰色部分）

[waruqi]

把 nvcc --ccbin gcc 全部设置到 --cu-ld就行了。

$ xmake f -c --cu-ld="/Developer/NVIDIA/CUDA-10.1/bin/nvcc -ccbin gcc"
$ xmake -r -v
[100%]: linking.release cuda_console
"/Developer/NVIDIA/CUDA-10.1/bin/nvcc -ccbin gcc" -o build/macosx/x86_64/release/cuda_console build/.objs/cuda_console/macosx/x86_64/release/src/main.cu.o -gencode arch=compute_30
,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_37,code=sm_37 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_6
0,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_70,code=compute_70 -m64 -Xlinker -arch -Xlinker x86_64 -L/Developer/NVID
IA/CUDA-10.1/lib -Xlinker -rpath -Xlinker /Developer/NVIDIA/CUDA-10.1/lib

[OpportunityLiu]

无所谓了，clang编译完用默认的gcc链接也不会怎么样，反正折腾了半天 nvlink 一碰 -flto 生成的 .o 就炸，还轮不到clang和gcc冲突