Use HIP as language in CMake

[ye-luo]

Since CMake 3.21, HIP can be used a language just like CUDA.
Right now we need both find_package(HIP) and find_package(hip). The former one needs findHIP module but it is not always available, for example AOMP release doesn't include this file. So I have to do funny gymnastics to use FindHIP.cmake from /opt/rocm but point it to the hip in the AOMP installation. hip-config.cmake is the recommended way and it works with /opt/rocm and AOMP installation. So we'd better to remove the former legacy module ASAP.

Note that: 3.21.3 is needed as some change was made https://cmake.org/cmake/help/latest/release/3.21.html#id3

[prckent]

Note: spack already supports 3.21.4, so we'll have no trouble with this update in the nightlies and soon to be added hip builds

[quantumsteve]

Waiting until nitrogen is back online.

[quantumsteve]

If I remove find_package(HIP) or place enable_language(HIP) before it, I get a linking error

https://github.com/QMCPACK/qmcpack/blob/develop/CMakeLists.txt#L792

[ 66%] Linking CXX executable test_io_hdf5
cd /home/svh/qmcpack/build/src/io/hdf/tests && /home/svh/spack/opt/spack/linux-rhel8-zen/gcc-8.4.1/cmake-3.21.4-pra5545ek64dksxro3piogh44xrbhhjy/bin/cmake -E cmake_link_script CMakeFiles/test_io_hdf5.dir/link.txt --verbose=1
/usr/bin/c++  -fopenmp -finline-limit=1000 -fstrict-aliasing -funroll-all-loops -Wno-deprecated -Wvla -Wcomment -Wmisleading-indentation -Wmaybe-uninitialized -Wuninitialized -Wreorder -Wno-unknown-pragmas -Wno-sign-compare -ffast-math -march=native -O2 -DNDEBUG CMakeFiles/test_io_hdf5.dir/test_hdf_archive.cpp.o CMakeFiles/test_io_hdf5.dir/test_hdf_parallel.cpp.o CMakeFiles/test_io_hdf5.dir/test_hdf_reshape.cpp.o CMakeFiles/test_io_hdf5.dir/test_hdf_hyperslab.cpp.o -o test_io_hdf5  -Wl,-rpath,/opt/rocm-4.5.0/hip/lib ../../../Message/libcatch_main.a ../libqmcio_hdf.a ../../../Message/libmessage.a /usr/lib64/libhdf5.so ../../OhmmsData/libqmcio_xml.a ../../../Containers/libcontainers.a ../../../Platforms/libplatform_runtime.a ../../../Platforms/libplatform_host_runtime.a ../../../Platforms/OMPTarget/libplatform_omptarget_runtime.a /opt/rocm-4.5.0/hip/lib/libamdhip64.so.4.4.40500 -L"/opt/rocm/llvm/lib/clang/13.0.0/../lib/linux" -lclang_rt.builtins-x86_64 /usr/lib64/libxml2.so 
/usr/bin/ld: cannot find -lclang_rt.builtins-x86_64
collect2: error: ld returned 1 exit status
make[3]: *** [src/io/hdf/tests/CMakeFiles/test_io_hdf5.dir/build.make:156: src/io/hdf/tests/test_io_hdf5] Error 1
make[3]: Leaving directory '/home/svh/qmcpack/build'
make[2]: *** [CMakeFiles/Makefile2:2835: src/io/hdf/tests/CMakeFiles/test_io_hdf5.dir/all] Error 2
make[2]: Leaving directory '/home/svh/qmcpack/build'
make[1]: *** [CMakeFiles/Makefile2:2842: src/io/hdf/tests/CMakeFiles/test_io_hdf5.dir/rule] Error 2
make[1]: Leaving directory '/home/svh/qmcpack/build'
make: *** [Makefile:569: test_io_hdf5] Error 2
[svh@nitrogen build]$ 

Linking succeeds if I replace /opt/rocm/llvm/lib/clang/13.0.0/../lib/linux with /opt/rocm/llvm/lib/clang/13.0.0/lib/linux

[ye-luo]

Why "/usr/bin/c++" is used for linking?

[ye-luo]

I guess you are trying legacy CUDA code path. Then GCC is OK. I think the key question is who adds -L"/opt/rocm/llvm/lib/clang/13.0.0/../lib/linux". Look like some no-good CMake. I mean "-L" is not good.

Here is from using find_package(HIP)

-L"/home/yeluo/rocm/aomp/lib/clang/14.0.0/include/../lib/linux"\

Maybe rocm 4.5 does better? It seems you are using 4.3.

[quantumsteve]

At the moment I have ENABLE_HIP=ON and ENABLE_ROCM=ON.

I haven't tried QMC_CUDA=ON and QMC_CUDA2HIP=ON.