Steps to compile and run the code

[fdiblen]

Let's describe the steps to compile the code and run it on various infrastructures (DAS5, DAS6, AWS, your local system). We can then use this information to create a developer documentation.

[jspaaks]

aws ec2 g4dn.xlarge instance (Euro 0.526 / hour that it is running + Euro 1.20 / month for 12 GB gp2 SSD storage space)

storage 12 GB, not default 8 GB

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.3 LTS
Release:    20.04
Codename:   focal
$ lspci | grep -i nvidia
00:1e.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1)
$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get install build-essential
$ gcc --version
gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ g++ --version
g++ (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ make --version
GNU Make 4.2.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
# the cmake version from the repositories is not enough, so download newer version >= 3.17
$ mkdir tmp
$ cd tmp
$ wget https://github.com/Kitware/CMake/releases/download/v3.22.2/cmake-3.22.2.tar.gz
$ tar -xvzf cmake-3.22.2.tar.gz
$ cd cmake-3.22.2
# cmake needs libssl headers
$ sudo apt install libssl-dev
$ ./bootstrap
$ make
$ sudo make install
$ cmake --version
cmake version 3.22.2

CMake suite maintained and supported by Kitware (kitware.com/cmake).
$ sudo apt-get install linux-headers-$(uname -r)
$ sudo apt remove ^nvidia --purge
$ sudo apt remove ^libnvidia --purge
$ sudo apt autoremove
$ sudo apt autoclean
$ sudo reboot
$ sudo apt install nvidia-driver-510
$ sudo reboot
$ lsmod | grep -i nvidia
nvidia_uvm           1044480  0
nvidia_drm             61440  2
nvidia_modeset       1159168  2 nvidia_drm
nvidia              39059456  85 nvidia_uvm,nvidia_modeset
drm_kms_helper        237568  1 nvidia_drm
drm                   548864  6 drm_kms_helper,nvidia,nvidia_drm
$ nvidia-smi
Wed Feb 16 12:07:08 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.47.03    Driver Version: 510.47.03    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla T4            Off  | 00000000:00:1E.0 Off |                    0 |
| N/A   32C    P0    26W /  70W |      0MiB / 15360MiB |     10%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
$ sudo apt install nvidia-cuda-toolkit
$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243
$ mkdir -p github/recruit
$ cd github/recruit
$ git clone https://github.com/nlesc-recruit/CUDA-wrappers (approximate version 0.1.0)
$ cd CUDA-wrappers
$ mkdir build
$ cd build
$ cmake -S .. -B .
$ make

[abelsiqueira]

I tried some others things before, but I think the following should work regardless of what I did.
I could not cmake, even after changing the version.

$ ssh USER@fs0.das5.cs.vu.nl
[USER@fs0 ~]$ module load cuda10.1
[USER@fs0 ~]$ git clone https://github.com/nlesc-recruit/CUDA-wrappers
[USER@fs0 ~]$ cd CUDA-wrappers/examples/vector_add/
[USER@fs0 vector_add]$ g++ ../../cu/*.cc -lcuda -lnvrtc vector_add.cc -I../../ -o vector_add
[USER@fs0 vector_add]$ alias gpurun="srun -N 1 -C TitanX --gres=gpu:1"
[USER@fs0 vector_add]$ gpurun ./vector_add
hurray! 2

Additional information

[USER@fs0 vector_add]$ gpurun nvidia-smi
Thu Feb 17 14:35:07 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 455.32.00    Driver Version: 455.32.00    CUDA Version: 11.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce GTX TIT...  On   | 00000000:03:00.0 Off |                  N/A |
| 22%   35C    P8    28W / 250W |      0MiB / 12212MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

[USER@fs0 vector_add]$ lsb_release -a
LSB Version:	:core-4.1-amd64:core-4.1-noarch
Distributor ID:	CentOS
Description:	CentOS Linux release 7.4.1708 (Core) 
Release:	7.4.1708
Codename:	Core

[USER@fs0 vector_add]$ gpurun lspci | grep -i nvidia
03:00.0 VGA compatible controller: NVIDIA Corporation GM200 [GeForce GTX TITAN X] (rev a1)
03:00.1 Audio device: NVIDIA Corporation GM200 High Definition Audio (rev a1)

[USER@fs0 vector_add]$ g++ --version
g++ (GCC) 6.3.0
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[USER@fs0 vector_add]$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243

[abelsiqueira]

DAS has cmake 3.15, which can be loaded with

$ module load cmake/3.15.4

But then we need to change CMakeLists.txt Lines

• 1: Version should be 3.15
• 17: CUDAToolkit should be changed to CUDA
  • 17: Remove find_package(CUDAToolkit REQUIRED). Since 3.10 and before 3.17, CUDA is identified by the language CUDA ref.

Then we can enter, in addition to the other commands I showed before, the following commands:

$ # Inside the CUDA-Wrappers repo
$ mkdir build
$ cd build
$ cmake -S ../ -B .
$ make
$ cd ../examples/vector_add
$ make
$ gpurun vector_add

[abelsiqueira]

What is the action for closing this issue? Adding the steps to compile in the README.dev?
edit: actually, there is already #33

[fdiblen]

We will use the information in this issue while working on #33. Closing the issue.

[jspaaks]

For the record, here are the steps for downloading the sources of cmake 3.17, building it, then installing in a user space directory (on linux):

# make temp dir
cd $(mktemp -d --tmpdir download-XXXXXX)

# download sources
wget https://cmake.org/files/v3.17/cmake-3.17.5.tar.gz

# extract gzipped file
tar -xzvf cmake-3.17.5.tar.gz

# cd into the extracted directory
cd cmake-3.17.5

# configure with the install directory (choose whichever you like)
./configure --prefix ~/opt/cmake

# build
make 

# install at the prefix location
make install