Skip to content

Home

Jump to bottom
Kevin Huck edited this page Mar 23, 2023 · 18 revisions

Welcome to the APEX wiki!

On this page, there will be several links to get you up and running with different APEX configurations on several different machines.

Note: For an APEX tutorial, please see the official APEX Tutorial.

The full APEX documentation is available on the Github web site.

Building APEX dependencies

APEX has a number of optional dependencies, here's how to build each one. These dependencies can optionally be built in-place with APEX, and the best scenario is to use installed modules on your system if they are available.

Useful Dependencies

  • Binutils
  • OTF2
  • Active Harmony
  • OMPT (OpenMP profiling support for GCC only)
  • PAPI

Supported Vendor or Open Source Runtimes/Models

  • CUDA (NVIDIA)
  • HIP (AMD ROCm)
  • Level0 (Intel OneAPI)
  • MPI (OpenMPI, MPICH variants)
  • HPX
  • StarPU
  • PhiProf

Less Common Libraries

  • TC Malloc (heap manager from Google Performance Tools, may reduce overhead, YMMV)
  • JE Malloc (another heap manager, may reduce overhead, YMMV)
  • Perfetto Native Trace Output (not as compact/useful as json.gz output, IMHO)

Binutils

Required to support address translation for OpenMP/CUDA/HIP/Level0/Pthreads, in order to find code locations of parallel regions, loops, kernels, functors, lambdas, etc. based on instruction pointers.

Sometimes, the system installed binutils will be sufficient - it depends whether it has demangling support (libiberty) as well. APEX can automatically download and build Binutils as part of its build with the -DAPEX_BUILD_BFD=TRUE CMake flag, or you can build your own Binutils with:

wget http://ftp.gnu.org/gnu/binutils/binutils-2.37.tar.bz2
tar -xjf binutils-2.37.tar.bz2
cd binutils-2.37
configure CC=${CC} CXX=${CXX} CFLAGS=-fPIC CXXFLAGS=-fPIC LDFLAGS=-fPIC --prefix=${INSTALL_PREFIX} --disable-dependency-tracking --enable-interwork --disable-multilib --enable-shared --enable-64-bit-bfd --target=${TARGET_ARCH} --enable-install-libiberty --disable-gold --program-prefix=g --disable-nls --disable-ld --disable-lto --disable-gas
make MAKEINFO=true
make MAKEINFO=true install

Active Harmony

Required to run APEX policies. Active Harmony has some code that doesn't compile with pedantic warnings as errors, so in that case, use the make options -i or -k to "ignore errors" and/or "keep going". APEX can automatically build Active Harmony with the -DAPEX_BUILD_ACTIVEHARMONY=TRUE CMake flag.

wget https://www.dyninst.org/sites/default/files/downloads/harmony/ah-4.6.0.tar.gz
tar -xzf ah-4.6.0.tar.gz
cd activeharmony-4.6.0
make MPICC=mpicc_disabled CFLAGS=-fPIC CXXFLAGS=-fPIC LDFLAGS=-fPIC
make MPICC=mpicc_disabled CFLAGS=-fPIC CXXFLAGS=-fPIC LDFLAGS=-fPIC install prefix=/path/to/install/activeharmony

OTF2

Required to generate OTF2 trace files (read by Vampir). APEX can build OTF2 with the -DAPEX_BUILD_OTF2=TRUE CMake flag, or you can build your own OTF2 with:

wget https://www.vi-hps.org/cms/upload/packages/otf2/otf2-2.1.1.tar.gz
tar -xzf otf2-2.1.1.tar.gz
cd otf2-2.1.1
./configure --prefix=/path/to/install/otf2 --enable-shared
make
make install

OMPT

Because the GNU compiler/runtime provides no OMPT support, this library is required in order to measure GCC OpenMP using the OpenMP Tools Interface. This support is limited to host-side measurement (no target offload support). Other LLVM-based compiler/runtimes already have OMPT 5.0/5.1 support built-in, for example LLVM Clang/Clang++/Flang, AMD Clang/Clang++/Flang, Intel, NVIDIA NVHPC. APEX can automatically build this support with the -DAPEX_BUILD_OMPT=TRUE CMake flag, or you can build it yourself with:

wget http://tau.uoregon.edu/LLVM-openmp-2021-05-14.tar.gz
tar -xzf LLVM-openmp-2021-05-14.tar.gz
cd LLVM-openmp
mkdir build
cd build
cmake -DCMAKE_C_COMPILER=${CC} -DCMAKE_CXX_COMPILER=${CXX} -DCMAKE_INSTALL_PREFIX=/path/to/install/ompt -DCMAKE_BUILD_TYPE=Release ../project_ompt
make
make install
Clone this wiki locally