SEMS Dev Env

For machines that are set up with SEMS development environment modules (e.g. usually nfs mounted or rsynced under /project/sems/) the configuration of Trilinos is easy. One just sources a single environment script then loads a single *.cmake configurations file and then select the desired enables.

To take advantage of this SEMS env, first source the script load_sems_dev_env.sh as:

$ cd <some-build-dir>/
$ source $TRILINOS_DIR/cmake/load_sems_dev_env.sh

(where TRILINOS_DIR points to the base Trilinos git repo source dir, e.g. $HOME/Trilinos). This does a module purge then loads SEMS modules for compilers, MPI, Git, CMake, Python, and several key TPLs that can be used by Trilinos like Boost, Zlib, HDF5, Netcdf, ParMETIS, and SuperLU. To see what is loaded, run module list which should return something similar to:

$ module list

Currently Loaded Modules:
  1) sems-archive-env                   8) sems-archive-boost/1.63.0/base
  2) sems-archive-python/2.7.9          9) sems-archive-zlib/1.2.8/base
  3) sems-archive-cmake/3.17.1         10) sems-archive-hdf5/1.10.6/parallel
  4) sems-archive-ninja_fortran/1.8.2  11) sems-archive-netcdf/4.7.3/parallel
  5) sems-archive-git/2.10.1           12) sems-archive-parmetis/4.0.3/parallel
  6) sems-archive-gcc/7.2.0            13) sems-archive-scotch/6.0.3/nopthread_64bit_parallel
  7) sems-archive-openmpi/1.10.1       14) sems-archive-superlu/4.3/base

(NOTE: Actual modules and versions may be different for the current version of Trilinos.)

Once a SEMS Dev Env has been loaded, then one can configure, build, and test as (for example):

$ cmake \
  -C $TRILINOS_DIR/cmake/std/sems/SEMSDevEnv.cmake \
  -DBUILD_SHARED_LIBS=ON \
  -DCMAKE_BUILD_TYPE=DEBUG \
  -DTPL_ENABLE_MPI=ON \
  -DTrilinos_ENABLE_<PKG0>=ON \
  -DTrilinos_ENABLE_<PKG1>=ON \
  -DTrilinos_ENABLE_TESTS=ON \
  $TRILNOS_DIR
$ make -j10
$ ctest -j10

NOTE: The SEMSDevEnv.cmake file can alternatively be loaded by passing:

  -D Trilinos_CONFIGURE_OPTIONS_FILE:STRING=cmake/std/sems/SEMSDevEnv.cmake

into the cmake command-line instead of the -C argument. This has the advantage of not needing to specify the base Trilinos source dir (again) and it will automatically cause a reconfigure if the file SEMSDevEnv.cmake or any of the files it includes change. See Trilinos_CONFIGURE_OPTIONS_FILE for more details.

The search paths for all of the TPLs supported by the loaded SEMS Dev Env will automatically be set. All that is needed is to enable the desired TPLs. The same loaded SEMS Dev Env can be used for MPI or serial (non-MPI) builds and shared lib or static lib builds. The SEMS Dev Env only needs to be changed when one wants a different compiler/version and/or MPI/version and/or CMake/version using, for example:

$ source $TRILINOS_DIR/cmake/load_sems_dev_env.sh \
   sems-archive-gcc/8.3.0  sems-archive-openmpi/1.10.1  sems-archive-cmake/3.19.1

Different compilers (GCC, Clang, Intel) and several versions of each can be selected. Different versions of OpenMPI and CMake can also be selected (see modules starting with sems- from module avail). See the default versions for each of these and more documentation in the script load_sems_dev_env.sh itself. The defaults for GCC, OpenMPI, and CMake are selected match the standard CI build for Trilinos (see checkin-test-sems.sh and the post-push CI build).

NOTES:

• WARNING: Sourcing load_sems_dev_env.sh clears out any modules one may have set before loading the new SEMS modules. Therefore, to avoid messing up your current env, one may want to start a new shell.

• If the script load_sems_dev_env.sh is not used, then one can load the various SEMS modules manually and then configure as shown above. The script load_sems_dev_env.sh does not have to be used.

• To unload a loaded dev env and get back to no loaded modules, source the script unload_sems_dev_env.sh. This will do module purge.

• On Mac OSX, this script will load an env but it loads a different env from Linux and Trilinos currently does not build with that env. This script can really only be used for Linux machines with the SEMS env and sh compatible shells (e.g. bash, sorry no (t)csh versions are currently available).

• This script currently uses the sems-archive-* modules instead of the new sems-* modules. But one can mix in the new sems-* modules for independent tools like sems-cmake/x.y.z, sems-ninja/x.y.z, etc., as these do not conflict with the compiler, MPI, or the related TPLs.