UncleBench

UncleBench is a tool for automating the running of complex benchmarks on HPC infrastructures. It relies on Jube as a benchmark engine but it can be customized to work with any benchmark engine.

In order to run a specific benchmark in a given platform you need to provide descriptions for both (benchmark and platform) in form of Jube files. Unclebench provides Jube recipes for several well known benchmarks in HPC, these recipies describe the whole cycle: downlad of files, compilation and preparation of code, execution, analysis and generation of reports. List of provided bencmarks:

• HPCC: http://icl.cs.utk.edu/hpcc/
• HPCG: http://www.hpcg-benchmark.org/
• HPL: http://www.netlib.org/benchmark/hpl/
• IO500: https://www.vi4io.org/std/io500/start
• IOR: https://github.com/LLNL/ior
• NAS: https://www.nas.nasa.gov/publications/npb.html
• STREAM: https://www.cs.virginia.edu/stream/
• TENSORFLOW: https://www.tensorflow.org/performance/benchmarks

Unclebench provides HP Zbook15 laptop as a platform example, you can use it as a model to write a jube file which better fit your platform. For a description of platform variables see

Getting started

Fetch a benchmark:

$ubench fetch -b nas

Run a benchmark:

$ubench run -b nas -p cluster  -w 2 4 8 16

Print benchmark result:

    $ubench result  -b nas -p cluster 
    Processing nas benchmark :
    ----analysing results
    benchmark_resulst_path: /scratch/cr3db69n/Ubench/benchmarks/porthos/nas/./benchmarks_runs/000002/
    ----extracting analysis

    processes  Mflops    mpi_version
    2          1358.23   OpenMPI-2.0.1
    4          2326.29   OpenMPI-2.0.1
    8          5173.65   OpenMPI-2.0.1
    16         9176.02   OpenMPI-2.0.1

Summary of UncleBench sub commands:

• fetch: download benchmark source codes, binaries and input files.
• run: run benchmark with parameter modification on the fly.
• list: list runs for given benchmark and platform.
• log/status: follow benchmark progress.
• result: print raw results.
• report: generate html annotated performance report.

For a complete documentation see.

For a description of provided benchmarks and their parameters see

For a description of UncleBench architecture see

Setting up a development environment

We use virtualenv to setup a virtual environment for test:

sudo apt-get install python-virtualenv
mkdir unclebench_env/
virtualenv unclebench_env/

To activate the virtual environment:

cd unclebench_env
source bin/activate

We install unclebench dependencies:

pip install -r requirements.txt
pip install pytest pytest_mock mock

To execute examples using platform Zbook15 we should install JUBE version 2.1.0 before, download JUBE:

tar -xf JUBE-2.1.0.tar.gz
cd JUBE-2.1.0/
python setup.py install

To run benchmarks locally:

export UBENCH_DIR_CREATE=true
source dev_env.sh

The variables 'UBENCH_RUN_DIR_BENCH' and 'UBENCH_RESOURCE_DIR' will be created automatically and they will point to directories in '/tmp/'. To execute unitary tests:

export PYTHONPATH=.
pytest

Some tests could take some minutes to see tests output execute:

pytest -s

Licence

UncleBench is distributed under the terms of the GPL v3 licence.