Skip to content

Understanding PBench Output

Jump to bottom
Steve Burnett edited this page May 14, 2024 · 6 revisions

Pbench creates a set of files for each benchmark run. This page describes where to find the PBench output files, what kinds of files are created by PBench, and how PBench defines the file names.

PBench Output File Location

A benchmark run creates files in an output directory and a subdirectory called a run directory. For the files created by a benchmark run, see PBench Output Files.

Output Directory

Output files are saved to the output directory. Use pbench run -o to set an output directory.

If the directory specified by -o does not exist, it is created.

If -o is not used, the default output directory is the current directory.

Each benchmark run saves the files for that run in a run directory in the output directory.

Run Directory

Each run creates a run directory in the output directory. Use pbench run -n to set a a run directory name.

The run directory name is either:

  • <main stage name>_<current time> (default)
  • the name assigned with -n

The <main stage name> is the names of the stage files separated by underscores.

Main Stage Name Examples

  • ./pbench run -s http://localhost:8080 pbench/benchmarks/tpc-ds/sf1.json pbench/benchmarks/tpc-ds/ds_rand5.json results in:

    • main stage name sf1_ds_rand5
    • run directory sf1_ds_rand5_<current time>

  • ./pbench/pbench -s http://localhost:8080 pbench/benchmarks/tpc-ds/ds_rand5.json pbench/benchmarks/tpc-ds/sf1.json results in:

    • main stage name ds_rand5_sf1
    • run directory of ds_rand5_sf1_<current time>

Output and Run Directory Examples

  • pbench run saves the output to <current directory>/<main stage name>-<current time>, because of the defaults for -o and -n.
  • pbench run -o pbench_results saves the output to the directory pbench_results/<main stage name>-<current time>.
  • pbench run -n today saves the output to <current directory>/today.
  • pbench run -o pbench_results -n today saves the output to pbench_results/today.

PBench Output Files

All PBench output files are saved into the run directory in the output directory.

Log file

Log files follow the PBench Output File Name Format, followed by an underscore, the run timestamp, and the suffix .log.

Example: ds_power_sf1_240509-130537.log

Output JSON File

Output files follow the PBench Output File Name Format, with the suffix .json.

Example: sf1_query_01.json

Error JSON File

Error files follow the PBench Output File Name Format, with the suffix .error.json.

Example: sf1_query_02.error.json

Column Metadata File

Column metadata files follow the PBench Output File Name Format, with the suffix .cols.json.

Example: sf1_query_01.cols.json

Summary File

A summary file is in CSV format. A summary file's name is formed using the <main stage name> with the suffix _summary.csv.

Example: sf1_summary.csv

PBench Output File Name Format

The query result file name of a file generated by "save_output": true is created using the following format:

[query_file]_[query_index]_[c|w][sequence_number]

  • [query_file] - One of the two following, depending on if the query is in queries or query_files:

    • queries in the queries array are displayed inline
    • queries in a query file display the file name without extension

  • [query_index] - The query's position in the queries array, or the position in the file named in query_files.

    The first query is assigned a query_index of 0.

    NOTE: [query_index] is omitted if the queries array, or the file named in query_files, only has one query.

  • [c|w] - Identifies the run as a cold run or a warm run.

    • c - cold
    • w - warm

  • [sequence_number] - The sequence number starts at 0. For example, if "cold_runs": 2 and "warm_runs": 3, then:

    • the sequence is 0-4
    • 0, 1 are the cold runs
    • 2, 3, 4 are the warm runs
Clone this wiki locally