Skip to content

2.3 - Test Series and Better Results
Compare
Choose a tag to compare
@Paul-Ferrell Paul-Ferrell released this 20 Jan 23:16
v2.3
4f8a1d8
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

2.3 Release Notes

  • The new 'pav series' command and associated files (stored in <config>/series/<name>.yaml)
    allow for the organization of groups of tests in a hierarchical manner with multiple testing steps.
    See https://pavilion2.readthedocs.io/en/v2.3/tutorials/writing_a_series.html
  • Revamped the result parsing system for greater flexibility. Backwards compatibility is mostly maintained (see below).
    The updated system allows for line selection regexes for every parser, so the parser can concentrate on parsing
    the line/lines themselves. See https://pavilion2.readthedocs.io/en/v2.3/results/basics.html
  • Added 'flatten_results' option to the base Pavilion config. Allows for
    producing a separate result log line for each 'per_file' result, making
    charting in Splunk significantly easier. The results can be significantly
    larger, however, especially on large machines.
  • Added unified test run filtering in 'status' and 'result'.
  • Added the 'list' command for searching test runs and series
  • Added 'maint' command for maintanence operations.
  • Results now include a 'uuid' key - A completely unique id for each test.
  • Added spack options to test config. Allows for spack packages to be built
    and/or loaded inside of a pavilion test. Requires a valid spack path to
    be provided in the Pavilion config under the new key spack_path.
  • Modules may now have nested paths (True lmod module organization)
  • Updated permissions management.
  • Added test series files for organizing multiple test runs.
  • Added 'pav graph' command for creating graphs from results on the fly. Requires matplotlib.
  • Added 'pav show hosts/modes/suites' commands.
  • Switched to using epoch time for all timestamps.
  • Test run attributes are now indexed for faster 'pav status' and 'pav result' searches.

See https://pavilion2.readthedocs.io/en/v2.3/ for documentation on many of these changes.

Backwards compatibility breaks

  • 'fn' and 'n' result fields have been merged into 'per_file'.
  • The 'status', and 'result' commands now search all available test runs
    and select recent (1 day) tests by the current user on the current
    machine. For the old behavior, use pav status last, etc.
  • Old tests will lack some expected properties for searching,
    as that information has been consolidated into the 'attributes' file
    which they won't have.
  • 'test_node_list' in the results has been replaced with the much shorter
    'test_node_list_short' key.
  • The results command '-l' (--show-logs) option is now '-L'.
  • The results command '--json' argument and '--oneline' argments were removed.
    Use --summary instead.
  • The table result parser has entirely changed. See pav show results --doc table.
  • match_type for the regex result parser is now general to all parsers, and called match_select.

Roadmap

2.4 will primarily focus on revamping and adding scheduler plugins. Expect additional options for scheduling tests, performing node sampling, consistent scaling studies, and more.

Assets 2
Loading