Added functionality to allow for JSON file comparison output

[nathan-weinberg]

Type of change

• Refactor
• New feature
• Bug fix
• Optimization
• Documentation Update

Description

This PR extends the compare.sh script functions run_benchmark_comparison and compare to allow for generation of JSON files - previously only CSVs were supported

Related Tickets & Documents

• Related Issue: NETOBSERV-1095

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.

Testing

• Please describe the System Under Test.

Tested locally using the OCP QE Elasticsearch instance and data generated with the Network Observability Prometheus and Elasticsearch (NOPE) tool

• Please provide detailed steps to perform tests related to this code change.

Set appropriate environmental variables and run:

$ source compare.sh
$ run_benchmark_comparison

• How were the fix/results from this change verified? Please provide relevant screenshots or results.

Test performed with the following env vars:

export GSHEET_KEY_LOCATION=/home/nathan/ocp/ci/gsheet_config.json
export CONFIG_LOC=/home/nathan/ocp/ci/ocp-qe-perfscale-ci/scripts/queries
export COMPARISON_CONFIG=netobserv_touchstone_statistics_config.json
export ES_SERVER=https://$ES_USERNAME:$ES_PASSWORD@search-ocp-qe-perf-scale-test-elk-hcm7wtsqpxy7xogbu72bor4uve.us-east-1.es.amazonaws.com
export EMAIL_ID_FOR_RESULTS_SHEET=myemail@redhat.com
export GEN_CSV=false
export GEN_JSON=true

export NETWORK_TYPE=OVNKubernetes
export WORKLOAD=node-density-heavy
export UUID=da822ab1-3492-4dca-9f53-99f98018edc2

Console output (errors can be ignored, unrelated to this PR):

[nathan@nathan-redhat utils (jsonerator)]$ run_benchmark_comparison 
Mon Jun 26 07:35:23 PM UTC 2023 benchmark
Mon Jun 26 07:35:23 PM UTC 2023 Installing touchstone
openshift-ovn-kubernetes
final json /tmp/node-density-heavy-da822ab1-3492-4dca-9f53-99f98018edc2/da822ab1-3492-4dca-9f53-99f98018edc2.json
config /home/nathan/ocp/ci/ocp-qe-perfscale-ci/scripts/queries/netobserv_touchstone_statistics_config.json
comparison output
Mon Jun 26 07:35:31 PM UTC 2023 Querying results
Mon Jun 26 07:35:31 PM UTC 2023 Running: touchstone_compare --database elasticsearch -url https://ocp-qe:Perfscale22!@search-ocp-qe-perf-scale-test-elk-hcm7wtsqpxy7xogbu72bor4uve.us-east-1.es.amazonaws.com -u da822ab1-3492-4dca-9f53-99f98018edc2 --config /tmp/node-density-heavy-da822ab1-3492-4dca-9f53-99f98018edc2/netobserv_touchstone_statistics_config.json -o json --output-file /home/nathan/ocp/ci/e2e-benchmarking/utils/node-density-heavy-da822ab1-3492-4dca-9f53-99f98018edc2.json
2023-06-26 15:35:32,660 - touchstone - ERROR - Error: Issue capturing results from elasticsearch using config {'filter': {'metric_name.keyword': 'nFlowsErroredPerMinuteTotals'}, 'buckets': ['metric_name.keyword'], 'aggregations': {'value': [{'percentiles': {'percents': [90]}}, 'avg']}}
2023-06-26 15:35:32,846 - touchstone - ERROR - Error: Issue capturing results from elasticsearch using config {'filter': {'metric_name.keyword': 'nFlowsErroredPerMinute'}, 'buckets': ['metadata.pod.keyword'], 'aggregations': {'value': [{'percentiles': {'percents': [90]}}, 'avg']}}
Mon Jun 26 07:35:34 PM UTC 2023 compare result: 0
Mon Jun 26 07:35:34 PM UTC 2023 copying over working JSON to final JSON
Mon Jun 26 07:35:34 PM UTC 2023 Removing touchstone

JSON file node-density-heavy-da822ab1-3492-4dca-9f53-99f98018edc2.json was generated successfully in the same directory as the script

[nathan@nathan-redhat utils (jsonerator)]$ ls
benchmark-operator.sh  csv_gen.py                                                    scale-ci-diagnosis
common.sh              csv_modifier.py                                               snappy-move-results
compare.sh             node-density-heavy-da822ab1-3492-4dca-9f53-99f98018edc2.json  touchstone-compare

[paigerube14]

/lgtm

Passed and same sheet as expected https://mastern-jenkins-csb-openshift-qe.apps.ocp-c1.prod.psi.redhat.com/job/scale-ci/job/paige-e2e-multibranch/job/kube-burner/223/console

[jtaleric]

lgtm

[rsevilla87]

lgtm!