Json output #21

shashank-boyapally · 2024-02-20T14:10:49Z

Type of change

Description

Some of the changes are from metadata change PR please ignore them. We output the change point detection in a tabular form which was inherently provided by hunter. Now we provide it as a json output in stdout with the use of the flag --output or -o, this flag defaults to text and can be set to json. Below is a concise version of the json output. This does not change the csv file output. This PR aims to just provide the logic for json output, this will be separated in daemon mode.

[{
        "uuid": "5b06c830-be55-4c84-b4da-92720b55c304",
        "timestamp": 1701866097,
        "metrics": {
            "etcdDisk_duration_avg": {
                "value": 0.0134358362,
                "percentage_change": 0
            },
            "P99": {
                "value": 13000,
                "percentage_change": 0
            },
            "etcdCPU_cpu_avg": {
                "value": 8.6963812859,
                "percentage_change": 0
            },
            "ovnCPU_cpu_avg": {
                "value": 2.8599566281,
                "percentage_change": 0
            },
            "apiserverCPU_cpu_avg": {
                "value": 11.0500421666,
                "percentage_change": 0
            }
        },
        "is_changepoint": false
    },
    {
        "uuid": "c5f95336-27d4-4615-94c1-d618e975a36b",
        "timestamp": 1703075837,
        "metrics": {
            "etcdDisk_duration_avg": {
                "value": 0.0137848279,
                "percentage_change": 42.39053258714572
            },
            "P99": {
                "value": 13000,
                "percentage_change": 0
            },
            "etcdCPU_cpu_avg": {
                "value": 8.2014834841,
                "percentage_change": 0
            },
            "ovnCPU_cpu_avg": {
                "value": 2.7795422015,
                "percentage_change": 0
            },
            "apiserverCPU_cpu_avg": {
                "value": 10.9252901816,
                "percentage_change": 0
            }
        },
        "is_changepoint": true
    },]

Related Tickets & Documents

Related Issue output as json #19
Closes #

Checklist before requesting a review

I have performed a self-review of my code.
If it is a core feature, I have added thorough tests.

Testing

Please describe the System Under Test.
Please provide detailed steps to perform tests related to this code change.
How were the fix/results from this change verified? Please provide relevant screenshots or results.

jtaleric · 2024-03-05T15:14:50Z

Can we add some additional data to the JSON like the build, platform, cni, etc?

shashank-boyapally · 2024-03-05T15:16:15Z

Thank you for the suggestion @jtaleric, will be adding and pushing the changes.

paigerube14 · 2024-03-06T18:12:39Z

orion.py

@@ -11,8 +12,13 @@
 import pandas as pd

 from fmatch.matcher import Matcher
-from utils.orion_funcs import run_hunter_analyze, get_metadata, \
-                                set_logging, load_config, get_metric_data
+from utils.orion_funcs import (


thoughts on just importing the whole file, so you dont have to specify every function as we continue to add

Sounds good, if we are using almost all the functions in that file.

vishnuchalla · 2024-03-25T01:06:38Z

orion.py

@@ -11,8 +12,13 @@
 import pandas as pd

 from fmatch.matcher import Matcher
-from utils.orion_funcs import run_hunter_analyze, get_metadata, \
-                                set_logging, load_config, get_metric_data
+from utils.orion_funcs import (


Sounds good, if we are using almost all the functions in that file.

vishnuchalla · 2024-03-25T01:09:44Z

utils/orion_funcs.py

+        output_table = report.produce_report(
+            test_name="test", report_type=ReportType.LOG
+        )
+        print(output_table)


can we use a logger here?

vishnuchalla · 2024-03-25T02:09:46Z

utils/orion_funcs.py

+        )
+        print(output_table)
+    elif output == "json":
+        change_points_by_metric = series.analyze().change_points


I think we don't need this custom logic at all. We can just use the same Report class to get change points reported in json using inbuilt functionality.

output = report.produce_report(test_name="test",report_type=ReportType.JSON) print(output) {"test": [{"time": 1703075837, "changes": [{"metric": "etcdDisk_duration_avg", "forward_change_percent": "43"}]}, {"time": 1704890296, "changes": [{"metric": "ovnCPU_cpu_avg", "forward_change_percent": "26"}, {"metric": "etcdCPU_cpu_avg", "forward_change_percent": "13"}]}, {"time": 1704994226, "changes": [{"metric": "apiserverCPU_cpu_avg", "forward_change_percent": "20"}]}]}

This will give the exact timestamps of all the change points along with metric and their percentage change attributes. If we want the value we can just get it from the merged dataframe by doing some thing like below.

index = df[df['timestamp'] == given_timestamp].index.item() value = df.loc[index, 'value']

Also instead of entire data dump, having only change points in the JSON output would be cleaner. just my .02 cents.

Thank you for the comment @vishnuchalla. I believe the main reason we find to have json is reusability of data and logic in other tools. If we have change points it might just be constrained to changepoint detection, if we have the whole data, the same can be used in cpt dashboards such that we can view the changepoints in graphs too. I think this is the major reason we went ahead with implementing a custom logic. Also custom logic allows us to add the buildUrl and else if we needed in the json document. Thoughts?

Understood. So can we have both regulated with a toogle between just changepoints and the entire list of docs with changepoints? Because getting the entire data might be very huge sometimes. Also can we have a follow-up story to only detect regressions within a given time range? To have something like this.

I'll have this toggle and time range implemented along with the next PR, as already the current PR #26 has too many tasks going on.

vishnuchalla · 2024-03-25T02:10:07Z

utils/orion_funcs.py

+        print(json.dumps(output_json, indent=4))
+
+
+def parse_json_output(merged_df, change_points_by_metric):


This can be removed based on the logic suggested in the above comment

vishnuchalla

.

Signed-off-by: Shashank Reddy Boyapally <sboyapal@redhat.com>

jtaleric

Should we suppress the log output when users pass --output json ? Usually they would run jq with the output, which might get messed up w/ additional output?

jtaleric

happy to see this merged and we add the option to suppress output.

vishnuchalla

lgtm

shashank-boyapally requested review from jtaleric and paigerube14 February 20, 2024 14:38

shashank-boyapally force-pushed the json-output branch from 3de05ff to 061b076 Compare February 22, 2024 18:06

paigerube14 reviewed Mar 6, 2024

View reviewed changes

vishnuchalla reviewed Mar 25, 2024

View reviewed changes

vishnuchalla mentioned this pull request Mar 25, 2024

display build URL for changepoint #15

Merged

7 tasks

vishnuchalla requested changes Mar 25, 2024

View reviewed changes

vishnuchalla mentioned this pull request Mar 25, 2024

Support for Daemon mode for orion #26

Merged

7 tasks

shashank-boyapally force-pushed the json-output branch from 061b076 to 354e0ca Compare May 20, 2024 21:12

shashank-boyapally added 6 commits May 21, 2024 09:19

added output flag

4a83a3b

Signed-off-by: Shashank Reddy Boyapally <sboyapal@redhat.com>

json output

26b9ee0

Signed-off-by: Shashank Reddy Boyapally <sboyapal@redhat.com>

pylint error fixed and formatted

3952133

Signed-off-by: Shashank Reddy Boyapally <sboyapal@redhat.com>

removed prints

b9d8559

Signed-off-by: Shashank Reddy Boyapally <sboyapal@redhat.com>

rebased, handled duplicates, changed table column names

6b3c58b

Signed-off-by: Shashank Reddy Boyapally <sboyapal@redhat.com>

rebased

ca86ceb

Signed-off-by: Shashank Reddy Boyapally <sboyapal@redhat.com>

shashank-boyapally force-pushed the json-output branch from 354e0ca to ca86ceb Compare May 21, 2024 13:45

fixed pylint errors

b73c893

Signed-off-by: Shashank Reddy Boyapally <sboyapal@redhat.com>

jtaleric reviewed May 21, 2024

View reviewed changes

jtaleric approved these changes May 21, 2024

View reviewed changes

vishnuchalla approved these changes May 21, 2024

View reviewed changes

jtaleric merged commit daad3af into cloud-bulldozer:main May 22, 2024
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Json output #21

Json output #21

shashank-boyapally commented Feb 20, 2024 •

edited

Loading

jtaleric commented Mar 5, 2024

shashank-boyapally commented Mar 5, 2024

paigerube14 Mar 6, 2024

vishnuchalla Mar 25, 2024

vishnuchalla Mar 25, 2024

vishnuchalla Mar 25, 2024

vishnuchalla Mar 25, 2024 •

edited

Loading

shashank-boyapally Mar 25, 2024

vishnuchalla Mar 25, 2024 •

edited

Loading

shashank-boyapally Mar 25, 2024

vishnuchalla Mar 25, 2024

vishnuchalla left a comment

jtaleric left a comment

jtaleric left a comment

vishnuchalla left a comment

		print(json.dumps(output_json, indent=4))


		def parse_json_output(merged_df, change_points_by_metric):

Json output #21

Json output #21

Conversation

shashank-boyapally commented Feb 20, 2024 • edited Loading

Type of change

Description

Related Tickets & Documents

Checklist before requesting a review

Testing

jtaleric commented Mar 5, 2024

shashank-boyapally commented Mar 5, 2024

paigerube14 Mar 6, 2024

Choose a reason for hiding this comment

vishnuchalla Mar 25, 2024

Choose a reason for hiding this comment

vishnuchalla Mar 25, 2024

Choose a reason for hiding this comment

vishnuchalla Mar 25, 2024

Choose a reason for hiding this comment

vishnuchalla Mar 25, 2024 • edited Loading

Choose a reason for hiding this comment

shashank-boyapally Mar 25, 2024

Choose a reason for hiding this comment

vishnuchalla Mar 25, 2024 • edited Loading

Choose a reason for hiding this comment

shashank-boyapally Mar 25, 2024

Choose a reason for hiding this comment

vishnuchalla Mar 25, 2024

Choose a reason for hiding this comment

vishnuchalla left a comment

Choose a reason for hiding this comment

jtaleric left a comment

Choose a reason for hiding this comment

jtaleric left a comment

Choose a reason for hiding this comment

vishnuchalla left a comment

Choose a reason for hiding this comment

shashank-boyapally commented Feb 20, 2024 •

edited

Loading

vishnuchalla Mar 25, 2024 •

edited

Loading

vishnuchalla Mar 25, 2024 •

edited

Loading