Adding feature direction rules

[amitgalitz]

Description

This PR expands on the custom supression rules that we have added in 2.17. Users now have the ability to configures rules which have x number of conditions in them regarding the feature.

Rules can look like this:

rules": [
            {
              "action": "IGNORE_ANOMALY",
              "conditions": [
                {
                  "feature_name": "logVolume",
                  "threshold_type": "
ACTUAL_OVER_EXPECTED_RATIO",
                  "value": 0.3,
                  "operator": LTE
                }
              ]
            }
    ]

This rule means that we will ignore anomalies if the actual value is less than or equal too 30% higher than the expected.
For example: "Ignore anomalies for feature logVolume when the actual value is no more than 30% above the expected value."

These first sets of threshold are based on either ration or value margin between the actual and expected value compared to a given value by the user. These rules are given to the RCF model and RCF code takes care of applying these.

In this PR we are adding two more generic rules that are simply checking if the actual value is below or above the expected value. These rules wont need a value or operator as they are more absolute comparison. Currently RCF doesn’t have the logic to handles such rules so we decided to go with the approach of overriding the anomaly grade in the Anomaly Result. This takes care of both historical, real time and preview results.

Newly accepted payloads:

rules": [
            {
              "action": "IGNORE_ANOMALY",
              "conditions": [
                {
                  "feature_name": "sum_http_4xx",
                  "threshold_type": "ACTUAL_IS_OVER_EXPECTED",
                  "value": null,
                  "operator": null
                }
              ]
            }
    ]

or

"conditions": [
                {
                  "feature_name": "sum_http_4xx",
                  "threshold_type": "ACTUAL_IS_OVER_EXPECTED"
                }
              ]

Edge cases:

• I first consider past values for comparison with expected value if they are present
• We only care about the feature with the highest attribution when looking at the rules, if they are equal then we look at both.

Related Issues

#616

Check List

• New functionality includes testing.
• New functionality has been documented.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

Attention: Patch coverage is 74.76636% with 27 lines in your changes missing coverage. Please review.

Project coverage is 80.04%. Comparing base (4c545ab) to head (6089c28).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
...in/java/org/opensearch/ad/model/AnomalyResult.java	72.22%	7 Missing and 8 partials ⚠️
...a/org/opensearch/ad/ml/IgnoreSimilarExtractor.java	33.33%	1 Missing and 3 partials ⚠️
...c/main/java/org/opensearch/ad/model/Condition.java	73.33%	0 Missing and 4 partials ⚠️
.../java/org/opensearch/ad/ml/ThresholdingResult.java	75.00%	0 Missing and 1 partial ⚠️
.../java/org/opensearch/ad/model/AnomalyDetector.java	80.00%	0 Missing and 1 partial ⚠️
...opensearch/ad/transport/AnomalyResultResponse.java	80.00%	0 Missing and 1 partial ⚠️
...rch/forecast/transport/ForecastResultResponse.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #1358      +/-   ##
============================================
- Coverage     80.04%   80.04%   -0.01%     
- Complexity     5667     5699      +32     
============================================
  Files           533      533              
  Lines         23438    23527      +89     
  Branches       2334     2367      +33     
============================================
+ Hits          18762    18833      +71     
- Misses         3569     3570       +1     
- Partials       1107     1124      +17     

Flag	Coverage Δ
plugin	80.04% <74.76%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...in/java/org/opensearch/ad/model/ThresholdType.java	90.90% <100.00%> (+2.02%)	⬆️
...arch/timeseries/ExecuteResultResponseRecorder.java	77.77% <ø> (ø)
.../java/org/opensearch/timeseries/model/Feature.java	96.72% <100.00%> (+0.35%)	⬆️
...a/org/opensearch/timeseries/model/FeatureData.java	100.00% <100.00%> (ø)
...timeseries/rest/handler/IndexJobActionHandler.java	89.27% <100.00%> (ø)
...pensearch/timeseries/transport/ResultResponse.java	100.00% <ø> (ø)
.../java/org/opensearch/ad/ml/ThresholdingResult.java	84.33% <75.00%> (-0.48%)	⬇️
.../java/org/opensearch/ad/model/AnomalyDetector.java	88.85% <80.00%> (-0.14%)	⬇️
...opensearch/ad/transport/AnomalyResultResponse.java	75.93% <80.00%> (+0.15%)	⬆️
...rch/forecast/transport/ForecastResultResponse.java	16.88% <0.00%> (-0.23%)	⬇️
... and 3 more

... and 11 files with indirect coverage changes

[ohltyler]

LGTM!