-
Notifications
You must be signed in to change notification settings - Fork 75
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add delete anomaly results API #131
add delete anomaly results API #131
Conversation
Signed-off-by: Yaliang Wu <ylwu@amazon.com>
ec5e872
to
63822a4
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A high level comment, do we need to consider FGAC with delete api?
Yes, check DeleteAnomalyResultsTransportAction.java line 79
This method adds user backend role filter, so users can only delete anomaly results which they have permission. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
client.execute(DeleteByQueryAction.INSTANCE, request, listener); | ||
} else { | ||
// Security is enabled and backend role filter is enabled | ||
try { | ||
addUserBackendRolesFilter(user, request.getSearchRequest().source()); | ||
client.execute(DeleteByQueryAction.INSTANCE, request, listener); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
client.execute() will be called anyway?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, split the code branch clearly to make it more readable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good to me.
…ld HC detector This change contains PR: opensearch-project#33 opensearch-project#42 opensearch-project#48 opensearch-project#51 opensearch-project#104 opensearch-project#123 opensearch-project#130 opensearch-project#131 opensearch-project#132 Signed-off-by: Yaliang Wu <ylwu@amazon.com>
…eld HC detector This change contains PR: opensearch-project#33 opensearch-project#42 opensearch-project#48 opensearch-project#51 opensearch-project#104 opensearch-project#123 opensearch-project#130 opensearch-project#131 opensearch-project#132
…eld HC detector This change contains PR: opensearch-project#33 opensearch-project#42 opensearch-project#48 opensearch-project#51 opensearch-project#104 opensearch-project#123 opensearch-project#130 opensearch-project#131 opensearch-project#132 Signed-off-by: Yaliang Wu <ylwu@amazon.com>
* tune data model for unified flow Signed-off-by: Yaliang Wu <ylwu@amazon.com> * support unified flow for single entity detector Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add realtime task Signed-off-by: Yaliang Wu <ylwu@amazon.com> * support preview with detector directly Signed-off-by: Yaliang Wu <ylwu@amazon.com> * support historical analysis for HC detector (#104) * support run historical analysis for HC detector Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * add more javadoc Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * address comments: rename method, remove comments etc Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * add more comments for task action Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * address comments from old CR: mainly renaming and add more comments Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> Co-authored-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * support HC task profiles in profile API;error handling (#123) * support HC task profiles in profile API;error handling Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add init progress of realtime task (#130) * add init progress of realtime task * add realtime task cache * remove DetectionStateHandler, will track error in realtime task * add more comments * add delete anomaly results API (#131) Signed-off-by: Yaliang Wu <ylwu@amazon.com> * maintain running historical task in hourly cron (#132) * maintain running historical task in hourly cron Signed-off-by: Yaliang Wu <ylwu@amazon.com> * maintain realtime tasks * address comments Co-authored-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com>
* tune data model for unified flow Signed-off-by: Yaliang Wu <ylwu@amazon.com> * support unified flow for single entity detector Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add realtime task Signed-off-by: Yaliang Wu <ylwu@amazon.com> * support preview with detector directly Signed-off-by: Yaliang Wu <ylwu@amazon.com> * support historical analysis for HC detector (opensearch-project#104) * support run historical analysis for HC detector Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * add more javadoc Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * address comments: rename method, remove comments etc Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * add more comments for task action Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * address comments from old CR: mainly renaming and add more comments Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> Co-authored-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * support HC task profiles in profile API;error handling (opensearch-project#123) * support HC task profiles in profile API;error handling Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add init progress of realtime task (opensearch-project#130) * add init progress of realtime task * add realtime task cache * remove DetectionStateHandler, will track error in realtime task * add more comments * add delete anomaly results API (opensearch-project#131) Signed-off-by: Yaliang Wu <ylwu@amazon.com> * maintain running historical task in hourly cron (opensearch-project#132) * maintain running historical task in hourly cron Signed-off-by: Yaliang Wu <ylwu@amazon.com> * maintain realtime tasks * address comments Co-authored-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com>
* tune data model for unified flow Signed-off-by: Yaliang Wu <ylwu@amazon.com> * support unified flow for single entity detector Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add realtime task Signed-off-by: Yaliang Wu <ylwu@amazon.com> * support preview with detector directly Signed-off-by: Yaliang Wu <ylwu@amazon.com> * support historical analysis for HC detector (opensearch-project#104) * support run historical analysis for HC detector Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * add more javadoc Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * address comments: rename method, remove comments etc Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * add more comments for task action Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * address comments from old CR: mainly renaming and add more comments Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> Co-authored-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * support HC task profiles in profile API;error handling (opensearch-project#123) * support HC task profiles in profile API;error handling Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add init progress of realtime task (opensearch-project#130) * add init progress of realtime task * add realtime task cache * remove DetectionStateHandler, will track error in realtime task * add more comments * add delete anomaly results API (opensearch-project#131) Signed-off-by: Yaliang Wu <ylwu@amazon.com> * maintain running historical task in hourly cron (opensearch-project#132) * maintain running historical task in hourly cron Signed-off-by: Yaliang Wu <ylwu@amazon.com> * maintain realtime tasks * address comments Co-authored-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com>
* tune data model for unified flow Signed-off-by: Yaliang Wu <ylwu@amazon.com> * support unified flow for single entity detector Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add realtime task Signed-off-by: Yaliang Wu <ylwu@amazon.com> * support preview with detector directly Signed-off-by: Yaliang Wu <ylwu@amazon.com> * support historical analysis for HC detector (#104) * support run historical analysis for HC detector Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * add more javadoc Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * address comments: rename method, remove comments etc Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * add more comments for task action Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * address comments from old CR: mainly renaming and add more comments Signed-off-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> Co-authored-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com> * support HC task profiles in profile API;error handling (#123) * support HC task profiles in profile API;error handling Signed-off-by: Yaliang Wu <ylwu@amazon.com> * add init progress of realtime task (#130) * add init progress of realtime task * add realtime task cache * remove DetectionStateHandler, will track error in realtime task * add more comments * add delete anomaly results API (#131) Signed-off-by: Yaliang Wu <ylwu@amazon.com> * maintain running historical task in hourly cron (#132) * maintain running historical task in hourly cron Signed-off-by: Yaliang Wu <ylwu@amazon.com> * maintain realtime tasks * address comments Co-authored-by: Yaliang Wu <ylwu@dev-dsk-ylwu-2c-e500d1cf.us-west-2.amazon.com>
Description
With historical analysis, user can generate much more anomaly result in a short time, especially for HC detector. AD plugin will roll over anomaly result index if its doc count reaches some limit, and will delete roll-over index after some period (30 days by default). So if user run historical analysis, it may generate too many AD results and speed up the rollover and AD result index cleanup.
This PR adds delete anomaly result API. User can use this API to delete anomaly results with specific query like delete all anomaly results of one detector/task.
Sample request
Issues Resolved
[List any issues this PR will resolve]
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.