[8.11] [ML] AIOps: Functional/API integration tests for text field support for log rate analysis (#168177) #168516
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…or log rate analysis (elastic#168177) This updates the artificial dataset generator for log rate analysis to allow to create variants including text fields. The artificial dataset is now used for 4 variants of functional and API integration tests: Testing spike and dip with both with and without a text field. The new tests surfaced some issues that were fixed as part of this PR: - Getting the counts of log patterns in combination with individual significant terms ended up with to granular groups. This PR adds additional queries to get counts for log patterns in combination with item sets already derived from significant terms. - The `support` value is returned by the frequent item sets agg and is used as a threshold whether to include an item set for grouping. This was missing from significant log patterns and is fixed by this PR. - Adds a check to not get frequent item sets for log patterns if there are no significant terms. - The way we fetched log patterns using a time filter that spans the whole of the baseline start to the deviation end caused problems with analysing dips. This PR updates those queries to only fetch the actual baseline and deviation time range. - The integration tests caught an issue where we'd still fetch the histogram for log patterns even if we'd request grouping information only. (cherry picked from commit 9259f48)
Merged
2 tasks
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
💚 Build Succeeded
Metrics [docs]
To update your PR or re-run it, just comment with: cc @walterra |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport
This will backport the following commits from
mainto8.11:Questions ?
Please refer to the Backport tool documentation