-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Monitoring][Alerting] Large shard alert #89410
Conversation
Pinging @elastic/stack-monitoring (Team:Monitoring) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is looking awesome! Great work here @igoristic!
I noted a few issues in comments!
const { | ||
_index: monitoringIndexName, | ||
_source: { source_node: sourceNode, index_stats: indexStats }, | ||
} = get(indexBucket, 'hits.hits.hits[0]'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of using get
, can we type this to the newly added interfaces for ES source fields? See https://github.com/elastic/kibana/blob/master/x-pack/plugins/monitoring/common/types/es.ts
shardIndex, | ||
shardSize, | ||
clusterUuid, | ||
nodeName, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
indexPattern: [], | ||
}; | ||
if (!inputValues.indexPattern) { | ||
errors.indexPattern.push( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
item: AlertData | null, | ||
cluster: AlertCluster | ||
) { | ||
const shardIndices = alertStates |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Igor, this looks great. |
I'm not filtering out any "internal" type indices (although it would be pretty easy to add), however, I think this should be a post enhancement. The UI could be a checkbox, kinda what we have in SM indices list page. I'm also not "strictly" validating the index name/patterns, although, I am escaping invalid characters based on the requirements noted here in the docs These are great points and I was aware of them, but didn't really put much thought into it since they weren't part of the acceptance criteria (at least not for this first iteration) |
I feel that we should have some solution for the internal indices in our delivery. With our current solution can we just change the default value in index pattern to exclude system indices? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Great work here!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
80k is quite a jump, but it's still under the implicit goal of 100k so I don't see any reason to object. If y'all would rather rely on the per-PR metrics to identify unexpectedly large PRs I think that's reasonable.
💚 Build SucceededMetrics [docs]Module Count
Async chunks
Page load bundle
History
To update your PR or re-run it, just comment with: |
* Shards alert draft * Added index pattern validation * Fixed ui/ux * Optimizing the response * CR feedback * import fix * Increased size limit # Conflicts: # packages/kbn-optimizer/limits.yml # x-pack/plugins/monitoring/public/components/elasticsearch/index/advanced.js
Backport: |
Resolves: #74820
This alert checks for large (primary) shards of the specified indices. It will only trigger if the shard is over the 55gb threshold. By default it will check all index patterns
*
, but you can use Elastic's "Multi-target syntax" pattern/standards to get a desired set of inclusion and negation.To test it simply set the threshold value (in gigabytes) to something very low (decimal values are also possible)
UI/UX: