[Monitoring]Add Filter in stack monitoring rules

[ravikesarwani]

Add “Filter” as an optional field in all the stack monitoring rules.
This can be similar to what we have in Metric threshold rule.

This will allows users more control in terms of the scope of alerts.
Users should be able to filter (include or exclude) based on things like cluster id, node id etc. fields.

This combined with allowing users to create additional rules in Rules and connectors will help users to setup different rules for different workloads when monitoring many production clusters to the same monitoring cluster (a typical scenario).

Also this can help with scenarios where a user wants to exclude certain cluster or nodes from a specific alert.
Example1: They do not want alerts to be generated for a Dev or QA cluster.
Example2: They want to exclude certain nodes for a specific alert (do not want disk alert on frozen tier nodes)

[elasticmachine]

Pinging @elastic/stack-monitoring (Team:Monitoring)

[elasticmachine]

Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui)

[ravikesarwani]

With 91145 in good shape for 7.15 (thanks team!) I was wondering if we maybe able to look at this one in the near future to see what it would take to implement?

[simianhacker]

@ravikesarwani For the KQL auto complete fields, do we want to just get the fields from the SM index patterns? OR do we want to use a curated list of fields?

[ravikesarwani]

@simianhacker I think keeping it versatile but simple for the most common use cases maybe the best option here.
In that perspective I was thinking we get the fields from the SM index pattern (versatile) but also provide some way for the user to quickly be able to add include/exclude filter for cluster and node (at least that’s the most common use cases I have seen asks for).
Maybe we provide some quick examples in the text form for key use cases like:

• Include/Exclude one or more cluster from the rule
• Include/Exclude one or more node from the rule