You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We currently use the Thanos mixing Jsonnet library to define alerts per instance of the component. However, because we don't customize the alert selectors in component/alerts.libsonnet, each instance's alerts will pick up metrics from all instances of the deployed Thanos component (e.g. receive).
This makes it currently impossible to have different sets of alerts for different instances of the same Thanos component in a single cluster. Since we advertise multi-instance support, we should make sure that alert rules are configured to only match their corresponding component instances (e.g. by adding 'namespace=%s'%instanceNs to the alert selector(s).
Steps to Reproduce the Problem
Create two instances of the component which deploy the same Thanos component (e.g. receive)
Disable alert rules for one of the instances
Observe that remaining alert rules will pick up both instances
Actual Behavior
Alert rules pick up all Thanos component instances on the cluster instead of only the ones associated with the Commodore component instance.
Expected Behavior
Alert rules are constrained to only cover Thanos instances associated with the Component instance
The text was updated successfully, but these errors were encountered:
We currently use the Thanos mixing Jsonnet library to define alerts per instance of the component. However, because we don't customize the alert selectors in component/alerts.libsonnet, each instance's alerts will pick up metrics from all instances of the deployed Thanos component (e.g.
receive
).This makes it currently impossible to have different sets of alerts for different instances of the same Thanos component in a single cluster. Since we advertise multi-instance support, we should make sure that alert rules are configured to only match their corresponding component instances (e.g. by adding
'namespace=%s'%instanceNs
to the alert selector(s).Steps to Reproduce the Problem
receive
)Actual Behavior
Alert rules pick up all Thanos component instances on the cluster instead of only the ones associated with the Commodore component instance.
Expected Behavior
Alert rules are constrained to only cover Thanos instances associated with the Component instance
The text was updated successfully, but these errors were encountered: