feat: adds alerts for thin pool usage #151
Conversation
openshift-ci
bot
added
the
do-not-merge/work-in-progress
|
Skipping CI for Draft Pull Request. |
openshift-ci
bot
removed
the
do-not-merge/work-in-progress
| - "alert": "ThinPoolMetaDataUsageAtThresholdCritical" | ||
| "annotations": | ||
| "description": "Thin pool metadata ultilization in the VolumeGroup is critically full. Data deletion or thin pool expansion is required." | ||
| "message": "Thin Pool metadata utilization in the VolumeGroup {{ $labels.device_class }} has crossed 8500 % on node {{ $labels.node }}. Free up some space or expand the thin pool immediately." |
There was a problem hiding this comment.
| "message": "Thin Pool metadata utilization in the VolumeGroup {{ $labels.device_class }} has crossed 8500 % on node {{ $labels.node }}. Free up some space or expand the thin pool immediately." | |
| "message": "Thin Pool metadata utilization in the VolumeGroup {{ $labels.device_class }} has crossed 85 % on node {{ $labels.node }}. Free up some space or expand the thin pool immediately." |
There was a problem hiding this comment.
fixed. Also marking other similar comments are resolved.
|
@aruniiird: changing LGTM is restricted to collaborators In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Adds alerts for thin pool data and metadata usage. Provides near full (75%) and critical (85%) usage alerts. Signed-off-by: Santosh Pillai <sapillai@redhat.com>
|
/retest |
|
@sp98: The following test failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
| - "alert": "ThinPoolDataUsageAtThresholdCritical" | ||
| "annotations": | ||
| "description": "Thin pool in the VolumeGroup is critically full. Data deletion or thin pool expansion is required." | ||
| "message": "Thin Pool data utilization in the VolumeGroup {{ $labels.device_class }} has crossed 85 % on node {{ $labels.node }}. Free up some space or expand the thin pool immediately." |
There was a problem hiding this comment.
Was this generated by the jsonnet? I don't see any changes to the libsonnet file.
There was a problem hiding this comment.
yes. These changes are in the monitoring/alerts/vgalerts.libsonnet file.
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: nbalacha, sp98 The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
bot
added
the
approved
Adds alerts for thin pool data and metadata usage.
Provides near full (75%) and critical (85%) usage alerts.
Note: We don't have thin pool name in the alerts since topolvm does not provide it as of now. But since we support only one thin pool per volume group, current alerts should suffice (IMO).
We can update it to include thin pool names once topolvm shows thin pool name in the metric labels.
Tested with lower thresholds.
Signed-off-by: Santosh Pillai sapillai@redhat.com