The Sensu assets packaged from this repository are built against the Sensu Ruby runtime environment. When using these assets as part of a Sensu Go resource (check, mutator or handler), make sure you include the corresponding Sensu Ruby runtime asset in the list of assets needed by the resource. The current ruby-runtime assets can be found here in the Bonsai Asset Index.
- bin/handler-pagerduty.rb
- bin/mutator-pagerduty-priority-override.rb
PagerDuty supports dedup. Dedup is useful when you want to create a single alert (for a group of checks). Only one alert is sent out even if the checks fail at the same time. The following example groups check_service_n together for a single host. dedup_rules take in regular expressions as keys and re-write rules as values. dedup_rules entry is optional.
{
"pagerduty": {
"api_key": "12345",
"team_name1": {
"api_key": "23456"
},
"team_name2": {
"api_key": "34567"
},
"dedup_rules": {
"(.*)/check_service_(\\d+)": "\\1/check_service"
}
}
}
In the Client hash you can define a pager_team key value pair. If the the client hash contains the pager_team key it will then no longer use the default pagerduty.api_key from the above hash but will look for the value given in the client. If the value given in the client is not found in the handler specified, then the default pagerduty.api_key will be used. The following client hash will alert using the team_name1 api key instead of the default api_key. This will allow different teams/hosts to alert different escalation paths.
{
"client": {
"name": "my.host.fqdn,
"address": "8.8.8.8",
"subscriptions": [
"production",
"webserver",
"mysql"
],
"pager_team": "team_name1"
}
}
Please note, this sets the escalation path on the whole host. If you want more granular control on escalation paths please view the Mutator section below.
In the Check hash you can define a pagerduty_contexts list to send contextual links and images with your events. This list should conform to the PagerDuty documentation about contexts.
Another approach would be define this on a per check basis like so:
{
"checks": {
"cpu": {
"command": "check-cpu.rb -w 90 -c 100",
"subscribers": [
"base"
],
"handlers": [
"pagerduty"
],
"interval": 15,
"pager_team": "team1",
"notification": "CPU is running hot!",
"occurrences": 8
}
}
}
This mutator allows you to have fine grain control on PagerDuty escalation paths based on data within the client hash. The mutator will look in the following locations where #{event_level} is warning and critical (unknown, is replaced by critical), and #{check_name} is the name of the check. Items located lower in the list take precedence:
client['pager_team']
client['pd_override']['baseline'][#{event_level}]
client['pd_override'][#{check_name}][#{event_level}]
For example if I have the following pager_duty escalations defined on my Sensu server:
{
"pagerduty": {
"api_key": "12345",
"low_priority": {
"api_key": "23456"
},
"ops": {
"api_key: "7890"
}
}
}
And I also have the following client hash:
{
"client": {
"name": "my.host.fqdn",
"address": "8.8.8.8",
"subscriptions": [
"production",
"webserver",
"mysql"
],
"pd_override": {
"baseline" : {
"warning": "low_priority"
},
"check_disk": {
"warning": "ops",
"critical": "ops"
}
}
}
}
{
"pagerduty": {
"api_key": "12345",
"proxy_host": "my.proxy.fqdn",
"proxy_port": "8080",
"proxy_username": "",
"proxy_password": ""
}
}
If a critical event is triggered from "my.host.fqdn" that is not named check_disk it will alert the default (with value api_key: 12345). If a warning event is triggered that is not check_disk it will alert the low_proirity escalation service. If any check_disk alert is triggerd it will the alert the ops escalation.
You can add a custom field from the Sensu client config as a description prefix, like the host name, to add more context to the event description:
{
"pagerduty": {
"api_key": "12345",
"dynamic_description_prefix_key" : "name",
}
}
By default handlers do not handle flapping incidents: handler configuration documentation in order to change this you must set handle_flapping in your handler config like this:
{
"pagerduty": {
"api_key": "12345",
"handle_flapping": true
}
}