Add support to track filtered/unfiltered notifications and do not re-notify if filtered states don't change

[icinga-migration]

This issue has been migrated from Redmine: https://dev.icinga.com/issues/12463

Created by darmagan on 2016-08-17 10:00:53 +00:00

Assignee: (none)
Status: New
Target Version: (none)
Last Update: 2016-08-17 10:00:53 +00:00 (in Redmine)

Icinga Version: 2.4.10
Backport?: Not yet backported
Include in Changelog: 1

---

Hi,

if Services are changing states all of the changes are notified. It's not possible to disable that behavior. Flapping detection will not apply cause it occurs temporally. In this example, if the Unknown state notifications are disabled/filtered, it looks like as if Icinga double notifies. And the notification interval is not helping, cause the event is triggered live.

Regards

[pefmeister]

This issue bothers me, too. This is also a real big bummer for our operations team. Critical services that have been acknowledged and experience such an Critical-Unknown-Critical transition lose their acknowledgement and (usually) their comment. This happens quite often as we also hit the problem that a config change increases the load on our satellites and create timeouts for the service checks.

[dnsmichi]

Why don't you use sticky acknowledgements then?

https://docs.icinga.com/icinga2/latest/doc/module/icinga2/toc#!/icinga2/latest/doc/module/icinga2/chapter/icinga2-api#icinga2-api-actions-acknowledge-problem

The original issue is about the fact that notification filters hide the state change transition and will notify 2x times critical. Which looks suspicious to the user, but works as designed. There are no acknowledgements involved.

[pefmeister]

I'll try that tomorrow, but since it is the default, I should have always used it :) I'll report back.

[pefmeister]

Tried it a couple of times and you're absolutely right, using the sticky flag works as designed. Is there any way to set this as a default in IW2?

[pefmeister]

OK, found it. Putting the following in /etc/icingaweb2/modules/monitoring/config.ini is what I need:

[settings]
acknowledge_sticky = 1

[dnsmichi]

Actually this requires changes in the state and notification logic to suppress notifications if some states in between have been filtered away. So to speak, Icinga 2 would need to keep track of filtered and unfiltered notifications.

I'm changing this into a feature request requiring a sponsor. Or you'll attach it to something like NoMa with advanced notification rules and history tracking.

[Al2Klimov]

@N-o-X Shall we just require not to filter out e.g. OK if you want to be notified for WARN->OK in addition to OK->WARN (wontfix)? Or shall we notify for both OK->WARN and WARN->OK if one allows notifications for WARN?

[Al2Klimov]

PING @N-o-X

[Al2Klimov]

It seems I have to re-address my question...

[julianbrost]

I'm not sure if I fully understand the problem and your suggestion, but isn't this that in a situation like OK -> WARNING -> UNKNOWN -> WARNING and you filter the notification states for WARNING (or OK + WARNING), you only want to receive one notification?

Shall we just require not to filter out e.g. OK if you want to be notified for WARN->OK in addition to OK->WARN (wontfix)? Or shall we notify for both OK->WARN and WARN->OK if one allows notifications for WARN?

I don't see how this would be possible with either of these suggestions.