[APM] Alerts for specific transaction groups and error groups

[gbamparop]

APM users can create rules for specific services but are unable to create rules for specific transaction groups or error groups. This is a much requested feature because some parts of an application are more important than others and have different tolerance for latency, throughput and failed transaction rate thresholds.

Latency threshold and Failed transaction threshold rules

These rules already allow users to filter by service.name, service.environment and transaction.type.
A transaction.name filter should also be added, enabling users to only be alerted when a given transaction group exceeds the threshold.

Error count threshold rule

When creating this rule users can filter by service.name and service.environment. By adding another filter for error.grouping_key users can be alerted when a single error group exceeds a given threshold.

Progress

Add transaction name filter

• Latency threshold - [APM] Add transaction name filter in latency threshold rule  #154241
• Failed rate threshold - [APM] Add transaction name filter in failed transaction rate rule type #155405
• Error count not requested we're not going to implement it

Add error grouping key filter

• Error count - [APM] Add error grouping key filter in error count rule type  #155410

Links

• Pitch (internal)

[elasticmachine]

Pinging @elastic/apm-ui (Team:APM)

[kpatticha]

As we discussed in a call earlier we agreed that we won't group by the transaction name as part of the ticket but just add the transaction name filter.

The grouping will be handled as part of #152990 #154535, #154536, #154537

cc @benakansara, @sqren, @dgieselaar

[kpatticha]

@gbamparop @sqren

In the implementation ticket, there is no reference of adding the transaction name filter in the error count rule type. However, I see #155001 (comment) #155001 is added as a grouping field. Should we also consider adding the filter ?

[sorenlouv]

@kpatticha I don't understand the context of the link (it points to a comment about an issue with slack for the latency threshold rule) but I think you are asking if transaction.name should be added as a filter to error count rule.
If so, I don't think we need to do that right now. We should however allow the user to group by transaction.name and error.grouping_key for the error count rule.

[kpatticha]

oops, wrong link (fixed now) but you understand correctly. I'll make a note and if we have the time we can address adding the transaction.name in the error count rule.

For grouping I will reach out to AO

[benakansara]

@kpatticha @sqren I have added group-by transaction.name in all 3 APM rules - Latency threshold, Error count, Failed transaction rate in #155001

Should I add a new field in the dropdown to group alerts on error.grouping_key in Error count rule?

[sorenlouv]

Should I add a new field in the dropdown to group alerts on error.grouping_key in Error count rule?

@benakansara yes please! Can you also add error.grouping_name as a grouping option? (also mentioned in #155410 (comment))

[edfi-al]

Hello! I have a service named platform and inside of it there is a transaction called actions. I want to create an alert for the actions transaction.

So, firstly I went to APM, I chose the Alerts->Latency and in my view I have only the service [which is platform and i can't change it], type [I have selected request], environment [I have selected prod], WHEN avg is above 1500ms for the last 5 minutes.

But as you can imagine, I can't choose the transaction that I want [actions] and in this way my alert is for the whole service [platform].

Can you please help me? Thank u!

[benakansara]

@edfi-al Could you tell us which version of Kibana are you using? We have added transaction name filter in Latency threshold rule in 8.8.0. There is a filter called "NAME".

Additionally, in 8.10.0, we have added KQL filter where you can specify any filters with KQL query.

[edfi-al]

Hello @benakansara and thank you for replying. I'm using v.7.11.1.

Must I upgrade to at least v.8.8.0 in order to use NAME in alert?

[benakansara]

@edfi-al yes, at least 8.8.0 is needed to use new NAME filter.