Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[APM] Display warning if transaction groups are dropped #146648

Closed
sorenlouv opened this issue Nov 29, 2022 · 4 comments · Fixed by #148625
Closed

[APM] Display warning if transaction groups are dropped #146648

sorenlouv opened this issue Nov 29, 2022 · 4 comments · Fixed by #148625
Assignees
@yngrdyn
Labels
apm:release-feature APM UI - Release Feature Goal apm:test-plan-done Pull request that was successfully tested during the test plan Team:APM All issues that need APM UI Team support v8.7.0

Comments

@sorenlouv
Copy link
Member

Background

Some services (e.g. RUM) may produce high cardinality transaction names, which could lead to an extremely high number of transaction groups. To avoid one service eating up the limit for all other services, a transaction group limit per-service is introduced.

Once the limit is reached, any additional transaction will be added to a dedicated "other" bucket, (transaction.name: "_other")

UI implications for transactions table

The "other" bucket will automatically show up in the transactions table on the service overview and transactions overview.
We should change the link for the "other" group so it is no longer possible to navigate to the transaction details page. Instead, we should show a warning sign with a tooltip that explains what the other group is ("The transaction group limit of 10.000 has been reached. Please see the APM Server docs for $SETTING_NAME to increase this").

Links
@sorenlouv sorenlouv added Team:APM All issues that need APM UI Team support apm:release-feature APM UI - Release Feature Goal v8.7.0 labels Nov 29, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/apm-ui (Team:APM)

@sorenlouv sorenlouv mentioned this issue Nov 30, 2022
Closed
@yngrdyn yngrdyn self-assigned this Jan 5, 2023
@yngrdyn
Copy link
Contributor

yngrdyn commented Jan 10, 2023

We created a synthtrace scenario to create more than 1000tx groups and we got this warning:

image

@sqren with this changes, should we get rid of this warning? I think it's not valid anymore

@sorenlouv
Copy link
Member Author

sorenlouv commented Jan 10, 2023 •
edited
Loading

The warning might be ok to keep in addition to the tooltip on the _other bucket. But the logic for showing the the warning needs to be changed. Currently the logic is:

isAggregationAccurate:
      (response.aggregations?.transaction_groups.sum_other_doc_count ?? 0) ===
      0,

The new logic should show a warning, if there is an _other bucket:

const didReachMaxNumberOfGroups = transactionGroups.includes(group => group.name === '_other')

@yngrdyn yngrdyn mentioned this issue Jan 10, 2023
Merged
yngrdyn added a commit that referenced this issue Jan 30, 2023
@yngrdyn @achyutjhunjhunwala
[APM] Show warning if transaction groups are dropped (#148625)
3d3a885 
Closes #146648.

### Changes
- `transaction_detail_link.tsx` prevent navigation to tx details when
bucket name is other and display the warning tooltip
- `xpack.apm.ui.transactionGroupBucketSize` ui setting was removed and
now we have a hardcoded limit of `1000` buckets when querying
transactions.

**After this change**


https://user-images.githubusercontent.com/1313018/214321647-e0ae59d6-0761-4d9e-a0fe-ef21397aeff7.mov


- When kibana limit has been reached
<img width="1391" alt="image"
src="https://user-images.githubusercontent.com/1313018/214321902-05fbfa6c-7c99-4b21-a67b-7e134c14ab73.png">

- When having only `_other`
<img width="1390" alt="image"
src="https://user-images.githubusercontent.com/1313018/214332887-d6fd59a2-d1ef-4b61-b6ed-79a6d458f0a0.png">

### Test instructions
1. Checkout PR branch
2. Execute Synthtrace scenario
    a. For reaching kibana limit callout
        ```
node scripts/synthtrace --clean other_transaction_group_bucket.ts
--scenarioOpts.txGroups=1001
        ```
    b. For `_other` callout
        ```
node scripts/synthtrace --clean other_transaction_group_bucket.ts
        ```

---------

Co-authored-by: Achyut Jhunjhunwala <achyut.jhunjhunwala@elastic.co>
@yngrdyn yngrdyn mentioned this issue Jan 31, 2023
Merged
yngrdyn added a commit that referenced this issue Feb 2, 2023
@yngrdyn
Combining other bucket groups scenarios into a single one (#149923)
418cad1 
No need to have `_other` bucket for services and transaction in separate
scenarios.
Additionally this scenario is prepared to receive number of services
and/or number of transaction groups through params:
```
e.g.
--scenarioOpts.services=10001 --scenarioOpts.txGroups=1001 
```

Relates to #146650 and
#146648.
kqualters-elastic pushed a commit to kqualters-elastic/kibana that referenced this issue Feb 6, 2023
@yngrdyn @achyutjhunjhunwala @kqualters-elastic
[APM] Show warning if transaction groups are dropped (elastic#148625)
97d3790 
Closes elastic#146648.

### Changes
- `transaction_detail_link.tsx` prevent navigation to tx details when
bucket name is other and display the warning tooltip
- `xpack.apm.ui.transactionGroupBucketSize` ui setting was removed and
now we have a hardcoded limit of `1000` buckets when querying
transactions.

**After this change**


https://user-images.githubusercontent.com/1313018/214321647-e0ae59d6-0761-4d9e-a0fe-ef21397aeff7.mov


- When kibana limit has been reached
<img width="1391" alt="image"
src="https://user-images.githubusercontent.com/1313018/214321902-05fbfa6c-7c99-4b21-a67b-7e134c14ab73.png">

- When having only `_other`
<img width="1390" alt="image"
src="https://user-images.githubusercontent.com/1313018/214332887-d6fd59a2-d1ef-4b61-b6ed-79a6d458f0a0.png">

### Test instructions
1. Checkout PR branch
2. Execute Synthtrace scenario
    a. For reaching kibana limit callout
        ```
node scripts/synthtrace --clean other_transaction_group_bucket.ts
--scenarioOpts.txGroups=1001
        ```
    b. For `_other` callout
        ```
node scripts/synthtrace --clean other_transaction_group_bucket.ts
        ```

---------

Co-authored-by: Achyut Jhunjhunwala <achyut.jhunjhunwala@elastic.co>
kqualters-elastic pushed a commit to kqualters-elastic/kibana that referenced this issue Feb 6, 2023
@yngrdyn @kqualters-elastic
Combining other bucket groups scenarios into a single one (elastic#14…
4b0dffb 
…9923)

No need to have `_other` bucket for services and transaction in separate
scenarios.
Additionally this scenario is prepared to receive number of services
and/or number of transaction groups through params:
```
e.g.
--scenarioOpts.services=10001 --scenarioOpts.txGroups=1001 
```

Relates to elastic#146650 and
elastic#146648.
@yngrdyn yngrdyn mentioned this issue Feb 7, 2023
Merged
yngrdyn added a commit that referenced this issue Feb 8, 2023
@yngrdyn
Making maxTransactionGroupsExceeded false by default (#150458)
a2df4d6 
Relates to #146648.

`maxTransactionGroupsExceeded` is currently `true` by default which is
causing a flickering in transactions page specifically for the table,
where we can see the callout for some milliseconds and then, after
getting results from endpoint, disappears.

### before this change


https://user-images.githubusercontent.com/1313018/217316916-e6434f4a-c4e6-4e12-ba2b-5f5ab0011984.mov
@yngrdyn
Copy link
Contributor

yngrdyn commented Feb 21, 2023

Tested on cloud deployment

  1. _other txGroup showing as Remaining Transactions in service overview transactions table

image

  1. Callout when txGroup exceeds Kibana limits

image

  1. _other txGroup showing as Remaining Transactions in transactions overview table

image

@yngrdyn yngrdyn added the apm:test-plan-done Pull request that was successfully tested during the test plan label Feb 21, 2023
bmorelli25 pushed a commit to bmorelli25/observability-docs that referenced this issue Apr 3, 2024
@yngrdyn @achyutjhunjhunwala
[APM] Show warning if transaction groups are dropped (#148625)
a23ac88 
Closes elastic/kibana#146648.

### Changes
- `transaction_detail_link.tsx` prevent navigation to tx details when
bucket name is other and display the warning tooltip
- `xpack.apm.ui.transactionGroupBucketSize` ui setting was removed and
now we have a hardcoded limit of `1000` buckets when querying
transactions.

**After this change**


https://user-images.githubusercontent.com/1313018/214321647-e0ae59d6-0761-4d9e-a0fe-ef21397aeff7.mov


- When kibana limit has been reached
<img width="1391" alt="image"
src="https://user-images.githubusercontent.com/1313018/214321902-05fbfa6c-7c99-4b21-a67b-7e134c14ab73.png">

- When having only `_other`
<img width="1390" alt="image"
src="https://user-images.githubusercontent.com/1313018/214332887-d6fd59a2-d1ef-4b61-b6ed-79a6d458f0a0.png">

### Test instructions
1. Checkout PR branch
2. Execute Synthtrace scenario
    a. For reaching kibana limit callout
        ```
node scripts/synthtrace --clean other_transaction_group_bucket.ts
--scenarioOpts.txGroups=1001
        ```
    b. For `_other` callout
        ```
node scripts/synthtrace --clean other_transaction_group_bucket.ts
        ```

---------

Co-authored-by: Achyut Jhunjhunwala <achyut.jhunjhunwala@elastic.co>
dedemorton pushed a commit to dedemorton/observability-docs that referenced this issue Aug 6, 2024
@yngrdyn @achyutjhunjhunwala @dedemorton
[APM] Show warning if transaction groups are dropped (#148625)
d873825 
Closes elastic/kibana#146648.

### Changes
- `transaction_detail_link.tsx` prevent navigation to tx details when
bucket name is other and display the warning tooltip
- `xpack.apm.ui.transactionGroupBucketSize` ui setting was removed and
now we have a hardcoded limit of `1000` buckets when querying
transactions.

**After this change**


https://user-images.githubusercontent.com/1313018/214321647-e0ae59d6-0761-4d9e-a0fe-ef21397aeff7.mov


- When kibana limit has been reached
<img width="1391" alt="image"
src="https://user-images.githubusercontent.com/1313018/214321902-05fbfa6c-7c99-4b21-a67b-7e134c14ab73.png">

- When having only `_other`
<img width="1390" alt="image"
src="https://user-images.githubusercontent.com/1313018/214332887-d6fd59a2-d1ef-4b61-b6ed-79a6d458f0a0.png">

### Test instructions
1. Checkout PR branch
2. Execute Synthtrace scenario
    a. For reaching kibana limit callout
        ```
node scripts/synthtrace --clean other_transaction_group_bucket.ts
--scenarioOpts.txGroups=1001
        ```
    b. For `_other` callout
        ```
node scripts/synthtrace --clean other_transaction_group_bucket.ts
        ```

---------

Co-authored-by: Achyut Jhunjhunwala <achyut.jhunjhunwala@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yngrdyn yngrdyn

Labels
apm:release-feature APM UI - Release Feature Goal apm:test-plan-done Pull request that was successfully tested during the test plan Team:APM All issues that need APM UI Team support v8.7.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[APM] Show warning if transaction groups are dropped yngrdyn/kibana
4 participants
@sorenlouv @yngrdyn @gbamparop @elasticmachine