Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Server Side batching for UI Metric Collectors #6721

Merged
merged 2 commits into from
May 22, 2024

Conversation

LDrago27
Copy link
Collaborator

@LDrago27 LDrago27 commented May 6, 2024

Description

This PR introduces Server Side batching of UI metric reports sent by clients to reduce the number of updates taking place at OpenSearch.

Issues Resolved

#6375

Testing the changes

  • Enable UI Metric Collector by setting the usageCollection.uiMetric.enabled to true
  • Ascertain the ui_metric in usage section is empty when you call the /api/stats?extended=true&legacy=true&exclude_usage=false
  • Make a POST call to the /api/ui_metric/report with the following request body {"report":{"reportVersion":1,"uiStatsMetrics":{"console-count-GET_cat.indices":{"key":"console-count-GET_cat.indices","appName":"console","eventName":"GET_cat.indices","type":"count","stats":{"min":0,"max":1,"avg":0.5,"sum":21}}}}}
  • Call /api/stats?extended=true&legacy=true&exclude_usage=false and verify that it has an entry {"key":"GET_cat.indices","value":21}
  • Make a POST call to the /api/ui_metric/report with the following request body {"report":{"reportVersion":1,"uiStatsMetrics":{"console-count-GET_cat.indices":{"key":"console-count-GET_cat.indices","appName":"console","eventName":"GET_cat.indices","type":"count","stats":{"min":0,"max":1,"avg":0.5,"sum":21}}}}}
  • Call /api/stats?extended=true&legacy=true&exclude_usage=false and verify that it has an entry {"key":"GET_cat.indices","value":21} the count shouldn't have been updated
  • Wait for a period of 60s. And Make a POST call to the /api/ui_metric/report with the following request body {"report":{"reportVersion":1,"uiStatsMetrics":{"console-count-GET_cat.indices":{"key":"console-count-GET_cat.indices","appName":"console","eventName":"GET_cat.indices","type":"count","stats":{"min":0,"max":1,"avg":0.5,"sum":21}}}}}
    • Call /api/stats?extended=true&legacy=true&exclude_usage=false and verify that it has an entry {"key":"GET_cat.indices","value":63} the count should be updated to the latest value

OR

You can pull in changes opensearch-project/opensearch-dashboards-functional-test#1267 update the flag "UIMETRIC_ENABLED": true and run the following command yarn cypress run --spec cypress/integration/core-opensearch-dashboards/opensearch-dashboards/apps/telemetry/server.spec.js . This test automates the above mentioned steps.

Changelog

  • feat: Add Server Side Batching for UI Metric Colector

Check List

  • All tests pass
    • yarn test:jest
    • yarn test:jest_integration
  • New functionality includes testing.
  • New functionality has been documented.
  • Update CHANGELOG.md
  • Commits are signed per the DCO using --signoff

Copy link
Contributor

github-actions bot commented May 6, 2024

❌ Changelog Entry Missing Hyphen

Changelog entries must begin with a hyphen (-).

opensearch-changeset-bot bot added a commit to LDrago27/OpenSearch-Dashboards that referenced this pull request May 6, 2024
Copy link

codecov bot commented May 6, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 67.57%. Comparing base (5258d83) to head (ce56129).
Report is 26 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6721      +/-   ##
==========================================
- Coverage   67.72%   67.57%   -0.15%     
==========================================
  Files        3415     3428      +13     
  Lines       66886    67343     +457     
  Branches    10884    10996     +112     
==========================================
+ Hits        45297    45508     +211     
- Misses      18943    19165     +222     
- Partials     2646     2670      +24     
Flag Coverage Δ
Linux_1 ?
Linux_2 55.13% <100.00%> (-0.46%) ⬇️
Linux_3 45.17% <0.00%> (-0.06%) ⬇️
Linux_4 34.73% <50.00%> (-0.13%) ⬇️
Windows_1 33.09% <0.00%> (-0.15%) ⬇️
Windows_2 55.10% <100.00%> (-0.46%) ⬇️
Windows_3 45.17% <0.00%> (-0.06%) ⬇️
Windows_4 34.73% <50.00%> (-0.13%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@LDrago27 LDrago27 force-pushed the serverSideBatching branch from 9e56ca1 to 364f2a0 Compare May 7, 2024 19:58
@kavilla kavilla linked an issue May 8, 2024 that may be closed by this pull request
@kavilla kavilla added the v2.15.0 label May 8, 2024
@LDrago27 LDrago27 force-pushed the serverSideBatching branch from 364f2a0 to 18adcfc Compare May 8, 2024 17:38
kavilla
kavilla previously approved these changes May 13, 2024
Signed-off-by: Suchit Sahoo <suchsah@amazon.com>
@LDrago27 LDrago27 force-pushed the serverSideBatching branch from bf467c5 to 1f9f140 Compare May 17, 2024 19:20
@LDrago27 LDrago27 requested a review from mengweieric as a code owner May 17, 2024 19:20
@LDrago27 LDrago27 requested a review from AMoo-Miki May 17, 2024 19:37
@kavilla kavilla merged commit 972cf10 into opensearch-project:main May 22, 2024
67 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request May 22, 2024
* Add Server Side batching for UI Metric Collectors

Signed-off-by: Suchit Sahoo <suchsah@amazon.com>

* Changeset file for PR #6721 created/updated

This PR introduces Server Side batching of UI metric reports sent by clients to reduce the number of updates taking place at OpenSearch.
Issues Resolved

#6375

---------

Signed-off-by: Suchit Sahoo <suchsah@amazon.com>
Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>
(cherry picked from commit 972cf10)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
kavilla pushed a commit that referenced this pull request May 23, 2024
* Add Server Side batching for UI Metric Collectors



* Changeset file for PR #6721 created/updated

This PR introduces Server Side batching of UI metric reports sent by clients to reduce the number of updates taking place at OpenSearch.
Issues Resolved

#6375

---------



(cherry picked from commit 972cf10)

Signed-off-by: Suchit Sahoo <suchsah@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>
LDrago27 added a commit to LDrago27/OpenSearch-Dashboards that referenced this pull request Jun 3, 2024
…#6721)

* Add Server Side batching for UI Metric Collectors

Signed-off-by: Suchit Sahoo <suchsah@amazon.com>

* Changeset file for PR opensearch-project#6721 created/updated

This PR introduces Server Side batching of UI metric reports sent by clients to reduce the number of updates taking place at OpenSearch.
Issues Resolved

opensearch-project#6375

---------

Signed-off-by: Suchit Sahoo <suchsah@amazon.com>
Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Server Side batching for saving UI metrics
5 participants