Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use dataset size instead of on-disk size for data stream stats #103342

Merged

Conversation

dakrone
Copy link
Member

@dakrone dakrone commented Dec 12, 2023

Previously we were using the size of the shard that was on local disk, however, in some cases (for object storage) this would return 0b, which is very confusing. This commit changes it to use the dataset size, which should be calculated correctly regardless of where the data resides.

Previously we were using the size of the shard that was on local disk, however, in some cases (for object storage) this would return 0b, which is very confusing. This commit changes it to use the dataset size, which should be calculated regardless of where the data resides.
@elasticsearchmachine
Copy link
Collaborator

Hi @dakrone, I've created a changelog YAML for you.

@elasticsearchmachine elasticsearchmachine added the Team:Data Management Meta label for data/management team label Dec 12, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

Copy link
Contributor

@andreidan andreidan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for fixing this Lee

@dakrone dakrone added auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) auto-backport-and-merge labels Dec 12, 2023
@elasticsearchmachine elasticsearchmachine merged commit fc0561b into elastic:main Dec 12, 2023
15 checks passed
@dakrone dakrone deleted the use-dataset-for-datastream-stats branch December 12, 2023 19:21
dakrone added a commit to dakrone/elasticsearch that referenced this pull request Dec 12, 2023
…ic#103342)

Previously we were using the size of the shard that was on local disk,
however, in some cases (for object storage) this would return 0b, which
is very confusing. This commit changes it to use the dataset size, which
should be calculated correctly regardless of where the data resides.
@elasticsearchmachine
Copy link
Collaborator

💚 Backport successful

Status Branch Result
8.12

elasticsearchmachine pushed a commit that referenced this pull request Dec 12, 2023
…) (#103345)

Previously we were using the size of the shard that was on local disk,
however, in some cases (for object storage) this would return 0b, which
is very confusing. This commit changes it to use the dataset size, which
should be calculated correctly regardless of where the data resides.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) >bug :Data Management/Data streams Data streams and their lifecycles Team:Data Management Meta label for data/management team v8.12.1 v8.13.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants