Skip to content

Commit

Permalink
Use dataset size instead of on-disk size for data stream stats (#103342)
Browse files Browse the repository at this point in the history
Previously we were using the size of the shard that was on local disk,
however, in some cases (for object storage) this would return 0b, which
is very confusing. This commit changes it to use the dataset size, which
should be calculated correctly regardless of where the data resides.
  • Loading branch information
dakrone authored Dec 12, 2023
1 parent f5be180 commit fc0561b
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 2 deletions.
5 changes: 5 additions & 0 deletions docs/changelog/103342.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pr: 103342
summary: Use dataset size instead of on-disk size for data stream stats
area: Data streams
type: bug
issues: []
Original file line number Diff line number Diff line change
Expand Up @@ -229,11 +229,11 @@ public DataStreamsStatsAction.Response newResponse(
assert dataStream != null;

// Aggregate global stats
totalStoreSizeBytes += shardStat.getStoreStats().sizeInBytes();
totalStoreSizeBytes += shardStat.getStoreStats().totalDataSetSizeInBytes();

// Aggregate data stream stats
AggregatedStats stats = aggregatedDataStreamsStats.computeIfAbsent(dataStream.getName(), s -> new AggregatedStats());
stats.storageBytes += shardStat.getStoreStats().sizeInBytes();
stats.storageBytes += shardStat.getStoreStats().totalDataSetSizeInBytes();
stats.maxTimestamp = Math.max(stats.maxTimestamp, shardStat.getMaxTimestamp());
}

Expand Down

0 comments on commit fc0561b

Please sign in to comment.