Skip to content

Commit

Permalink
Use dataset size instead of on-disk size for data stream stats (elast…
Browse files Browse the repository at this point in the history
…ic#103342)

Previously we were using the size of the shard that was on local disk,
however, in some cases (for object storage) this would return 0b, which
is very confusing. This commit changes it to use the dataset size, which
should be calculated correctly regardless of where the data resides.
  • Loading branch information
dakrone committed Dec 12, 2023
1 parent dc3d00e commit 92356f6
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 2 deletions.
5 changes: 5 additions & 0 deletions docs/changelog/103342.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pr: 103342
summary: Use dataset size instead of on-disk size for data stream stats
area: Data streams
type: bug
issues: []
Original file line number Diff line number Diff line change
Expand Up @@ -229,11 +229,11 @@ public DataStreamsStatsAction.Response newResponse(
assert dataStream != null;

// Aggregate global stats
totalStoreSizeBytes += shardStat.getStoreStats().sizeInBytes();
totalStoreSizeBytes += shardStat.getStoreStats().totalDataSetSizeInBytes();

// Aggregate data stream stats
AggregatedStats stats = aggregatedDataStreamsStats.computeIfAbsent(dataStream.getName(), s -> new AggregatedStats());
stats.storageBytes += shardStat.getStoreStats().sizeInBytes();
stats.storageBytes += shardStat.getStoreStats().totalDataSetSizeInBytes();
stats.maxTimestamp = Math.max(stats.maxTimestamp, shardStat.getMaxTimestamp());
}

Expand Down

0 comments on commit 92356f6

Please sign in to comment.