Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include min/max/average file size in Searchable Snapshots Stats API #70294

Merged
merged 3 commits into from
Mar 11, 2021

Conversation

tlrx
Copy link
Member

@tlrx tlrx commented Mar 11, 2021

This pull request adds the minimum, maximum and average length of files in the Searchable Snapshots Stats API:

 {
      "file_ext" : "fdt",
      "num_files" : 31,
      "total_size" : 417698246269,
      "min_size" : "480mb",
      "min_size_in_bytes" : 503370620,
      "max_size" : "12.9gb",
      "max_size_in_bytes" : 13947816041,
      "avg_size" : "12.5gb",
      "avg_size_in_bytes" : 13474136976,
      ...
}

These information are useful in understanding why some data is still being read from the blob store when the cached blob in .snapshot-blob-cache should have been sufficient to server all the read operations.

@tlrx tlrx added >enhancement :Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs v8.0.0 v7.13.0 labels Mar 11, 2021
@elasticmachine elasticmachine added the Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. label Mar 11, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

Copy link
Contributor

@ywelsch ywelsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very useful to add this info to the stats API. I've left some minor comments w.r.t. consistency between total and min/max fields. I'm also wondering we should fold CacheIndexInputStats.{total_size,min_size,max_size,avg_size} into a size section (i.e. have fields "total,min,max,avg" under a "size" object)

@tlrx
Copy link
Member Author

tlrx commented Mar 11, 2021

@ywelsch thanks for your feedback. I've updated the code, can you please have another look when you have time?

@tlrx tlrx requested a review from ywelsch March 11, 2021 15:37
Copy link
Contributor

@ywelsch ywelsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tlrx tlrx merged commit 8d2c172 into elastic:master Mar 11, 2021
@tlrx tlrx deleted the avg-size-ss-stats branch March 11, 2021 16:38
@tlrx
Copy link
Member Author

tlrx commented Mar 11, 2021

Thanks Yannick

tlrx added a commit to tlrx/elasticsearch that referenced this pull request Mar 11, 2021
…lastic#70294)

This commit adds the minimum, maximum and average length 
of files in the Searchable Snapshots Stats API.
tlrx added a commit that referenced this pull request Mar 12, 2021
… API (#70327)

This commit adds the minimum, maximum and average length
of files in the Searchable Snapshots Stats API.

Backport of #70294 for 7.x
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs >enhancement Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. v7.13.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants