Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Remote Store] Optimize remote store restore flow to not download all the files from remote if local disk already has data #9984

Closed
linuxpi opened this issue Sep 12, 2023 · 3 comments
Labels
enhancement Enhancement or improvement to existing feature or request Storage:Remote

Comments

@linuxpi
Copy link
Collaborator

linuxpi commented Sep 12, 2023

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

  • When we trigger remote restore via RemoteStoreRecoverySource, we download all the segment files from remote irrespective of the fact that local disk may contain some of those files.
  • We should provide a choice through RemoteStoreRestore flow whether all segment files should be downloaded from remote even if local disk already contains some of them

Describe the solution you'd like
Provide a parameter/setting to the user which allows them to control whether to download all segment files from remote during recovery of a remote backed shard

@shwetathareja
Copy link
Member

@linuxpi does this issue still exist?

@linuxpi
Copy link
Collaborator Author

linuxpi commented Nov 26, 2023

@shwetathareja I dont think we changes anything around this. I'll update this issue with more details

@gbbafna
Copy link
Collaborator

gbbafna commented Aug 8, 2024

Storage Triage : Attendees 1 2 3 4 5 6

This is not relevant anymore as we only recover unassigned shards in remote restore.

@gbbafna gbbafna closed this as completed Aug 8, 2024
@github-project-automation github-project-automation bot moved this from 🆕 New to ✅ Done in Storage Project Board Aug 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Storage:Remote
Projects
Status: ✅ Done
Development

No branches or pull requests

5 participants