Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Remote Store] Garbage Collection for Remote Backed Storage #8378

Open
psychbot opened this issue Jun 30, 2023 · 0 comments
Open

[Remote Store] Garbage Collection for Remote Backed Storage #8378

psychbot opened this issue Jun 30, 2023 · 0 comments
Labels
enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework Storage Issues and PRs relating to data and metadata storage

Comments

@psychbot
Copy link
Member

Is your feature request related to a problem? Please describe.
As part of #8201 we are moving stale commit deletion to asynchronous flow. Its possible that there can be files which doesn't get deleted from remote store due asynchronous nature.

Eg - If the deletion fails while closing an index this might lead to lingering files present in S3.

Describe the solution you'd like
A background time based job which will try to keep only the files which are referenced locally and delete anything which is not referenced from remote store.

@psychbot psychbot added enhancement Enhancement or improvement to existing feature or request untriaged labels Jun 30, 2023
@Rishikesh1159 Rishikesh1159 added the Storage:Durability Issues and PRs related to the durability framework label Jul 5, 2023
@Bukhtawar Bukhtawar added the Storage Issues and PRs relating to data and metadata storage label Jul 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework Storage Issues and PRs relating to data and metadata storage
Projects
Status: 🆕 New
Development

No branches or pull requests

4 participants