refactor(meta): prefixed auto full GC #18522
Open
+197
−8
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
After introducing time travel, there is expected to be much more objects in object store. To reduce the overhead associated with scanning the entire data direcoty during auto full GC, this PR refines auto full GC by adding a rotational prefix when listing objects.
Note that the path to an object within a cluster can either be
data_dir/shard_id/object_name
ordata_dir/object_name
, depending on the specific configuration settings. The setting is not straightforward to obtain. So by default this fine-grain auto full GC is not enabled. It should be set later accordingly.Checklist
./risedev check
(or alias,./risedev c
)Documentation
Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.