recreate compacted boltdb files from compactor when they are more than 12 hours old #4853
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
This PR changes the compactor code to re-create compacted boltdb files using bbolt.Compact for 2 reasons:
See database file size not updating? boltdb/bolt#308 for more details.
This setting is optimal for unordered writes.
bbolt.Compact fills the whole page by setting FillPercent to 1 which works well here since while copying the data, it receives
the index entries in order. The storage space goes down from anywhere between 25% to 50% as per my tests.
The compacted dbs would be re-created when:
To simplify the code, I have also changed the compactor code to drop the requirement of having more than 4 files for doing the compaction.
Checklist
CHANGELOG.md
about the changes.