[extension/filestorage] Online file storage compaction

[pmm-sumo]

Description:

Adds online filestorage compaction

Link to tracking Issue: Addresses core/#5159

Testing: Benchmarks and several tests added. Also tested manually in several scenarios

Documentation: README.md updated

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: swiatekm-sumo / name: Mikołaj Świątek (9a32567897acce8c04418802d6eb744f95d9bf56)

[swiatekm]

I wonder if we shouldn't separate the changes adding the new compaction condition from changes to the interface. Before, client.Compact() was a destructive operation that returned a new client, now it operates on the client in-place. Making that a separate change would also let us fix the issue where compaction failing on start makes the client creation fail, whereas it could just continue as is.

[swiatekm]

I did some integration testing of this PR. With changes proposed in this review, it worked perfectly fine in a standard k8s environment, pushing around 10k records per second through compactions as the persistent queue grew and drained.

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 14 days.

[github-actions]

Closed as inactive. Feel free to reopen if this PR is still being worked on.

[overmeulen]

Hi, isn't this dev going to be merged ?
I was really looking forward to having this feature

[pmm-sumo]

@overmeulen sorry for the delay, I'm getting back to this right now

[djaglowski]

Were these terms, "total size above" and "size below", chosen for this extension, or are they established elsewhere?

[pmm-sumo]

This are new terms. I spent some time to figure out names that would be a good match here and this is what I came up with eventually (e.g. using "rebound" for what happens here). If you have any other suggestions, those are always welcome :)

[djaglowski]

I think "rebound" is a very intuitive term.

I found these two parameters to be a little less intuitive, but ultimately I think between the diagram and descriptions it becomes reasonably clear.

If I were to chose alternate terms, I might try to include the word "threshold", which I think would convey that something happens when the value is crossed. I also think it might be helpful to present these in a way that more directly reflects the statefulness of the system. Maybe something like:

• compaction.rebound_needed_threshold_mib (default: 100) - when allocated data exceeds this amount, the "compaction needed" flag will be set
• compaction.rebound_trigger_threshold_mib (default: 10) - if the "compaction needed" flag is set and allocated data drops below this amount, compaction will begin and the "compaction needed" flag will be cleared

[pmm-sumo]

That's a great advice, I like these suggestions a lot @djaglowski. What do you think @swiatekm-sumo? I am going to apply these

[djaglowski]

@pmm-sumo, looks good to me. Let me know if I should merge or wait for @swiatekm-sumo to weigh in

[swiatekm]

Some minor nits, looks good overall.