groot/internal/rcompress: better buffer reuse

[sbinet]

we should consider introducing a better buffer+reader reuse strategy (e.g. for decompressor that support Reseting their readers)

this would reduce memory allocation and memory pressure.
this would then reduce the amount of work needed by the garbage collector.

issues:

• as decompression/compression are tasks that greatly benefit from concurrency, we should devise a way not to limit it too drastically
• races
• for decompression, we need a mechanism to declare that a buffer is no longer in use (and a way to correctly do that)
• consider using a type-parametrized sync.Map as proposed in proposal: sync, sync/atomic: add PoolOf, MapOf, ValueOf golang/go#47657

[bburghgr]

I stumbled upon this when playing around with some benchmarks that read a file using zlib compression, and a sync.Pool seems to work well enough. I think I can put something together for the various Resetters, if there's an interest.

In my toy benchmark, it reduces memory allocation (in my toy test) by about 60%, with a corresponding reduction in the number of GC cycles. Sadly, it doesn't seem to improve the event rate as much as I'd have hoped. On my machine, a file that needs ~30 seconds to process is sped up to ~28 seconds (wall time). I think I'm actually bottlenecked on something else, I'll open a separate issue describing what that may be.

[sbinet]

yes, if you have a PR that exercizes this, it'd be great.
(bonus points if it comes w/ benchstat data from groot-bench)

from your comment, I suspect that's what I tried to achieve with sbinet/pzlib?