Add more compression algorithms to prometheus_remote_write

[zamazan4ik]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

According to the issue, VictoriaMetrics team is interested in gzip/zstd compression support.

Vector right now supports only Snappy: https://github.com/vectordotdev/vector/blob/master/src/sinks/prometheus/remote_write.rs#L357

Attempted Solutions

No response

Proposal

Add GZip and ZSTD support to prometheus_remote_write sink.

References

No response

Version

No response

[zamazan4ik]

Found another point for improvement. We need to document:

• Supported compression algorithms by prometheus_remote_write source. Right now it supports Deflate, Gzip, Snappy, Zstd
• After the addition Gzip and Zstd compression algorithms to prometheus_remote_write sink - do not forget to document it.

By the way, what is the better way to add custom compression algorithms to prometheus_remote_write sink? Usual compression field in the config of the sink?

And once again I would like to highlight the issue - we still do not have a generic way to specify the compression level for the compression algorithms. It could be useful for tuning compression/CPU consumption ratio.

[jszwedko]

By the way, what is the better way to add custom compression algorithms to prometheus_remote_write sink? Usual compression field in the config of the sink?

Yeah, I think we'd just follow the existing patterns to add these new algorithms.

[valyala]

It would be great to have the ability to use zstd compression for Prometheus remote_write protocol in order to be compatible with VictoriaMetrics components - see https://victoriametrics.com/blog/victoriametrics-remote-write/