Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[confighttp] Support snappy and zstd compression on the server side #7632

Closed
swiatekm opened this issue May 5, 2023 · 8 comments
Closed

[confighttp] Support snappy and zstd compression on the server side #7632

swiatekm opened this issue May 5, 2023 · 8 comments
Labels
area:config

Comments

@swiatekm
Copy link
Contributor

swiatekm commented May 5, 2023

We should support the same compression methods on the client and server sides of confighttp. Right now we have:

  • Client side: gzip, zlib, deflate, zstd, snappy
  • Server side: gzip, zlib, deflate

If there isn't a good reason we don't support zstd and snappy on the server side, then I can work on adding them.
@atoulme
Copy link
Contributor

atoulme commented May 8, 2023

Please take a look at #7636

This was referenced May 8, 2023
Closed
Merged
@tigrannajaryan
Copy link
Member

Ideally we also want a compression method negotiation added to OTLP spec so that we can have an "auto" setting.

@atoulme
Copy link
Contributor

atoulme commented May 9, 2023

How would that work? Would you use a similar mechanism as Accept-Encoding? https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding

@tigrannajaryan
Copy link
Member

How would that work? Would you use a similar mechanism as Accept-Encoding? https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding

Yes, with 2-step negotiation for the server's accepting side. The initial client request should be in old encoding until it gets a response from the server about acceptance of the new encoding. Client should also be ready to go back (can happen in load-balanced deployments, mid new version rollout).

@atoulme
Copy link
Contributor

atoulme commented May 10, 2023

That sounds like it warrants its own issue. I will create one and link it here.

@atoulme atoulme mentioned this issue May 10, 2023
Open
@swiatekm
Copy link
Contributor Author

zstd was added in #7927, but snappy is still missing.

@JustinMason JustinMason mentioned this issue Nov 22, 2023
Merged
@JustinMason
Copy link
Contributor

JustinMason commented Nov 22, 2023 •
edited
Loading

Seeking authorization to commit for the linux foundation, but in the meantime this WIP handles the snappy compression, supporting a Prometheus remote write endpoint using the confighttp core lib.
#8983

codeboten pushed a commit that referenced this issue Feb 12, 2024
@JustinMason @bogdandrutu
Added support for snappy to enable prometheus remote write (#8983)
1a57837 
Adding a feature.

This adds `snappy` support for the confighttp compression package.
This provides support for enabling the prometheus remote write receiver
which only supports snappy.

#7632

Added unit test coverage and testing integration with prometheus remote
write compatibility.

---------

Co-authored-by: Bogdan Drutu <bogdandrutu@gmail.com>
@swiatekm
Copy link
Contributor Author

Looks like this can be closed with #8983.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area:config
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[config/confighttp] Support zstd and snappy decompress
4 participants
@atoulme @JustinMason @tigrannajaryan @swiatekm