Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enable gzip and deflate in http client #328

Merged
merged 3 commits into from
Jul 15, 2021

Conversation

jeremyandrews
Copy link
Member

@jeremyandrews jeremyandrews commented Jul 14, 2021

  • enable gzip support in the http client
  • can be disabled with --no-gzip or GooseDefault::NoGzip
  • documented how to manually enable deflate and/or brotli compression

@jeremyandrews
Copy link
Member Author

jeremyandrews commented Jul 15, 2021

I ran current -dev w/o this patch, and current -dev w/ this patch, each 3 times as follows (restarting all server processes before each batch of 3):

cargo run --release --example drupal_loadtest -- --host http://apache -v -u12 -t10m && sleep 30 && cargo run --release --example drupal_loadtest -- --host http://apache -v -u12 -t10m && sleep 30 && cargo run --release --example drupal_loadtest -- --host http://apache -v -u12 -t10m

Here's the data:
https://docs.google.com/spreadsheets/d/1lsOagHY1yISQ_HcpwbIxoxXx8s2q5KxKYXvZmhgsMYs/edit#gid=0

Goose made >30% more anonymous requests, but 5.6% fewer authenticated requests with this patch applied. Without compression it averaged 6,473 requests per second. With compression it averaged 9,542 requests per second. It would be good to re-run the test with fewer GooseUsers and an evenly balanced number of Anonymous and Authenticated users: my assumption is that the reduction in authenticated traffic is simply because anonymous traffic was being returned so quickly by Varnish that it was getting scheduled far more.

The Apache/Varnish server saw no change in CPU usage (as Varnish caches compressed pages), whereas the Goose server saw an increase in CPU usage, visible here (Goose is running in a 32-core VM):
CPU usage
(On the left is with compression, on the right is without compression: the spike is re-compiling Goose)

Though it made considerably more requests, obviously there was considerably less network traffic generated:
Network usage
(On the left is without compression, on the right is with compression)

@jeremyandrews jeremyandrews changed the title enable gzip and deflate in http client [WIP] enable gzip and deflate in http client Jul 15, 2021
@LionsAd
Copy link
Collaborator

LionsAd commented Jul 15, 2021

Let’s also add the request headers to the debug log and:or add a typical HTTP header to the docs.

@jeremyandrews jeremyandrews changed the title [WIP] enable gzip and deflate in http client enable gzip and deflate in http client Jul 15, 2021
@jeremyandrews
Copy link
Member Author

Authenticated requests slow down as Apache is compressing each request due to how my server is configured. There's additional CPU usage showing up on the Apache server because of this.

@jeremyandrews jeremyandrews merged commit 38bb62b into tag1consulting:main Jul 15, 2021
@jeremyandrews jeremyandrews deleted the compress branch August 11, 2021 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants