Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Packetbeat] Redis: Limit memory used by replication #12741

Merged
merged 2 commits into from
Jul 2, 2019

Conversation

adriansr
Copy link
Contributor

@adriansr adriansr commented Jul 1, 2019

This patch limits the amount of memory that can be used by outstanding requests by adding two new configuration options to the Redis protocol:

  • max_queue_size: Limits the total size of the queued requests (in bytes).
  • max_queue_length: Limits the number of requests queued.

These limits apply to individual connections. The defaults are to queue up to 1MB or 20.000 requests. This is enough to limit the currently unbounded memory used by replication streams while at the same time allow for pipelining requests.

Closes #12657

This patch limits the amount of memory that can be used by outstanding
requests by adding two new configuration options to the Redis protocol:

- max_queue_size: Limits the total size of the queued requests (in bytes).
- max_queue_length: Limits the number of requests queued.

These limits apply to individual connections. The defaults are to queue
up to 1MB or 20.000 requests. This is enough to limit the currently
unbounded memory used by replication streams while at the same time
allow for pipelining requests.

Closes elastic#12657
Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

packetbeat/protos/redis/messagequeue.go Outdated Show resolved Hide resolved
Co-Authored-By: Andrew Kroh <andrew.kroh@elastic.co>
@adriansr adriansr merged commit 1275ee8 into elastic:master Jul 2, 2019
@adriansr adriansr added the needs_backport PR is waiting to be backported to other branches. label Jul 2, 2019
adriansr added a commit to adriansr/beats that referenced this pull request Jul 2, 2019
This patch limits the amount of memory that can be used by outstanding
requests by adding two new configuration options to the Redis protocol:

- max_queue_size: Limits the total size of the queued requests (in bytes).
- max_queue_length: Limits the number of requests queued.

These limits apply to individual connections. The defaults are to queue
up to 1MB or 20.000 requests. This is enough to limit the currently
unbounded memory used by replication streams while at the same time
allow for pipelining requests.

Closes elastic#12657

(cherry picked from commit 1275ee8)
adriansr added a commit to adriansr/beats that referenced this pull request Jul 2, 2019
This patch limits the amount of memory that can be used by outstanding
requests by adding two new configuration options to the Redis protocol:

- max_queue_size: Limits the total size of the queued requests (in bytes).
- max_queue_length: Limits the number of requests queued.

These limits apply to individual connections. The defaults are to queue
up to 1MB or 20.000 requests. This is enough to limit the currently
unbounded memory used by replication streams while at the same time
allow for pipelining requests.

Closes elastic#12657

(cherry picked from commit 1275ee8)
@adriansr adriansr added v7.2.1 and removed needs_backport PR is waiting to be backported to other branches. labels Jul 2, 2019
adriansr added a commit to adriansr/beats that referenced this pull request Jul 3, 2019
This patch limits the amount of memory that can be used by outstanding
requests by adding two new configuration options to the Redis protocol:

- max_queue_size: Limits the total size of the queued requests (in bytes).
- max_queue_length: Limits the number of requests queued.

These limits apply to individual connections. The defaults are to queue
up to 1MB or 20.000 requests. This is enough to limit the currently
unbounded memory used by replication streams while at the same time
allow for pipelining requests.

Closes elastic#12657

(cherry picked from commit 1275ee8)
@adriansr adriansr added the v6.8.2 label Jul 3, 2019
adriansr added a commit that referenced this pull request Jul 4, 2019
This patch limits the amount of memory that can be used by outstanding
requests by adding two new configuration options to the Redis protocol:

- max_queue_size: Limits the total size of the queued requests (in bytes).
- max_queue_length: Limits the number of requests queued.

These limits apply to individual connections. The defaults are to queue
up to 1MB or 20.000 requests. This is enough to limit the currently
unbounded memory used by replication streams while at the same time
allow for pipelining requests.

Closes #12657

(cherry picked from commit 1275ee8)
adriansr added a commit that referenced this pull request Jul 4, 2019
…eplication (#12752)

This patch limits the amount of memory that can be used by outstanding
requests by adding two new configuration options to the Redis protocol:

- max_queue_size: Limits the total size of the queued requests (in bytes).
- max_queue_length: Limits the number of requests queued.

These limits apply to individual connections. The defaults are to queue
up to 1MB or 20.000 requests. This is enough to limit the currently
unbounded memory used by replication streams while at the same time
allow for pipelining requests.

Closes #12657

(cherry picked from commit 1275ee8)
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
…ed by replication (elastic#12752)

This patch limits the amount of memory that can be used by outstanding
requests by adding two new configuration options to the Redis protocol:

- max_queue_size: Limits the total size of the queued requests (in bytes).
- max_queue_length: Limits the number of requests queued.

These limits apply to individual connections. The defaults are to queue
up to 1MB or 20.000 requests. This is enough to limit the currently
unbounded memory used by replication streams while at the same time
allow for pipelining requests.

Closes elastic#12657

(cherry picked from commit 4d0ea18)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[packetbeat] memory usage is growing continuously when capturing redis
2 participants