-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a memory limiter processor #498
Add a memory limiter processor #498
Conversation
For it to be effective it needs to be right after the receivers, ie.: first processor on the pipeline. |
Right, I keep forgetting that batch is also async. |
Codecov Report
@@ Coverage Diff @@
## master #498 +/- ##
==========================================
+ Coverage 75.73% 75.96% +0.22%
==========================================
Files 119 122 +3
Lines 7414 7551 +137
==========================================
+ Hits 5615 5736 +121
- Misses 1530 1543 +13
- Partials 269 272 +3
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code LGTM, just one comment about the "recommended" frequency for the check.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
This adds a processor that drops data according to configured memory limits. The processor is important for high load situations when receiving rate exceeds exporting rate (and an extreme case of this is when the target of exporting is unavailable). Typical production run will need to have this processor included in every pipeline immediately after the batch processor.
34123a2
to
c2ed0c7
Compare
This adds a processor that drops data according to configured memory limits.
The processor is important for high load situations when receiving rate exceeds exporting
rate (and an extreme case of this is when the target of exporting is unavailable).
Typical production run will need to have this processor included in every pipeline
as the first processor (it needs to be the first to be effective).