Set Netty's maxOrder options to previous default #15925
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ff the user hans't specified anything in his option (both file or LS_JVM_OPTS environment variable)
… to startup Logstash
|
0.0% Coverage on New Code
💚 Build Succeeded
History
cc @andsel |
|
Note for reviewer: the SonarCube report is failing because no tests are run on the |
Please open an issue about this, we should fix it, the jvm options parser is self contained enough that running these tests should be done at the PR level |
|
#15927 to handle the testing on PR CI |
|
@logstashmachine backport 8.12 |
github-actions bot
pushed a commit
that referenced
this pull request
Feb 9, 2024
Updates Netty's configuration of maxOrder to a previously proven value, if not already customised by the user. Adds a step to the JvmOption parsing tool, which is used to compose the JVM options string to pass down to Logstash at startup. The added step rework the parsed options to set the allocator max order -Dio.netty.allocator.maxOrder=11 so that the maximum pooled buffer is up to 16MB and not 4MB. This option is added iff it's not yet specified by the user (cherry picked from commit 52ce3ff)
3 tasks
|
@logstashmachine backport 7.17 |
andsel
added a commit
that referenced
this pull request
Feb 9, 2024
…default (#15928) Updates Netty's configuration of maxOrder to a previously proven value, if not already customised by the user. Adds a step to the JvmOption parsing tool, which is used to compose the JVM options string to pass down to Logstash at startup. The added step rework the parsed options to set the allocator max order -Dio.netty.allocator.maxOrder=11 so that the maximum pooled buffer is up to 16MB and not 4MB. This option is added iff it's not yet specified by the user (cherry picked from commit 52ce3ff) Co-authored-by: Andrea Selva <selva.andre@gmail.com>
This was referenced Dec 9, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Release notes
Updates Netty's configuration of maxOrder to a previously proven value, if not already customised by the user.
What does this PR do?
Adds a step to the JvmOption parsing tool, which is used to compose the JVM options string to pass down to Logstash at startup.
The added step rework the parsed options to set the allocator max order
-Dio.netty.allocator.maxOrder=11so that the maximum pooled buffer is up to 16MB and not 4MB.This option is added iff it's not yet specified by the user
Why is it important/What is the impact to the user?
It brings back the performance of Netty based plugins to the same as of Logstash previous of
8.7.0, which introduced an update of Netty library.With messages bigger then 2Kb each it triggered a problem of using unpooled buffers, which kills performance. With this change it moved back to 8Kb.
Checklist
[ ] I have made corresponding changes to the documentation[ ] I have made corresponding change to the default configuration files (and/or docker env variables)Author's Checklist
How to test this PR locally
The test splits in 2, verify that the line
-Dio.netty.allocator.maxOrder=11is added when the flag is not inconfig/jvm.optionsand verify that if the flag is already present it's not overwritten.Run Logstash with
bin/logstash -e "input {stdin {}} output {stdout{}}"-second test: edit
config/jvm.optionsand add a customio.netty.allocator.maxOrder, start Logstash and verify logs contains the definition was added.Related issues
Use cases
Screenshots
Logs