Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NETWORKING: Fix Netty Leaks by upgrading to 4.1.28 #32511

Merged
merged 1 commit into from
Aug 1, 2018

Conversation

original-brownbear
Copy link
Member

@original-brownbear original-brownbear commented Jul 31, 2018

@original-brownbear original-brownbear added :Distributed Coordination/Network Http and internode communication implementations >test-failure Triaged test failures from CI v7.0.0 v6.5.0 labels Jul 31, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra

* Upgrade to `4.1.28` since the problem reported in elastic#32487 is a bug in Netty itself (see netty/netty#7337)
* Fixed other leaks in test code that now showed up due to fixes improvements in leak reporting in the newer version
* Needed to extend permissions for netty common package because it now sets a classloader at runtime after changes in netty/netty@63bae09
* Adjusted forbidden APIs check accordingly
* Closes elastic#32487
@jasontedor jasontedor added >upgrade and removed >test-failure Triaged test failures from CI labels Jul 31, 2018
Copy link
Member

@jasontedor jasontedor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. If CI is happy, I am happy. This is so much better than the circus we had in #31232.

@original-brownbear
Copy link
Member Author

Jenkins test this
(CI was green, packaging test didn't trigger for some reason)

Copy link
Contributor

@Tim-Brooks Tim-Brooks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

The code does not really bother me since it is in unit tests. But it looks like netty does have a facility for releasing RCs in testing without try/finally.

http://netty.io/wiki/reference-counted-objects.html

public void testSomething() throws Exception {
    // ReferenceCountUtil.releaseLater() will keep the reference of buf,
    // and then release it when the test thread is terminated.
    ByteBuf buf = ReferenceCountUtil.releaseLater(Unpooled.directBuffer(512));
    ...
}

I don't know how effective that is. But something to maybe look at in future fixes.

@original-brownbear
Copy link
Member Author

@tbrooks8 thanks!

I'm just fixing those tests, so the leak detector doesn't go red :) io.netty.util.ReferenceCountUtil#releaseLater(T) is deprecated (and in my experience/opinion rightfully so). It's just taking too much memory potentially because buffers are kept around for the whole duration of the test run which can really add up in unexpected ways.

@original-brownbear original-brownbear merged commit 4b199dd into elastic:master Aug 1, 2018
@original-brownbear original-brownbear deleted the 32478 branch August 1, 2018 00:35
dnhatn added a commit that referenced this pull request Aug 3, 2018
* master:
  HLRC: Move commercial clients from XPackClient (#32596)
  Add cluster UUID to Cluster Stats API response (#32206)
  Security: move User to protocol project (#32367)
  [TEST] Test for shard failures, add debug to testProfileMatchesRegular
  Minor fix for javadoc (applicable for java 11). (#32573)
  Painless: Move Some Lookup Logic to PainlessLookup (#32565)
  TEST: Avoid merges in testSeqNoAndCheckpoints
  [Rollup] Remove builders from HistoGroupConfig (#32533)
  Mutes failing SQL string function tests due to #32589
  fixed elements in array of produced terms (#32519)
  INGEST: Enable default pipelines (#32286)
  Remove cluster state initial customs (#32501)
  Mutes LicensingDocumentationIT due to #32580
  [ML] Remove multiple_bucket_spans (#32496)
  [ML] Rename JobProvider to JobResultsProvider (#32551)
  Correct minor typo in explain.asciidoc for HLRC
  Build: Add elastic maven to repos used by BuildPlugin (#32549)
  Clarify the error message when a pipeline agg is used in the 'order' parameter. (#32522)
  Revert "[test] turn on host io cache for opensuse (#32053)"
  Enable packaging tests on suse boxes
  [ML] Improve error when no available field exists for rule scope (#32550)
  [ML] Improve error for functions with limited rule condition support (#32548)
  Painless: Clean Up PainlessField (#32525)
  Add @AwaitsFix for #32554
  Remove broken @link in Javadoc
  Scripting: Conditionally use java time api in scripting (#31441)
  [ML] Fix thread leak when waiting for job flush (#32196) (#32541)
  Add AwaitsFix to failing test - see #32546
  Core: Minor size reduction for AbstractComponent (#32509)
  SQL: Added support for string manipulating functions with more than one parameter (#32356)
  [DOCS] Reloadable Secure Settings (#31713)
  Watcher: Reenable HttpSecretsIntegrationTests#testWebhookAction test (#32456)
  [Rollup] Remove builders from TermsGroupConfig (#32507)
  Use hostname instead of IP with SPNEGO test (#32514)
  Switch x-pack rolling restart to new style Requests (#32339)
  NETWORKING: Fix Netty Leaks by upgrading to 4.1.28 (#32511)
  [DOCS] Small fixes in rule configuration page (#32516)
  Painless: Clean up PainlessMethod (#32476)
  Build: Remove shadowing from benchmarks (#32475)
  Docs: Add all JDKs to CONTRIBUTING.md
  Add licensing enforcement for FIPS mode (#32437)
  SQL: Add test for handling of partial results (#32474)
  Mute testFilterCacheStats
  [ML][DOCS] Fix typo applied_to => applies_to
  Scripting: Fix painless compiler loader to know about context classes (#32385)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Coordination/Network Http and internode communication implementations >upgrade v7.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CI] SslIntegrationTests fail with Netty buffer leak
5 participants