-
Notifications
You must be signed in to change notification settings - Fork 30k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
investigate flaky test-http2-large-writes-session-memory-leak on Windows CI #31089
Comments
test-rackspace-win2012r2_vs2017-x64-3 21:55:20 not ok 298 parallel/test-http2-large-writes-session-memory-leak
21:55:20 ---
21:55:20 duration_ms: 0.401
21:55:20 severity: fail
21:55:20 exitcode: 1
21:55:20 stack: |-
21:55:20 events.js:298
21:55:20 throw er; // Unhandled 'error' event
21:55:20 ^
21:55:20
21:55:20 Error [ERR_HTTP2_STREAM_ERROR]: Stream closed with error code NGHTTP2_ENHANCE_YOUR_CALM
21:55:20 at ClientHttp2Stream._destroy (internal/http2/core.js:2160:13)
21:55:20 at ClientHttp2Stream.destroy (internal/streams/destroy.js:41:8)
21:55:20 at Http2Stream.onStreamClose (internal/http2/core.js:506:12)
21:55:20 Emitted 'error' event on ClientHttp2Stream instance at:
21:55:20 at emitErrorNT (internal/streams/destroy.js:96:8)
21:55:20 at emitErrorCloseNT (internal/streams/destroy.js:68:3)
21:55:20 at processTicksAndRejections (internal/process/task_queues.js:84:21) {
21:55:20 code: 'ERR_HTTP2_STREAM_ERROR'
21:55:20 }
21:55:20 ... |
Didn't fail on my local Windows vs2017 machine with |
Well, stress CI is green, perhaps the machine should be under heavy load (from other tests) for this to actually fail. Anyway, this doesn't seem to fail often (at all anymore?) so closing this for now. |
test-rackspace-win2012r2_vs2017-x64-4
|
Should the test gracefully handle |
Given what the test is doing, this is most likely hitting up against the max session memory limit. If the test has not accounted for that already, perhaps try increasing that limit significantly and seeing if the failure continues |
Hmmm, this comment in the test suggests that the failures may indicate a memory leak.
|
https://ci.nodejs.org/job/node-test-binary-windows-js-suites/4749/RUN_SUBSET=2,nodes=win2012r2-COMPILED_BY-vs2019-x86/console failed too. test-rackspace-win2012r2_vs2017-x64-4 As with the other one I posted earlier and all the ones before that, it's a rackspace host. I haven't seen this fail on an azure host yet. If I recall correctly, the rackspace hosts are configured with different memory and CPU than the azure hosts. |
I have been unable to duplicate this in the node-stress-single-test, running things in parallel, etc. This is frustrating.... |
The test sets maxSessionMemory to 1 Mb:
|
Add some debugging output for flaky http2 test to try to figure out what's going on the next time it fails in CI. Refs: nodejs#31089
I've figured out how to replicate this in CI. I've applied 5a8e165 which makes changes to Here's the master branch with that patch applied showing many failures: https://ci.nodejs.org/job/node-test-commit-windows-fanned/37176/ Will apply to the first branch that introduced this test and see if it was failing from the beginning. If not, I'll bisect to find the commit that introduced the failures. |
Argh, I can't get 18a1796 to compile in CI so I can't test it. Here's the result: https://ci.nodejs.org/view/All/job/node-compile-windows/34942/ Any chance someone knowledgable from @nodejs/build and/or @nodejs/platform-windows can take a look and maybe make a suggestion for getting it to compile? I guess something changed with how we build in CI at some point? I suspect the issue is this, but I'm also pretty ignorant about building on Windows:
|
Turns out I don't need to compile that commit because a later commit (415bba) built just fine and stress test was good, so I'm going to bisect and find the bad commit now.... |
Bisected! The problematic commit is 51ccf1b. |
This reverts commit 51ccf1b. Fixes: nodejs#31089
This reverts commit 51ccf1b. Fixes: nodejs#31089 PR-URL: nodejs#34315 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: David Carlier <devnexen@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Denys Otrishko <shishugi@gmail.com>
https://ci.nodejs.org/job/node-test-binary-windows-js-suites/634/RUN_SUBSET=3,nodes=win2012r2-COMPILED_BY-vs2019-x86/console
test-rackspace-win2012r2_vs2019-x64-2
@nodejs/http2 @nodejs/platform-windows
The text was updated successfully, but these errors were encountered: