-
Notifications
You must be signed in to change notification settings - Fork 30k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Stream finished does not always work with http incoming message #38657
Comments
This works correctly for me on Node 16.0+, but not on Node 15.0+. |
Add a test to verify that stream.finished works correctly on IncomingMessage refs: nodejs#38657
Changing let http = require("http");
let { finished } = require("stream");
let server = http.createServer(async function(req, res)
{
req.destroy()
console.log("waiting");
await new Promise(res => setTimeout(res,1));
await new Promise((res) => finished(req, res));
console.log("sending");
res.end();
});
(async function()
{
await new Promise(resolve => server.listen(resolve));
let req = http.request({ port: server.address().port, method: "post" }).end("abc");
try
{
let res = await new Promise((resolve, reject) => req.on("response", resolve).on("error", reject));
await finished(res.resume());
}
catch(e)
{
console.log(e);
}
}()); |
Seems on nodejs 16 it is working correctly. |
cc @nodejs/http @nodejs/streams |
FYI: Node v15 is going out of maintenance in a bit more than a month, it is unlikely to receive a fix for this. A few notes:
@dnlup @ronag do you think it could be possible to fix this in v12 and v14 in a non-breaking way? |
Wait, what? Can you point to some elaboration on this, please? |
Instead of doing: emitter.once('foo', async () => {
// no one is awaiting this, some stuff has best effort cleanup
await someOtherStuffOrCleanup();
}); vs. await once(emitter, 'foo');
await someOtherStuffOrCleanup(); // now everything is listened for and is in sequence. |
@benjamingr I understand your point about orphaning a Promise, but I'm missing how that relates to the examples leading up to @mcollina 's comment. I surmise that the "event handler or callback" he was referring to is the
What am I not getting? |
If you throw inside the async function it will become an unhandled exception. |
I could create an async generator from |
I disagree. Throwing in an async function is obviously catchable. Maybe you mean "throwing in an event handler will be unhandled"? But then, that is true whether it is sync or async. |
I mean literally: do not mix promises and callbacks. This is a good talk on the topic from @jasnell: https://youtu.be/XV-u_Ow47s0. I would recommend you to use Koa, Hapi or Fastify (and others), as they provide a higher level API that is promise-friendly. There are a few tricky bits in using the low-level APIs with async await that you do not want to deal with. The code you are proving is something that I would not recommend anybody to write. You are deliberately destroying something and then checking for it to be destroyed. Note that when you destroy the request you also destroy the response - there is no reason to continue. |
For the purpose of this example it is ok when it aborts and the program terminates in case
Yes it is a minimal test case. There are situations when in some part of code one does not know whether the incoming message is already destroyed or not. There is just need to wait until it is finished. If it was already destroyed then |
I am not sure yet, I think the problem could be how the |
This is exactly what I recommend not to do. Don't do anything after you have sent a response to the user (minus observability requirements/logging etc). |
Add a test to verify that stream.finished works correctly on IncomingMessage refs: #38657 PR-URL: #38661 Refs: #38657 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Juan José Arboleda <soyjuanarbol@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
I think this is rather a question of practicality - maybe there are not many practical things which one could do after a response is sent but I do not see any reason why one could not do anything he/she wants. For example there could be an api call where the user sends some data and the server stores it in db and responds "ok" then starts some long lasting background job which does some processing of this data and there can be another api call which queries whether this job is done and retrieves results. Why would be this a bad idea? |
Having some background processing is ok. Having a race between two code paths to send an HTTP response is an anti-pattern and the main source of your problem. When moving the processing on the background, no reference to the req/res should be kept around. |
Add a test to verify that stream.finished works correctly on IncomingMessage refs: #38657 PR-URL: #38661 Refs: #38657 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Juan José Arboleda <soyjuanarbol@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
It might be worth noting that the apollo-server-hapi plugin exhibits this behavior in node 16. I'm not sure the cause but I do know that I was seeing readable streams(http requests) get destroyed during the execution of the hapi middleware chain. hapi has a listener on close which eventually just sets the request context to null. Not that this is a viable workaround but if I commented out this event handler in hapi core, it no longer crashed the hapi server. You can see the issue if you run this code: https://github.com/bizob2828/apollo-hapi-nr-test and make this curl request:
|
No I did not mean anything like that. It would not send another response after processing is done. I mentioned another different api call for retrieving status and result (if finished) of that background processing.
Of course. |
So this is probably no longer true in nodejs 16: Lines 184 to 189 in d798de1
|
What steps will reproduce the bug?
The
stream.finished
never resolves or rejects when applied onto a destroyed incoming message like in example below. It finishes for example when applied on a destroyed file stream. Also it finishes when the line withawait new Promise(r => setTimeout(r, 1000));
is commented. This looks really inconsistent.How often does it reproduce? Is there a required condition?
Always.
What is the expected behavior?
What do you see instead?
Additional information
The text was updated successfully, but these errors were encountered: