stdout not flushed upon exit with output redirect

[nzakas]

Using Node.js v0.4.10 on Ubuntu.

If I write a large amount of data to stdout, either via console.log() or process.stdout.write(), and then try to redirect the output to a file, the file remains empty even though the output never appears in the console. I'm not forcing the process to exit, it's completing its task and then exiting on its own. I expected stdout to flush at this point, but it appears not to.

This behavior can be seen with CSS Lint v0.5.0

npm install csslint@0.5.0
csslint some_css_file.css > results.txt

I tried working around this by using:

fs.writeSync(1, message);

However, this is causing other errors.

Is this behavior expected? If so, how can I ensure stdout is flushed so that output can be captured as usual?

[isaacs]

You can force it to flush like this:

var flushed = process.stdout.flush()
if (!flushed) process.once("drain", function () {
  process.exit(0)
})

However, pending data on the stdout socket should prevent an exit, so this is a bug.

[nzakas]

Thanks, I'll use that as a workaround.

[nzakas]

Looks like flush() was removed in the latest Node. Is there another way to force the flush now?

[sh1mmer]

Flush was unofficial/undocumented, sorry.

Nicholas would you be able to provide an atomic testcase of your large data case to reproduce this bug. Then we can get a patch in.

Thanks

[nzakas]

Hey, travelling at the moment so don't have time to whip a standalone demo up. If you try installing csslint@0.5.0 and run it on the demo file (https://github.com/stubbornella/csslint/blob/master/demos/demo.css), then try to capture the output, you'll end up with an empty file. CSS Lint also has tagged versions in source control, so you could pull the v0.5.0 tag from GitHub and get the source locally as well.

[gcp]

Flush was unofficial/undocumented, sorry.

That may be, but as nzakas already asked, is there any way to control flushing now? stdout/stderr were made unbuffered for TTY's and files, but not for pipes. I have an application that uses pipes and it's surprising that node 0.6.x is now suddenly a regression for that use case, because there's no way to ensure data is actually flushed as soon as possible.

Using writeSync seems to have the wanted behavior, but there is nothing in the documentation saying this will ensure a stdout flush (IIRC standard Unix behavior is to flush when there's a newline for text mode, or something?), so I'd appreciate a definite word on this.

[ichernev]

Here is a simple test to see if its (not) working. For me (node v0.4.12) its not working correctly.

https://gist.github.com/1357983

[kripken]

I hit this issue too, and can reproduce it with ichernev's test case from the last comment on node 0.5.5pre. When output is redirected, nothing is written, but when it is not redirected, things are printed out properly.

[cowboy]

FWIW, I've worked around this in my own code with the super-hacky:

process.stdout = process.stderr;

But since I'm developing a CLI tool that might have its output piped, this is less than ideal.

+1 for a fix!

[cowboy]

Is this still an issue? I just tried @ichernev's example in v0.6.9 and don't see any issues.

[ichernev]

Yep, v0.6.8 works for me too.

[mpareja]

This is still an issue in v0.8.12 for Windows 7 64-bit.

I'm working around this by exiting on the 'drain' event and writing empty string to the stream. I write the empty string to the stream in case there is nothing more to 'drain' by the time I'm registering for the event:

process.stdout.once('drain', function () { process.exit(0); });
process.stdout.write('');

[bnoordhuis]

@mpareja Can you open a new issue for that? Please include a test case. Thanks.

[ghost]

Same error with mocha called from webstorm console. I thought the problem is with webstorm because from git bash I got the whole error stack in the console. Strange behavior... :S
Here is how to reproduce it:
http://yieldthedog.github.com/blog/2012/03/12/configure-webstorm-to-run-and-debug-mocha-tests/
In webstorm console with stdout always the first line appears, with stderr about 65% the first line appears, about 35% the whole err.stack with a custom reporter.

I got this in webstorm console:
"Process finished with exit code 59"
which means
"An unexpected network error occurred."

Hmm it depends only on how long I have to wait for the response. So I think it could be a timeout problem, not necessarily this bug.

[matthiasg]

this is a real problem. mocha uses console.log to log errors. but error stack traces on windows are reproducible not redirected to a file or other process while the actual errors are.

node.js should make sure console.log output is flushed before the process is exited.

[bnoordhuis]

Added a 'windows' label. This shouldn't be an issue any longer on Unices.

[matthiasg]

should the ticket not be reopened then ?

[mpareja]

I should have created a new issue. Sorry folks, life's been crazy and I haven't done this. I'll open a ticket with a repro sample at some point soon and reference it from this issue.

[matthiasg]

is this reason for gruntjs/grunt#504

[matthiasg]

ah the issue is #3584