stream, doc: an ambiguity in the readable.read([size]) doc

[vsemozhetbyt]

• Version: 6.2.2
• Platform: Windows 7 x64
• Subsystem: stream, doc

The readable.read() method pulls some data out of the internal buffer and returns it. If no data available to be read, null is returned. By default, the data will be returned as a Buffer object unless an encoding has been specified using the readable.setEncoding() method or the stream is operating in object mode.

The optional size argument specifies a specific number of bytes to read. If size bytes are not available to be read, null will be returned unless the stream has ended, in which case all of the data remaining in the internal buffer will be returned (even if it exceeds size bytes).

The first paragraph makes distinction between buffer and string mode, but in the second paragraph only bytes are mentioned. It could seem that even in the string mode the size argument means bytes. However, in the string mode the size argument means decoded characters.

Save this script in the UTF-8 with BOM and run it:

/******************************************************************************/
'use strict';
/******************************************************************************/
const fs = require('fs');

const readStream = fs.createReadStream(__filename, { encoding: 'utf8' });

readStream.once('readable', () => {
  console.log(readStream.read(3));
});

The output:

 /*

If in the string mode the size argument meant bytes, the output would be just the BOM.

[nouex]

Related: #6798

[vsemozhetbyt]

By the way, is this another lacuna about readable stream? Should I post it as an issue here or do I just not understand the doc well?

[joaocgreis]

cc @nodejs/streams

[mcollina]

I think it's s duplicate of #6798.

What is BOM?

[addaleax]

What is BOM?

The byte-order mark, aka U+FEFF. It’s primarily used at the start of text for indicating the endianness of a UTF16/UTF32/etc-encoded text. Although it’s discouraged, it can also be used at the start of UTF-8 text (encoded as ef bb bf, so 3 bytes).

[jasnell]

ping @nodejs/streams

[calvinmetcalf]

sounds like we need to update the docs to specify that it's in encoding dependent units

[mcollina]

Closing as a duplicate of #6798.