From 0a43604737b11224b758d197b0446b8af5ed5175 Mon Sep 17 00:00:00 2001 From: James M Snell Date: Mon, 6 Jul 2020 15:38:42 -0700 Subject: [PATCH] doc: buffer documentation improvements PR-URL: https://github.com/nodejs/node/pull/34230 Reviewed-By: Anna Henningsen --- doc/api/buffer.md | 732 ++++++++++++++++++++++++++++++++++++---------- 1 file changed, 576 insertions(+), 156 deletions(-) diff --git a/doc/api/buffer.md b/doc/api/buffer.md index 87c75ac7831119..22ecfc867c5e5a 100644 --- a/doc/api/buffer.md +++ b/doc/api/buffer.md @@ -6,20 +6,12 @@ -In Node.js, `Buffer` objects are used to represent binary data in the form -of a sequence of bytes. Many Node.js APIs, for example streams and file system -operations, support `Buffer`s, as interactions with the operating system or -other processes generally always happen in terms of binary data. +`Buffer` objects are used to represent a fixed-length sequence of bytes. Many +Node.js APIs support `Buffer`s. -The `Buffer` class is a subclass of the [`Uint8Array`][] class that is built -into the JavaScript language. A number of additional methods are supported -that cover additional use cases. Node.js APIs accept plain [`Uint8Array`][]s -wherever `Buffer`s are supported as well. - -Instances of `Buffer`, and instances of [`Uint8Array`][] in general, -are similar to arrays of integers from `0` to `255`, but correspond to -fixed-sized blocks of memory and cannot contain any other values. -The size of a `Buffer` is established when it is created and cannot be changed. +The `Buffer` class is a subclass of JavaScript's [`Uint8Array`][] class and +extends it with methods that cover additional use cases. Node.js APIs accept +plain [`Uint8Array`][]s wherever `Buffer`s are supported as well. The `Buffer` class is within the global scope, making it unlikely that one would need to ever use `require('buffer').Buffer`. @@ -165,11 +157,10 @@ changes: description: The `Buffer`s class now inherits from `Uint8Array`. --> -`Buffer` instances are also [`Uint8Array`][] instances, which is the language’s -built-in class for working with binary data. [`Uint8Array`][] in turn is a -subclass of [`TypedArray`][]. Therefore, all [`TypedArray`][] methods are also -available on `Buffer`s. However, there are subtle incompatibilities between -the `Buffer` API and the [`TypedArray`][] API. +`Buffer` instances are also JavaScript [`Uint8Array`][] and [`TypedArray`][] +instances. All [`TypedArray`][] methods are available on `Buffer`s. There are, +however, subtle incompatibilities between the `Buffer` API and the +[`TypedArray`][] API. In particular: @@ -182,26 +173,37 @@ In particular: * [`buf.toString()`][] is incompatible with its `TypedArray` equivalent. * A number of methods, e.g. [`buf.indexOf()`][], support additional arguments. -There are two ways to create new [`TypedArray`][] instances from a `Buffer`. +There are two ways to create new [`TypedArray`][] instances from a `Buffer`: + +* Passing a `Buffer` to a [`TypedArray`][] constructor will copy the `Buffer`s + contents, interpreted an array array of integers, and not as a byte sequence + of the target type. + +```js +const buf = Buffer.from([1, 2, 3, 4]); +const uint32array = new Uint32Array(buf); + +console.log(uint32array); -When passing a `Buffer` to a [`TypedArray`][] constructor, the `Buffer`’s -elements will be copied, interpreted as an array of integers, and not as a byte -array of the target type. For example, -`new Uint32Array(Buffer.from([1, 2, 3, 4]))` creates a 4-element -[`Uint32Array`][] with elements `[1, 2, 3, 4]`, rather than a -[`Uint32Array`][] with a single element `[0x1020304]` or `[0x4030201]`. +// Prints: Uint32Array(4) [ 1, 2, 3, 4 ] +``` -In order to create a [`TypedArray`][] that shares its memory with the `Buffer`, -the underlying [`ArrayBuffer`][] can be passed to the [`TypedArray`][] -constructor instead: +* Passing the `Buffer`s underlying [`ArrayBuffer`][] will create a + [`TypedArray`][] that shares its memory with the `Buffer`. ```js const buf = Buffer.from('hello', 'utf16le'); const uint16arr = new Uint16Array( - buf.buffer, buf.byteOffset, buf.length / Uint16Array.BYTES_PER_ELEMENT); + buf.buffer, + buf.byteOffset, + buf.length / Uint16Array.BYTES_PER_ELEMENT); + +console.log(uint16array); + +// Prints: Uint16Array(5) [ 104, 101, 108, 108, 111 ] ``` -It is also possible to create a new `Buffer` that shares the same allocated +It is possible to create a new `Buffer` that shares the same allocated memory as a [`TypedArray`][] instance by using the `TypedArray` object’s `.buffer` property in the same way. [`Buffer.from()`][`Buffer.from(arrayBuf)`] behaves like `new Uint8Array()` in this context. @@ -214,6 +216,7 @@ arr[1] = 4000; // Copies the contents of `arr`. const buf1 = Buffer.from(arr); + // Shares memory with `arr`. const buf2 = Buffer.from(arr.buffer); @@ -773,7 +776,7 @@ range is between `0x00` and `0xFF` (hex) or `0` and `255` (decimal). This operator is inherited from `Uint8Array`, so its behavior on out-of-bounds access is the same as `Uint8Array`. In other words, `buf[index]` returns -`undefined` when `index` is negative or `>= buf.length`, and +`undefined` when `index` is negative or greater or equal to `buf.length`, and `buf[index] = value` does not modify the buffer if `index` is negative or `>= buf.length`. @@ -795,8 +798,8 @@ console.log(buf.toString('utf8')); ### `buf.buffer` -* {ArrayBuffer} The underlying `ArrayBuffer` object based on - which this `Buffer` object is created. +* {ArrayBuffer} The underlying `ArrayBuffer` object based on which this `Buffer` + object is created. This `ArrayBuffer` is not guaranteed to correspond exactly to the original `Buffer`. See the notes on `buf.byteOffset` for details. @@ -811,16 +814,15 @@ console.log(buffer.buffer === arrayBuffer); ### `buf.byteOffset` -* {integer} The `byteOffset` on the underlying `ArrayBuffer` object based on - which this `Buffer` object is created. +* {integer} The `byteOffset` of the `Buffer`s underlying `ArrayBuffer` object. When setting `byteOffset` in `Buffer.from(ArrayBuffer, byteOffset, length)`, -or sometimes when allocating a buffer smaller than `Buffer.poolSize`, the -buffer doesn't start from a zero offset on the underlying `ArrayBuffer`. +or sometimes when allocating a `Buffer` smaller than `Buffer.poolSize`, the +buffer does not start from a zero offset on the underlying `ArrayBuffer`. This can cause problems when accessing the underlying `ArrayBuffer` directly using `buf.buffer`, as other parts of the `ArrayBuffer` may be unrelated -to the `buf` object itself. +to the `Buffer` object itself. A common issue when creating a `TypedArray` object that shares its memory with a `Buffer` is that in this case one needs to specify the `byteOffset` correctly: @@ -1340,6 +1342,21 @@ deprecated: v8.0.0 The `buf.parent` property is a deprecated alias for `buf.buffer`. ### `buf.readBigInt64BE([offset])` + + +* `offset` {integer} Number of bytes to skip before starting to read. Must + satisfy: `0 <= offset <= buf.length - 8`. **Default:** `0`. +* Returns: {bigint} + +Reads a signed, big-endian 64-bit integer from `buf` at the specified `offset`. + +Integers read from a `Buffer` are interpreted as two's complement signed +values. + ### `buf.readBigInt64LE([offset])` + +* `offset` {integer} Number of bytes to skip before starting to read. Must + satisfy: `0 <= offset <= buf.length - 8`. **Default:** `0`. +* Returns: {bigint} + +Reads an unsigned, little-endian 64-bit integer from `buf` at the specified +`offset`. + +```js +const buf = Buffer.from([0x00, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff]); console.log(buf.readBigUInt64LE(0)); // Prints: 18446744069414584320n ``` ### `buf.readDoubleBE([offset])` -### `buf.readDoubleLE([offset])` + +* `offset` {integer} Number of bytes to skip before starting to read. Must + satisfy `0 <= offset <= buf.length - 8`. **Default:** `0`. +* Returns: {number} + +Reads a 64-bit, little-endian double from `buf` at the specified `offset`. + +```js +const buf = Buffer.from([1, 2, 3, 4, 5, 6, 7, 8]); + console.log(buf.readDoubleLE(0)); // Prints: 5.447603722011605e-270 console.log(buf.readDoubleLE(1)); @@ -1414,7 +1465,6 @@ console.log(buf.readDoubleLE(1)); ``` ### `buf.readFloatBE([offset])` -### `buf.readFloatLE([offset])` + +* `offset` {integer} Number of bytes to skip before starting to read. Must + satisfy `0 <= offset <= buf.length - 4`. **Default:** `0`. +* Returns: {number} + +Reads a 32-bit, little-endian float from `buf` at the specified `offset`. + +```js +const buf = Buffer.from([1, 2, 3, 4]); + console.log(buf.readFloatLE(0)); // Prints: 1.539989614439558e-36 console.log(buf.readFloatLE(1)); @@ -1473,7 +1542,6 @@ console.log(buf.readInt8(2)); ``` ### `buf.readInt16BE([offset])` -### `buf.readInt16LE([offset])` + +* `offset` {integer} Number of bytes to skip before starting to read. Must + satisfy `0 <= offset <= buf.length - 2`. **Default:** `0`. +* Returns: {integer} + +Reads a signed, little-endian 16-bit integer from `buf` at the specified +`offset`. + +Integers read from a `Buffer` are interpreted as two's complement signed values. + +```js +const buf = Buffer.from([0, 5]); + console.log(buf.readInt16LE(0)); // Prints: 1280 console.log(buf.readInt16LE(1)); @@ -1505,7 +1595,6 @@ console.log(buf.readInt16LE(1)); ``` ### `buf.readInt32BE([offset])` -### `buf.readInt32LE([offset])` + +* `offset` {integer} Number of bytes to skip before starting to read. Must + satisfy `0 <= offset <= buf.length - 4`. **Default:** `0`. +* Returns: {integer} + +Reads a signed, little-endian 32-bit integer from `buf` at the specified +`offset`. + +Integers read from a `Buffer` are interpreted as two's complement signed values. + +```js +const buf = Buffer.from([0, 0, 0, 5]); + console.log(buf.readInt32LE(0)); // Prints: 83886080 console.log(buf.readInt32LE(1)); @@ -1537,7 +1648,6 @@ console.log(buf.readInt32LE(1)); ``` ### `buf.readIntBE(offset, byteLength)` -### `buf.readIntLE(offset, byteLength)` + +* `offset` {integer} Number of bytes to skip before starting to read. Must + satisfy `0 <= offset <= buf.length - byteLength`. +* `byteLength` {integer} Number of bytes to read. Must satisfy + `0 < byteLength <= 6`. +* Returns: {integer} + +Reads `byteLength` number of bytes from `buf` at the specified `offset` +and interprets the result as a little-endian, two's complement signed value +supporting up to 48 bits of accuracy. + +```js +const buf = Buffer.from([0x12, 0x34, 0x56, 0x78, 0x90, 0xab]); + +console.log(buf.readIntLE(0, 6).toString(16)); +// Prints: -546f87a9cbee +``` + ### `buf.readUInt8([offset])` + +* `offset` {integer} Number of bytes to skip before starting to read. Must + satisfy `0 <= offset <= buf.length - 2`. **Default:** `0`. +* Returns: {integer} + +Reads an unsigned, little-endian 16-bit integer from `buf` at the specified +`offset`. + +```js +const buf = Buffer.from([0x12, 0x34, 0x56]); + +console.log(buf.readUInt16LE(0).toString(16)); +// Prints: 3412 console.log(buf.readUInt16LE(1).toString(16)); // Prints: 5634 console.log(buf.readUInt16LE(2).toString(16)); @@ -1632,7 +1787,6 @@ console.log(buf.readUInt16LE(2).toString(16)); ``` ### `buf.readUInt32BE([offset])` -### `buf.readUInt32LE([offset])` + +* `offset` {integer} Number of bytes to skip before starting to read. Must + satisfy `0 <= offset <= buf.length - 4`. **Default:** `0`. +* Returns: {integer} + +Reads an unsigned, little-endian 32-bit integer from `buf` at the specified +`offset`. + +```js +const buf = Buffer.from([0x12, 0x34, 0x56, 0x78]); + console.log(buf.readUInt32LE(0).toString(16)); // Prints: 78563412 console.log(buf.readUInt32LE(1).toString(16)); @@ -1662,7 +1837,6 @@ console.log(buf.readUInt32LE(1).toString(16)); ``` ### `buf.readUIntBE(offset, byteLength)` -### `buf.readUIntLE(offset, byteLength)` + +* `offset` {integer} Number of bytes to skip before starting to read. Must + satisfy `0 <= offset <= buf.length - byteLength`. +* `byteLength` {integer} Number of bytes to read. Must satisfy + `0 < byteLength <= 6`. +* Returns: {integer} + +Reads `byteLength` number of bytes from `buf` at the specified `offset` +and interprets the result as an unsigned, little-endian integer supporting +up to 48 bits of accuracy. + +```js +const buf = Buffer.from([0x12, 0x34, 0x56, 0x78, 0x90, 0xab]); + +console.log(buf.readUIntLE(0, 6).toString(16)); +// Prints: ab9078563412 +``` + ### `buf.subarray([start[, end]])` + +* `value` {bigint} Number to be written to `buf`. +* `offset` {integer} Number of bytes to skip before starting to write. Must + satisfy: `0 <= offset <= buf.length - 8`. **Default:** `0`. +* Returns: {integer} `offset` plus the number of bytes written. + +Writes `value` to `buf` at the specified `offset` as little-endian. + +`value` is interpreted and written as a two's complement signed integer. + +```js +const buf = Buffer.allocUnsafe(8); + +buf.writeBigInt64LE(0x0102030405060708n, 0); + +console.log(buf); +// Prints: +``` + ### `buf.writeBigUInt64BE(value[, offset])` + + +* `value` {bigint} Number to be written to `buf`. +* `offset` {integer} Number of bytes to skip before starting to write. Must + satisfy: `0 <= offset <= buf.length - 8`. **Default:** `0`. +* Returns: {integer} `offset` plus the number of bytes written. + +Writes `value` to `buf` at the specified `offset` as big-endian. + +```js +const buf = Buffer.allocUnsafe(8); + +buf.writeBigUInt64BE(0xdecafafecacefaden, 0); + +console.log(buf); +// Prints: +``` + ### `buf.writeBigUInt64LE(value[, offset])` + +* `value` {number} Number to be written to `buf`. +* `offset` {integer} Number of bytes to skip before starting to write. Must + satisfy `0 <= offset <= buf.length - 8`. **Default:** `0`. +* Returns: {integer} `offset` plus the number of bytes written. + +Writes `value` to `buf` at the specified `offset` as little-endian. The `value` +must be a JavaScript number. Behavior is undefined when `value` is anything +other than a JavaScript number. + +```js +const buf = Buffer.allocUnsafe(8); buf.writeDoubleLE(123.456, 0); @@ -2119,7 +2381,6 @@ console.log(buf); ``` ### `buf.writeFloatBE(value[, offset])` -### `buf.writeFloatLE(value[, offset])` + +* `value` {number} Number to be written to `buf`. +* `offset` {integer} Number of bytes to skip before starting to write. Must + satisfy `0 <= offset <= buf.length - 4`. **Default:** `0`. +* Returns: {integer} `offset` plus the number of bytes written. + +Writes `value` to `buf` at the specified `offset` as little-endian. Behavior is +undefined when `value` is anything other than a JavaScript number. + +```js +const buf = Buffer.allocUnsafe(4); + buf.writeFloatLE(0xcafebabe, 0); console.log(buf); @@ -2185,7 +2467,6 @@ console.log(buf); ``` ### `buf.writeInt16BE(value[, offset])` -### `buf.writeInt16LE(value[, offset])` + +* `value` {integer} Number to be written to `buf`. +* `offset` {integer} Number of bytes to skip before starting to write. Must + satisfy `0 <= offset <= buf.length - 2`. **Default:** `0`. +* Returns: {integer} `offset` plus the number of bytes written. + +Writes `value` to `buf` at the specified `offset` as little-endian. The `value` +must be a valid signed 16-bit integer. Behavior is undefined when `value` is +anything other than a signed 16-bit integer. + +The `value` is interpreted and written as a two's complement signed integer. + +```js +const buf = Buffer.allocUnsafe(2); + +buf.writeInt16LE(0x0304, 0); + +console.log(buf); +// Prints: ``` ### `buf.writeInt32BE(value[, offset])` -### `buf.writeInt32LE(value[, offset])` + +* `value` {integer} Number to be written to `buf`. +* `offset` {integer} Number of bytes to skip before starting to write. Must + satisfy `0 <= offset <= buf.length - 4`. **Default:** `0`. +* Returns: {integer} `offset` plus the number of bytes written. + +Writes `value` to `buf` at the specified `offset` as little-endian. The `value` +must be a valid signed 32-bit integer. Behavior is undefined when `value` is +anything other than a signed 32-bit integer. + +The `value` is interpreted and written as a two's complement signed integer. + +```js +const buf = Buffer.allocUnsafe(4); + +buf.writeInt32LE(0x05060708, 0); + +console.log(buf); +// Prints: ``` ### `buf.writeIntBE(value, offset, byteLength)` -### `buf.writeIntLE(value, offset, byteLength)` + +* `value` {integer} Number to be written to `buf`. +* `offset` {integer} Number of bytes to skip before starting to write. Must + satisfy `0 <= offset <= buf.length - byteLength`. +* `byteLength` {integer} Number of bytes to write. Must satisfy + `0 < byteLength <= 6`. +* Returns: {integer} `offset` plus the number of bytes written. + +Writes `byteLength` bytes of `value` to `buf` at the specified `offset` +as little-endian. Supports up to 48 bits of accuracy. Behavior is undefined +when `value` is anything other than a signed integer. + +```js +const buf = Buffer.allocUnsafe(6); + buf.writeIntLE(0x1234567890ab, 0, 6); console.log(buf); @@ -2318,7 +2679,6 @@ console.log(buf); ``` ### `buf.writeUInt16BE(value[, offset])` -### `buf.writeUInt16LE(value[, offset])` + +* `value` {integer} Number to be written to `buf`. +* `offset` {integer} Number of bytes to skip before starting to write. Must + satisfy `0 <= offset <= buf.length - 2`. **Default:** `0`. +* Returns: {integer} `offset` plus the number of bytes written. + +Writes `value` to `buf` at the specified `offset` as little-endian. The `value` +must be a valid unsigned 16-bit integer. Behavior is undefined when `value` is +anything other than an unsigned 16-bit integer. + +```js +const buf = Buffer.allocUnsafe(4); buf.writeUInt16LE(0xdead, 0); buf.writeUInt16LE(0xbeef, 2); @@ -2355,7 +2737,6 @@ console.log(buf); ``` ### `buf.writeUInt32BE(value[, offset])` -### `buf.writeUInt32LE(value[, offset])` + +* `value` {integer} Number to be written to `buf`. +* `offset` {integer} Number of bytes to skip before starting to write. Must + satisfy `0 <= offset <= buf.length - 4`. **Default:** `0`. +* Returns: {integer} `offset` plus the number of bytes written. + +Writes `value` to `buf` at the specified `offset` as little-endian. The `value` +must be a valid unsigned 32-bit integer. Behavior is undefined when `value` is +anything other than an unsigned 32-bit integer. + +```js +const buf = Buffer.allocUnsafe(4); buf.writeUInt32LE(0xfeedface, 0); @@ -2390,7 +2793,6 @@ console.log(buf); ``` ### `buf.writeUIntBE(value, offset, byteLength)` -### `buf.writeUIntLE(value, offset, byteLength)` + +* `value` {integer} Number to be written to `buf`. +* `offset` {integer} Number of bytes to skip before starting to write. Must + satisfy `0 <= offset <= buf.length - byteLength`. +* `byteLength` {integer} Number of bytes to write. Must satisfy + `0 < byteLength <= 6`. +* Returns: {integer} `offset` plus the number of bytes written. + +Writes `byteLength` bytes of `value` to `buf` at the specified `offset` +as little-endian. Supports up to 48 bits of accuracy. Behavior is undefined +when `value` is anything other than an unsigned integer. + +```js +const buf = Buffer.allocUnsafe(6); buf.writeUIntLE(0x1234567890ab, 0, 6); @@ -2555,7 +2982,13 @@ changes: See [`Buffer.from(string[, encoding])`][`Buffer.from(string)`]. -## `buffer.INSPECT_MAX_BYTES` +## `buffer` module APIs + +While, the `Buffer` object is available as a global, there are additional +`Buffer`-related APIs that are available only via the `buffer` module +accessed using `require('buffer')`. + +### `buffer.INSPECT_MAX_BYTES` @@ -2566,10 +2999,7 @@ Returns the maximum number of bytes that will be returned when `buf.inspect()` is called. This can be overridden by user modules. See [`util.inspect()`][] for more details on `buf.inspect()` behavior. -This is a property on the `buffer` module returned by -`require('buffer')`, not on the `Buffer` global or a `Buffer` instance. - -## `buffer.kMaxLength` +### `buffer.kMaxLength` @@ -2578,10 +3008,7 @@ added: v3.0.0 An alias for [`buffer.constants.MAX_LENGTH`][]. -This is a property on the `buffer` module returned by -`require('buffer')`, not on the `Buffer` global or a `Buffer` instance. - -## `buffer.transcode(source, fromEnc, toEnc)` +### `buffer.transcode(source, fromEnc, toEnc)` @@ -2632,7 +3056,7 @@ See [`Buffer.allocUnsafeSlow()`][]. This was never a class in the sense that the constructor always returned a `Buffer` instance, rather than a `SlowBuffer` instance. -### `new SlowBuffer(size)` +#### `new SlowBuffer(size)` @@ -2643,15 +3067,12 @@ deprecated: v6.0.0 See [`Buffer.allocUnsafeSlow()`][]. -## Buffer constants +### Buffer constants -`buffer.constants` is a property on the `buffer` module returned by -`require('buffer')`, not on the `Buffer` global or a `Buffer` instance. - -### `buffer.constants.MAX_LENGTH` +#### `buffer.constants.MAX_LENGTH` @@ -2663,7 +3084,7 @@ On 64-bit architectures, this value currently is `(2^31)-1` (~2GB). This value is also available as [`buffer.kMaxLength`][]. -### `buffer.constants.MAX_STRING_LENGTH` +#### `buffer.constants.MAX_STRING_LENGTH` @@ -2807,7 +3228,6 @@ introducing security vulnerabilities into an application. [`TypedArray#slice()`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray/slice [`TypedArray#subarray()`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray/subarray [`TypedArray`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray -[`Uint32Array`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint32Array [`Uint8Array`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array [`buf.buffer`]: #buffer_buf_buffer [`buf.compare()`]: #buffer_buf_compare_target_targetstart_targetend_sourcestart_sourceend