ERR_BUFFER_OUT_OF_BOUNDS error with Google Vision package on Node 22.7.0

[Ecostack]

Version

22.7.0

Platform

Darwin Sebastian-MacBook-Pro.local 23.5.0 Darwin Kernel Version 23.5.0: Wed May  1 20:12:58 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6000 arm64

Subsystem

No response

What steps will reproduce the bug?

Create a file called test.js with the following content.
Add the @google-cloud/vision package to your package.json and run npm install.

const vision = require('@google-cloud/vision');

// Set up your Google Cloud Vision client
const client = new vision.ImageAnnotatorClient({
    keyFilename: 'XXX.json'
});

Start via node test.js

How often does it reproduce? Is there a required condition?

On each NodeJS start

What is the expected behavior? Why is that the expected behavior?

It should not crash.

The same code starts with version 22.5.0, here is the output:

node test.js                                                                                                                                                                       
(node:34273) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)

What do you see instead?

node:internal/buffer:1066
      throw new ERR_BUFFER_OUT_OF_BOUNDS('length');
      ^

RangeError [ERR_BUFFER_OUT_OF_BOUNDS]: "length" is outside of buffer bounds
    at proto.utf8Write (node:internal/buffer:1066:13)
    at Op.writeStringBuffer [as fn] (/Users/sebastianscheibe/Code/XXX/node_modules/protobufjs/src/writer_buffer.js:61:13)
    at BufferWriter.finish (/Users/sebastianscheibe/Code/XXX/node_modules/protobufjs/src/writer.js:453:14)
    at /Users/sebastianscheibe/Code/XXX/node_modules/@grpc/proto-loader/build/src/index.js:177:109
    at Array.map (<anonymous>)
    at createPackageDefinition (/Users/sebastianscheibe/Code/XXX/node_modules/@grpc/proto-loader/build/src/index.js:177:39)
    at Object.fromJSON (/Users/sebastianscheibe/Code/XXX/node_modules/@grpc/proto-loader/build/src/index.js:230:12)
    at GrpcClient.loadProtoJSON (/Users/sebastianscheibe/Code/XXX/node_modules/google-gax/build/src/grpc.js:228:51)
    at new ImageAnnotatorClient (/Users/sebastianscheibe/Code/XXX/node_modules/@google-cloud/vision/build/src/v1/image_annotator_client.js:148:38)
    at Object.<anonymous> (/Users/sebastianscheibe/Code/XXX/test.js:5:16) {
  code: 'ERR_BUFFER_OUT_OF_BOUNDS'
}

Node.js v22.7.0

Additional information

No response

[RedYetiDev]

Hi! Can you make a reproduction without using any dependencies?

[Ecostack]

Here is an example using purely Buffer.

let example = new Buffer("1234567890")
example.utf8Write("abc", 1)

Using 22.7.0

node:internal/buffer:1066
      throw new ERR_BUFFER_OUT_OF_BOUNDS('length');
      ^

RangeError [ERR_BUFFER_OUT_OF_BOUNDS]: "length" is outside of buffer bounds
    at proto.utf8Write (node:internal/buffer:1066:13)
    at Object.<anonymous> (/Users/sebastianscheibe/Code/XXX/test_github.js:4:9)
    at Module._compile (node:internal/modules/cjs/loader:1546:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1691:10)
    at Module.load (node:internal/modules/cjs/loader:1317:32)
    at Module._load (node:internal/modules/cjs/loader:1127:12)
    at TracingChannel.traceSync (node:diagnostics_channel:315:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:217:24)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:166:5)
    at node:internal/main/run_main_module:30:49 {
  code: 'ERR_BUFFER_OUT_OF_BOUNDS'
}

Node.js v22.7.0

Using 22.5

 node test_github.js                                                                                                                                                                   
(node:45481) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `node --trace-deprecation ...` to show where the warning was created)

[Ecostack]

There seems to be a change in the latest version which introduces this check regarding the offset.

22.5:

node/lib/internal/buffer.js

Line 1045 in fd9233a

proto.utf8Write = utf8Write;

22.7:

node/lib/internal/buffer.js

Line 1065 in 65eff1e

if (length < 0 || length > this.byteLength - offset) {

[RedYetiDev]

Thanks! @nodejs/buffer PTAL

[ericrange]

i dont know if my issue has something to do with this, but since 22.7 the encoding of buffers wont work anymore.

MongoError: text contains invalid UTF-8

even expressjs delivers garbage.

yesterday, i build my app on node:22-alpine and all works fine.
since: https://hub.docker.com/layers/library/node/22-alpine/images/sha256-3dbc5d17cf89e1d8ae6f4a8a562f8a5b5df52b4c7060bfb359de86de6a3ecc3c

all is broken. nothing changed in the code base!

btw. "node:20-alpine" works as expected (like node:22-alpine yesterday)

[RedYetiDev]

I was able to reproduce using both your snippet, and the slightly modified one below:

Buffer.from("abc").utf8Write("abc", 1)

$ node repro.js
node:internal/buffer:1066
      throw new ERR_BUFFER_OUT_OF_BOUNDS('length');
      ^

RangeError [ERR_BUFFER_OUT_OF_BOUNDS]: "length" is outside of buffer bounds
    at proto.utf8Write (node:internal/buffer:1066:13)
    at Object.<anonymous> (/repro.js:1:20)
    at Module._compile (node:internal/modules/cjs/loader:1546:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1691:10)
    at Module.load (node:internal/modules/cjs/loader:1317:32)
    at Module._load (node:internal/modules/cjs/loader:1127:12)
    at TracingChannel.traceSync (node:diagnostics_channel:315:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:217:24)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:166:5)
    at node:internal/main/run_main_module:30:49 {
  code: 'ERR_BUFFER_OUT_OF_BOUNDS'
}

Node.js v22.7.0

[andreialecu]

Leaving this here as it may help someone, this also happens with OpenTelemetry:

Error: PeriodicExportingMetricReader: metrics export failed (error RangeError [ERR_BUFFER_OUT_OF_BOUNDS]: "length" is outside of buffer bounds)
at doExport (/app/node_modules/@opentelemetry/sdk-metrics/src/export/PeriodicExportingMetricReader.ts:133:15) |  

[RedYetiDev]

Fixed by #54524

[elliottAtTreatment]

We are getting the same thing with Firestore queries when we use the same version. We are downgrading node

[quinnspraut]

In case helpful - still seeing the RangeError [ERR_BUFFER_OUT_OF_BOUNDS]: "length" is outside of buffer bounds error when using GCP Secret Manager - temporarily resolved by reverting to 22.5.0

const { SecretManagerServiceClient } = require('@google-cloud/secret-manager');
...
const [response] = await (new SecretManagerServiceClient()).accessSecretVersion({
    name: secretPath,
});

[RafaelGSS]

It should be fixed in next week's release (22.8.0) which includes: #54524 as a patch. I will take care of it.

[nicolasnoble]

Did we just witness an example of Hyrum's law?

[ericrange]

i dont know if my issue has something to do with this, but since 22.7 the encoding of buffers wont work anymore.

MongoError: text contains invalid UTF-8

even expressjs delivers garbage.

yesterday, i build my app on node:22-alpine and all works fine. since: https://hub.docker.com/layers/library/node/22-alpine/images/sha256-3dbc5d17cf89e1d8ae6f4a8a562f8a5b5df52b4c7060bfb359de86de6a3ecc3c

all is broken. nothing changed in the code base!

btw. "node:20-alpine" works as expected (like node:22-alpine yesterday)

#54543