Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V8 serialize is not deterministic #2448

Closed
trueshura opened this issue Jan 28, 2020 · 3 comments
Closed

V8 serialize is not deterministic #2448

trueshura opened this issue Jan 28, 2020 · 3 comments

Comments

@trueshura
Copy link

trueshura commented Jan 28, 2020

  • node version v10.17.0:
  • Linux 4.19.19-041919-lowlatency #201901310331 SMP PREEMPT Thu Jan 31 08:37:07 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux:
  • npm version:
{ npm: '6.12.0',
  ares: '1.15.0',
  brotli: '1.0.7',
  cldr: '35.1',
  http_parser: '2.8.0',
  icu: '64.2',
  modules: '64',
  napi: '5',
  nghttp2: '1.39.2',
  node: '10.17.0',
  openssl: '1.1.1d',
  tz: '2019a',
  unicode: '12.1',
  uv: '1.28.0',
  v8: '6.8.275.32-node.54',
  zlib: '1.2.11' }

v8.serialize produce different output, still decodable but it's different bytes, and different length. it's matter for me.
same PC, same version.
just want to know: what cause that difference and how to avoid it.

const v8 = require('v8');

let data = {
    key: 'value',
    arrValue: [1,2,3,4],
    objValue: {
        nestedKey: 'value'
    }
};

const buffData = v8.serialize(data);
const dataCopy = v8.deserialize(buffData);

// it seems we still have same object
console.log(JSON.stringify(dataCopy) === JSON.stringify(data));

// but encoded buffer is different!
const buffEncoded = v8.serialize(Object.assign({}, dataCopy));
console.log(buffData.length !== buffEncoded.length);

console.log(buffData.toString('hex'));
console.log(buffEncoded.toString('hex'));
@trueshura
Copy link
Author

If you remove array element - it works like a charm again!
i mean

let data = {
    key: 'value',
    objValue: {
        nestedKey: 'value'
    }
};

encodes as expected

@addaleax
Copy link
Member

V8 uses different internal representations for some values, for example, Arrays in V8 can be dense or sparse, with special cases for integer arrays or floating-point-value arrays, and strings can have different internal representations as well, e.g. using UTF16 encoding or using ISO-8859-1.

Generally, you can’t rely on the exact output from v8.serialize(), that’s correct. If it’s important to you to have deterministic output, it might not be the right tool for you.

@trueshura
Copy link
Author

V8 uses different internal representations for some values, for example, Arrays in V8 can be dense or sparse, with special cases for integer arrays or floating-point-value arrays, and strings can have different internal representations as well, e.g. using UTF16 encoding or using ISO-8859-1.

it's ok, but i didn't expect diffrent output for "same" values.

addaleax added a commit to addaleax/node that referenced this issue Jan 28, 2020
codebytere pushed a commit to nodejs/node that referenced this issue Feb 17, 2020
Fixes: nodejs/help#2448

PR-URL: #31548
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
codebytere pushed a commit to nodejs/node that referenced this issue Mar 15, 2020
Fixes: nodejs/help#2448

PR-URL: #31548
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
codebytere pushed a commit to nodejs/node that referenced this issue Mar 17, 2020
Fixes: nodejs/help#2448

PR-URL: #31548
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
codebytere pushed a commit to nodejs/node that referenced this issue Mar 30, 2020
Fixes: nodejs/help#2448

PR-URL: #31548
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants