"FATAL ERROR: invalid array length Allocation failed - process out of memory" introduced in 0.11.15

[SheetJSDev]

The js-xls test script passes in node <= 0.11.14 as well as in io.js but not in node 0.11.15 0.11.16 or 0.12

https://travis-ci.org/SheetJS/js-xls/builds/50244885 shows where the fails occur. It works in io.js as well as in node 0.10 and versions up to 0.11.14 so I suspect that 0.11.15 changed Buffer behavior.

@trevnorris was there a fix in iojs that hasn't been applied to node?

[trevnorris]

@SheetJSDev Can you post the URL to the exact test that's failing?

[trevnorris]

I pushed a fix that was created by @bnoordhuis in 04b63e0. Can you let me know if that fixes the issue?

[SheetJSDev]

@trevnorris that did not resolve the issue, unfortunately :( I pulled from the v0.12 branch which appears to have that commit.

The repo is https://github.com/SheetJS/js-xls and the travis commands are https://github.com/SheetJS/js-xls/blob/master/.travis.yml (warning: it downloads a ton of test files)

It's difficult to point to a specific test because each individual test case works on its own (the repo includes a CLI script xls.njs and that passes on each file, even in 0.11.15 and 0.12). It just blows up after a certain number of tests.

[trevnorris]

@SheetJSDev Thanks. I've been able to reproduce the issue. I'll do some testing and let you know what I find.

[trevnorris]

Update: this is a race condition, but for what I can tell. If I run the tests using latest v0.12 debug build it doesn't have that issue.

On the flip side, it generated this error:

#
# Fatal error in ../deps/v8/src/heap/heap.cc, line 3324
# CHECK(!lo_space()->Contains(object)) failed
#

==== C stack trace ===============================

 1: V8_Fatal
 2: void v8::internal::Heap::RightTrimFixedArray<(v8::internal::Heap::InvocationMode)1>(v8::internal::FixedArrayBase*, int)
 3: _ZN2v88internal20FastElementsAccessorINS0_31FastHoleyObjectElementsAccessorENS0_18ElementsKindTraitsILNS0_12ElementsKindE3EEEE25SetLengthWithoutNormalizeENS0_6HandleINS0_14FixedArrayBaseEEENS7_INS0_7J
 4: _ZN2v88internal20ElementsAccessorBaseINS0_31FastHoleyObjectElementsAccessorENS0_18ElementsKindTraitsILNS0_12ElementsKindE3EEEE13SetLengthImplENS0_6HandleINS0_8JSObjectEEENS7_INS0_6ObjectEEENS7_INS0_14
 5: v8::internal::ElementsAccessorBase<v8::internal::FastHoleyObjectElementsAccessor, v8::internal::ElementsKindTraits<(v8::internal::ElementsKind)3> >::SetLength(v8::internal::Handle<v8::internal::JSArray>, v8::internal::Handle<v8::internal::Object>)
 6: v8::internal::JSArray::SetElementsLength(v8::internal::Handle<v8::internal::JSArray>, v8::internal::Handle<v8::internal::Object>)
 7: v8::internal::Accessors::ArrayLengthSetter(v8::Local<v8::String>, v8::Local<v8::Value>, v8::PropertyCallbackInfo<void> const&)
 8: v8::internal::PropertyCallbackArguments::Call(void (*)(v8::Local<v8::String>, v8::Local<v8::Value>, v8::PropertyCallbackInfo<void> const&), v8::Local<v8::String>, v8::Local<v8::Value>)
 9: v8::internal::Object::SetPropertyWithAccessor(v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::Object>, v8::internal::StrictMode)
10: v8::internal::Object::SetProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::StrictMode, v8::internal::Object::StoreFromKeyed)
11: v8::internal::Object::SetProperty(v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::StrictMode, v8::internal::Object::StoreFromKeyed)
12: v8::internal::StoreIC::Store(v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::Object::StoreFromKeyed)
13: ??
14: v8::internal::StoreIC_Miss(int, v8::internal::Object**, v8::internal::Isolate*)
15: ??
Segmentation fault (core dumped)

I've bisected the bad range from 9116b24 to 7a0cfe9. That series of commits is where we upgrade V8 to 3.28.73. So it appears the issue is coming from V8 itself.

[SheetJSDev]

@trevnorris thanks for looking into this!

[trevnorris]

@SheetJSDev I've opened up a PR to help take care of this issue (see #9185). Thanks for reporting it.

[indutny]

@trevnorris it seems that v8 has attempted to trim fixed array in Large Object space. (probably you already figured it out)

[indutny]

And it doesn't seem like recent v8 has that assertion anymore... Searching for particular commit...

[indutny]

@trevnorris v8/v8@5e57059?diff=unified#diff-27e3acf610fe20cbc4721c969ad00f62L3324 see RightTrimFixedArray. I think this patch could be backported if really needed.

[indutny]

And big kudos for using stale v8 version again, gosh...

[trevnorris]

@indutny thanks for narrowing that down. I ported the fix, and while it does fix the abort in the debug build, it doesn't prevent a FATAL ERROR from a release build.

[trevnorris]

And, again for the record, the reason V8 hasn't been updated was because there are too many customers on el5/6, etc., that don't have easy access to gcc that supports compiling native add-ons using c++11. If it weren't for that, I would have made sure we'd upgrade V8 before the v0.12.0 release.

[SheetJSDev]

@trevnorris @indutny again thank you very much for looking into this! We should have picked up on this sooner but most of our users are using the 0.10 stable series.

There are many people who are discovering node through our modules, and the last time we had an issue with a node stable version (0.10.31 #8208) we advised people to roll back to 0.10.30. What should we recommend for new people who install node 0.12.0: use stable 0.10, switch to iojs or wait for 0.12.1?

[trevnorris]

@SheetJSDev Unfortunately NodeSummit is happening at the moment, so most of the team is tied up. I'd say wait until v0.12.1.

[misterdjules]

@SheetJSDev #9185 landed as #18206, and I believe it fixes this issue.

Before the above mentioned changes, I was able to reproduce the FATAL ERROR:

➜  js-xls git:(master) make coveralls-spin
make coveralls & bash misc/spin.sh $!
mocha --require blanket --reporter mocha-lcov-reporter | ./node_modules/coveralls/bin/coveralls.js
 [-]FATAL ERROR: invalid array length Allocation failed - process out of memory
[error] "2015-05-04T23:21:59.235Z"  'error from lcovParse: ' 'Failed to parse string'
[error] "2015-05-04T23:21:59.237Z"  'input: ' ''
[error] "2015-05-04T23:21:59.238Z"  'error from convertLcovToCoveralls'

/Users/JulienGilli/dev/repros/gh-9181/js-xls/node_modules/coveralls/bin/coveralls.js:18
        throw err;
              ^
Failed to parse string
make[1]: *** [coveralls] Error 1

After the changes, I get a different error:

➜  js-xls git:(master) make coveralls-spin
make coveralls & bash misc/spin.sh $!
mocha --require blanket --reporter mocha-lcov-reporter | ./node_modules/coveralls/bin/coveralls.js
 [/][error] "2015-05-04T23:22:40.699Z"  'error from lcovParse: ' 'Failed to parse string'
[error] "2015-05-04T23:22:40.700Z"  'input: ' ''
[error] "2015-05-04T23:22:40.701Z"  'error from convertLcovToCoveralls'

/Users/JulienGilli/dev/repros/gh-9181/js-xls/node_modules/coveralls/bin/coveralls.js:18
        throw err;
              ^
Failed to parse string
make[1]: *** [coveralls] Error 1

Could you please confirm that #18206 fixes your issue?

[SheetJSDev]

Sorry for very late reply, but this was resolved in 0.12.18 and most likely in a much older version too.