build fails with --enable-asan --ninja on Ubuntu 18.04

[gabrielschulhof]

Linking cctest fails with errors of the form below referring to many symbols:

`_ZZN4node14ThreadPoolWorkC4EPNS_11EnvironmentEE19__PRETTY_FUNCTION__' referenced in
section `.data.rel.local' of obj/src/libnode.node_api.o: defined in discarded section 
`.rodata._ZZN4node14ThreadPoolWorkC4EPNS_11EnvironmentEE19__PRETTY_FUNCTION__[_ZZN4node14ThreadPoolWorkC4EPNS_11EnvironmentEE4args]' 
of obj/src/libnode.node_api.o

[gabrielschulhof]

Maybe this is related to https://github.com/nodejs/node/pull/32244/checks?check_run_id=506791476

[gabrielschulhof]

gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0

[sam-github]

I suggest reverting #31902 in #32324

[mmarchini]

@gabrielschulhof did you get this error on GitHub Actions on your machine?

[FullyArticulate]

Same error here, and I'm not using GIthub actions.

[mmarchini]

@FullyArticulate @gabrielschulhof can you share more on the error output? This could be the same as our GitHub Actions issue (not enough memory to link) or could be something else. On my machine it's building fine (Ubuntu 19.10, gcc (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008).

Note: test runs will fail, our codebase is not ASAN compliant yet. But cctest builds just fine here.

[richardlau]

Linking cctest fails with errors of the form below referring to many symbols:

`_ZZN4node14ThreadPoolWorkC4EPNS_11EnvironmentEE19__PRETTY_FUNCTION__' referenced in
section `.data.rel.local' of obj/src/libnode.node_api.o: defined in discarded section 
`.rodata._ZZN4node14ThreadPoolWorkC4EPNS_11EnvironmentEE19__PRETTY_FUNCTION__[_ZZN4node14ThreadPoolWorkC4EPNS_11EnvironmentEE4args]' 
of obj/src/libnode.node_api.o

Same error in https://github.com/nodejs/node/pull/32352/checks#step:4:3938 that was run for #32352.

[mmarchini]

That's an out of memory error (the important line is collect2: ...). Users can double-check that's the case with dmesg, there will be a kernel message saying ld was killed to free memory.

I don't think there's anything we can do to fix this on developers machines. ASAN (just as Debug) builds are heavy, and linking requires a lot of memory. I've solved it on other projects by using less parallelism (a lower -jN number).

[gengjiawen]

I have run ASAN checks on a 16GB machine, works fine. I think it's a issue related to gcc or other toolchain. @gabrielschulhof Can you try run asan in docker image gengjiawen/node-build:2020-02-14, it's our current CI using now.

[mmarchini]

@gengjiawen I tried with the image limiting the memory available for the container (docker run -m 7g ...) and I got the same failure, which means if the host doesn't have enough memory, it will cause the same failure.

Also, users trying to build ASAN on Docker on OS X will probably have a hard time, since Docker on OS X runs on top of a Linux VM with 2Gb Ram by default (unless they changed the default recently). Not sure if that's the case here, but I thought it was worth mentioning.

[gengjiawen]

on Github Action the memory error is

FAILED: mkcodecache 
c++ -fsanitize=address -pthread -rdynamic -m64 -Wl,--whole-archive obj/deps/zlib/libzlib.a -Wl,--no-whole-archive -Wl,--whole-archive obj/deps/uv/libuv.a -Wl,--no-whole-archive -Wl,-z,noexecstack -Wl,--whole-archive obj/tools/v8_gypfiles/libv8_snapshot.a -Wl,--no-whole-archive -Wl,-z,relro -Wl,-z,now -Wl,--whole-archive,obj/deps/openssl/libopenssl.a -Wl,--no-whole-archive -pthread -o mkcodecache -Wl,--start-group obj/src/mkcodecache.node_snapshot_stub.o obj/src/mkcodecache.node_code_cache_stub.o obj/tools/code_cache/mkcodecache.mkcodecache.o obj/tools/code_cache/mkcodecache.cache_builder.o obj/libnode.a obj/deps/histogram/libhistogram.a obj/deps/uvwasi/libuvwasi.a obj/tools/v8_gypfiles/libv8_snapshot.a obj/tools/v8_gypfiles/libv8_libplatform.a obj/tools/icu/libicui18n.a obj/deps/zlib/libzlib.a obj/deps/llhttp/libllhttp.a obj/deps/cares/libcares.a obj/deps/uv/libuv.a obj/deps/nghttp2/libnghttp2.a obj/deps/brotli/libbrotli.a obj/deps/openssl/libopenssl.a obj/tools/icu/libicuucx.a obj/tools/icu/libicudata.a obj/tools/v8_gypfiles/libv8_base_without_compiler.a obj/tools/v8_gypfiles/libv8_libbase.a obj/tools/v8_gypfiles/libv8_libsampler.a obj/tools/v8_gypfiles/libv8_zlib.a obj/tools/v8_gypfiles/libv8_compiler.a obj/tools/v8_gypfiles/libv8_initializers.a  -lm -ldl -Wl,--end-group
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
##[error]Process completed with exit code 1.

Not like the error @gabrielschulhof posted. Maybe not the same problem ?

[mmarchini]

Correct me if I'm wrong, but the "referenced in section..." message looks like a warning, not an error. And that line is also present in the CI:

[gengjiawen]

In this case, it looks like the memory issue. I don't is there a way to improve this. But for now, a big memory machine is needed for ASAN build.

[gengjiawen]

This is fixed by @mmalecki patch for gyp.