From 12d0fe0361461794f49338a4fcc3ac655099116f Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Fri, 24 Jan 2020 11:46:40 -0500 Subject: [PATCH 01/73] doc: guide - using valgrind to debug memory leaks Add doc for using valgrind to debug native memory leaks. Started writing this up as part of an effort in the Diagnostic WG but think it's better to have it in the core guides and then be referenced by the docs in the Diagnostic WG repo. For more details on the Diagnostic WG effort see https://github.com/nodejs/diagnostics/issues/254#issuecomment-538853390 This guide is related to `/step3 - using_native_tools.md` --- .../investigating_native_memory_leak.md | 441 ++++++++++++++++++ 1 file changed, 441 insertions(+) create mode 100644 doc/guides/investigating_native_memory_leak.md diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md new file mode 100644 index 00000000000000..9a5b9b6eae6bce --- /dev/null +++ b/doc/guides/investigating_native_memory_leak.md @@ -0,0 +1,441 @@ +# Investigating Memory Leaks with valgrind + +A Node.js process may run out of memory due to excessive consumption of +`Native memory`. `Native Memory` is memory which is not managed by the the +V8 Garbage collector and is allocated either by the Node.js runtime, it's +dependancies or native [addons](://nodejs.org/docs/latest/api/n-api.html). + +This guide provides informaiton on how to use valgrind to investigate these +issues. + +## valgrind + +[Valgrind](https://valgrind.org/docs/manual/quick-start.html) is a +tool available on linux distributions which can be used to investigate +memory usage including identifying memory leaks (memory which is +allocated and not freed) and other memory related problems +like double freeing memory. + +To use valgrind: + +* Be patient, running under valgrind slows execution significantly due + to the checks being performed. +* Reduce your test case to the smallest reproduce. Due to the slowdown it is + important to run the minimum testcase in order to be able to do it in + a reasonable time. + +## Installation + +It is an optional package in most cases and must be installed. For example on +ubuntu: + +```bash +apt-get install valgrind +``` + +## Invocation +The simplest invocation of valgrind is simply + +```bash +valgrind node test.js +``` + +with the output being: + +```bash +user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ valgrind test.js +valgrind: test.js: command not found +user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ valgrind node test.js +==28993== Memcheck, a memory error detector +==28993== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. +==28993== Using valgrind-3.13.0 and LibVEX; rerun with -h for copyright info +==28993== Command: node test.js +==28993== +==28993== Use of uninitialised value of size 8 +==28993== at 0x12F2279: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==28993== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==28993== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==28993== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==28993== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==28993== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==28993== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==28993== by 0x12F3E9C: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==28993== by 0x12F3C77: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==28993== by 0xC7C9CF: v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==28993== by 0xC7CE87: v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle, v8::internal::Handle, int, v8::internal::Handle*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==28993== by 0xB4CF3A: v8::Function::Call(v8::Local, v8::Local, int, v8::Local*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==28993== +--28993-- WARNING: unhandled amd64-linux syscall: 332 +--28993-- You may be able to write your own handler. +--28993-- Read the file README_MISSING_SYSCALL_OR_IOCTL. +--28993-- Nevertheless we consider this a bug. Please report +--28993-- it at http://valgrind.org/support/bug_reports.html. +==28993== +==28993== HEAP SUMMARY: +==28993== in use at exit: 6,140 bytes in 23 blocks +==28993== total heap usage: 12,888 allocs, 12,865 frees, 13,033,244 bytes allocated +==28993== +==28993== LEAK SUMMARY: +==28993== definitely lost: 0 bytes in 0 blocks +==28993== indirectly lost: 0 bytes in 0 blocks +==28993== possibly lost: 304 bytes in 1 blocks +==28993== still reachable: 5,836 bytes in 22 blocks +==28993== suppressed: 0 bytes in 0 blocks +==28993== Rerun with --leak-check=full to see details of leaked memory +==28993== +==28993== For counts of detected and suppressed errors, rerun with: -v +==28993== Use --track-origins=yes to see where uninitialised values come +``` + +This reports that Node.js is not `completely` clean as there is some memory +that was allocated but not freed when the process shut down. It is often +impractical/not worth being completely clean in this respect and Node.js +does a pretty good job only leaving in the order or 6k bytes that are +not freed on shutdown. + +## An obvious memory leak + +Leaks can be introduced in native addons and the following is a simple +example generated by using the Hello world example in +[node-addon-examples](https://github.com/nodejs/node-addon-examples). + +In that example a loop which allocates ~1MB of memory and never frees it +has been added: + +```C++ +void* malloc_holder = nullptr; +napi_value Method(napi_env env, napi_callback_info info) { + napi_status status; + napi_value world; + status = napi_create_string_utf8(env, "world", 5, &world); + assert(status == napi_ok); + + // NEW LEAK HERE + for (int i=0; i< 1024; i++) { + malloc_holder = malloc(1024); + } + + return world; +} +``` + +When trying to introduce a memory leak you need to ensure that +the compiler has not optimized out the code you've introduced to generate +the leak. For example, with assigning the result of malloc to either a +global variable or a variable that is read the compiler will `optimize` +out the loop with the malloc and valgrind does not report +a problem (since once does not exist in the code being run). + +Running valgrind on the example program with this addition shows the following: + +```bash +user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ valgrind node hello.js +==1504== Memcheck, a memory error detector +==1504== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. +==1504== Using V#algrind-3.13.0 and LibVEX; rerun with -h for copyright info +==1504== Command: node hello.js +==1504== +==1504== Use of uninitialised value of size 8 +==1504== at 0x12F2279: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==1504== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==1504== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==1504== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==1504== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==1504== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==1504== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==1504== by 0x12F3E9C: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==1504== by 0x12F3C77: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==1504== by 0xC7C9CF: v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==1504== by 0xC7CE87: v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle, v8::internal::Handle, int, v8::internal::Handle*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==1504== by 0xB4CF3A: v8::Function::Call(v8::Local, v8::Local, int, v8::Local*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==1504== +--1504-- WARNING: unhandled amd64-linux syscall: 332 +--1504-- You may be able to write your own handler. +--1504-- Read the file README_MISSING_SYSCALL_OR_IOCTL. +--1504-- Nevertheless we consider this a bug. Please report +--1504-- it at http://valgrind.org/support/bug_reports.html. +world +==1504== +==1504== HEAP SUMMARY: +==1504== in use at exit: 1,008,003 bytes in 1,032 blocks +==1504== total heap usage: 17,603 allocs, 16,571 frees, 18,306,103 bytes allocated +==1504== +==1504== LEAK SUMMARY: +==1504== definitely lost: 996,064 bytes in 997 blocks +==1504== indirectly lost: 0 bytes in 0 blocks +==1504== possibly lost: 3,304 bytes in 4 blocks +==1504== still reachable: 8,635 bytes in 31 blocks +==1504== of which reachable via heuristic: +==1504== multipleinheritance: 48 bytes in 1 blocks +==1504== suppressed: 0 bytes in 0 blocks +==1504== Rerun with --leak-check=full to see details of leaked memory +==1504== +==1504== For counts of detected and suppressed errors, rerun with: -v +==1504== Use --track-origins=yes to see where uninitialised values come from +==1504== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) +``` + +Valgrind is reporting a problem as it shows 996,604 bytes as +definitely lost and the question is how to find where that memory is +allocated. The next step is to rerun as suggested in the +output with `--leak-check=full` + +``` bash +user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ valgrind --leak-check=full node hello.js +==4174== Memcheck, a memory error detector +==4174== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. +==4174== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info +==4174== Command: node hello.js +==4174== +==4174== Use of uninitialised value of size 8 +==4174== at 0x12F2279: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F3E9C: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F3C77: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0xC7C9CF: v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0xC7CE87: v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle, v8::internal::Handle, int, v8::internal::Handle*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0xB4CF3A: v8::Function::Call(v8::Local, v8::Local, int, v8::Local*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== +--4174-- WARNING: unhandled amd64-linux syscall: 332 +--4174-- You may be able to write your own handler. +--4174-- Read the file README_MISSING_SYSCALL_OR_IOCTL. +--4174-- Nevertheless we consider this a bug. Please report +--4174-- it at http://valgrind.org/support/bug_reports.html. +world +==4174== +==4174== HEAP SUMMARY: +==4174== in use at exit: 1,008,003 bytes in 1,032 blocks +==4174== total heap usage: 17,606 allocs, 16,574 frees, 18,305,977 bytes allocated +==4174== +==4174== 64 bytes in 1 blocks are definitely lost in loss record 17 of 35 +==4174== at 0x4C3017F: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) +==4174== by 0x9AEAD5: napi_module_register (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x4010732: call_init (dl-init.c:72) +==4174== by 0x4010732: _dl_init (dl-init.c:119) +==4174== by 0x40151FE: dl_open_worker (dl-open.c:522) +==4174== by 0x5D052DE: _dl_catch_exception (dl-error-skeleton.c:196) +==4174== by 0x40147C9: _dl_open (dl-open.c:605) +==4174== by 0x4E3CF95: dlopen_doit (dlopen.c:66) +==4174== by 0x5D052DE: _dl_catch_exception (dl-error-skeleton.c:196) +==4174== by 0x5D0536E: _dl_catch_error (dl-error-skeleton.c:215) +==4174== by 0x4E3D734: _dlerror_run (dlerror.c:162) +==4174== by 0x4E3D050: dlopen@@GLIBC_2.2.5 (dlopen.c:87) +==4174== by 0x9B29A0: node::binding::DLOpen(v8::FunctionCallbackInfo const&)::{lambda(node::binding::DLib*)#1}::operator()(node::binding::DLib*) const (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== +==4174== 304 bytes in 1 blocks are possibly lost in loss record 27 of 35 +==4174== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) +==4174== by 0x40134A6: allocate_dtv (dl-tls.c:286) +==4174== by 0x40134A6: _dl_allocate_tls (dl-tls.c:530) +==4174== by 0x5987227: allocate_stack (allocatestack.c:627) +==4174== by 0x5987227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644) +==4174== by 0xAAF9DC: node::inspector::Agent::Start(std::string const&, node::DebugOptions const&, std::shared_ptr, bool) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x9A8BE7: node::Environment::InitializeInspector(std::unique_ptr >) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0xA1C9A5: node::NodeMainInstance::CreateMainEnvironment(int*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0xA1CB42: node::NodeMainInstance::Run() (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x9ACB67: node::Start(int, char**) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x5BBFB96: (below main) (libc-start.c:310) +==4174== +==4174== 2,000 bytes in 2 blocks are possibly lost in loss record 33 of 35 +==4174== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) +==4174== by 0x9794979: Method(napi_env__*, napi_callback_info__*) (in /home/user1/valgrind/node-addon-examples/1_hello_world/napi/build/Release/hello.node) +==4174== by 0x98F764: v8impl::(anonymous namespace)::FunctionCallbackWrapper::Invoke(v8::FunctionCallbackInfo const&) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0xBA6FC8: v8::internal::MaybeHandle v8::internal::(anonymous namespace)::HandleApiCallHelper(v8::internal::Isolate*, v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, v8::internal::BuiltinArguments) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0xBA8DB6: v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x1376358: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== +==4174== 997,000 bytes in 997 blocks are definitely lost in loss record 35 of 35 +==4174== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) +==4174== by 0x9794979: Method(napi_env__*, napi_callback_info__*) (in /home/user1/valgrind/node-addon-examples/1_hello_world/napi/build/Release/hello.node) +==4174== by 0x98F764: v8impl::(anonymous namespace)::FunctionCallbackWrapper::Invoke(v8::FunctionCallbackInfo const&) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0xBA6FC8: v8::internal::MaybeHandle v8::internal::(anonymous namespace)::HandleApiCallHelper(v8::internal::Isolate*, v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, v8::internal::BuiltinArguments) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0xBA8DB6: v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x1376358: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== +==4174== LEAK SUMMARY: +==4174== definitely lost: 997,064 bytes in 998 blocks +==4174== indirectly lost: 0 bytes in 0 blocks +==4174== possibly lost: 2,304 bytes in 3 blocks +==4174== still reachable: 8,635 bytes in 31 blocks +==4174== of which reachable via heuristic: +==4174== multipleinheritance: 48 bytes in 1 blocks +==4174== suppressed: 0 bytes in 0 blocks +==4174== Reachable blocks (those to which a pointer was found) are not shown. +==4174== To see them, rerun with: --leak-check=full --show-leak-kinds=all +==4174== +==4174== For counts of detected and suppressed errors, rerun with: -v +==4174== Use --track-origins=yes to see where uninitialised values come from +==4174== ERROR SUMMARY: 5 errors from 5 contexts (suppressed: 0 from 0) +user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ +``` + +This is the most interesting report: + +```bash +==4174== 997,000 bytes in 997 blocks are definitely lost in loss record 35 of 35 +==4174== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) +==4174== by 0x9794979: Method(napi_env__*, napi_callback_info__*) (in /home/user1/valgrind/node-addon-examples/1_hello_world/napi/build/Release/hello.node) +==4174== by 0x98F764: v8impl::(anonymous namespace)::FunctionCallbackWrapper::Invoke(v8::FunctionCallbackInfo const&) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0xBA6FC8: v8::internal::MaybeHandle v8::internal::(anonymous namespace)::HandleApiCallHelper(v8::internal::Isolate*, v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, v8::internal::BuiltinArguments) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0xBA8DB6: v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x1376358: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +``` + +From the stack trace we can tell that the leak came from a native addon: + +```bash +==4174== by 0x9794979: Method(napi_env__*, napi_callback_info__*) (in /home/user1/valgrind/node-addon-examples/1_hello_world/napi/build/Release/hello.node) +``` + +What we can't tell is where in the native addon the memory is being +allocated. This is because by default the addon is compiled without +the debug symbols which valgrind needs to be able to provide more +information. + +## Enabling debug symbols to get more information + +Leaks may be either in addons or Node.js itself. The sections which +follow cover the steps needed to enable debug symbols to get more info. + +### addons + +To enable debug symbols for all of your addons that are compiled on +install simply use: + +```bash +npm install --debug +``` + +Any options which are not consumed by npm are passed on to node-gyp and this +results in the addons being compiled with the debug option. + +If the native addon contains pre-built binaries you will need to force +a rebuild. + +The next step is to run valgrind after the rebuild. This time the information +for the leaking location includes the name of the source file and the +line number! + +```bash +==18481== 997,000 bytes in 997 blocks are definitely lost in loss record 35 of 35 +==18481== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) +>>>>> ==18481== by 0x9794989: Method(napi_env__*, napi_callback_info__*) (hello.cc:13) <<<<< +==18481== by 0x98F764: v8impl::(anonymous namespace)::FunctionCallbackWrapper::Invoke(v8::FunctionCallbackInfo const&) (in /home/user1/val grind/node-v12.14.1-linux-x64/bin/node) +==18481== by 0xBA6FC8: v8::internal::MaybeHandle v8::internal::(anonymous namespace)::HandleApiCallHelper(v8::internal:: Isolate*, v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, v8::internal::BuiltinArguments) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==18481== by 0xBA8DB6: v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) (in /home/user1/valgrind/node-v12.14.1-linux- x64/bin/node) +==18481== by 0x1376358: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==18481== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==18481== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==18481== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==18481== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==18481== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==18481== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +``` + +which shows us exactly where the leak is occuring in the file hello.cc: + +```C++ +6 void* malloc_holder = nullptr; + 7 napi_value Method(napi_env env, napi_callback_info info) { + 8 napi_status status; + 9 napi_value world; + 10 status = napi_create_string_utf8(env, "world", 5, &world); + 11 assert(status == napi_ok); + 12 for (int i=0; i< 1000; i++) { + 13 malloc_holder = malloc(1000); <<<<<< Yup this is where we are allocating the memory that is not freed + 14 } + 15 return world; + 16 } +``` + +### Node.js binary + +If the leak is not in an addon and in the Node.js binary itself you may need +to compile node yourself and turn on debug symbols. Looking at this entry +reported by valgrind, with the release binary we see: + +```bash +74== 304 bytes in 1 blocks are possibly lost in loss record 27 of 35 +==4174== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) +==4174== by 0x40134A6: allocate_dtv (dl-tls.c:286) +==4174== by 0x40134A6: _dl_allocate_tls (dl-tls.c:530) +==4174== by 0x5987227: allocate_stack (allocatestack.c:627) +==4174== by 0x5987227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644) +==4174== by 0xAAF9DC: node::inspector::Agent::Start(std::string const&, node::DebugOptions const&, std::shared_ptr, bool) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x9A8BE7: node::Environment::InitializeInspector(std::unique_ptr >) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0xA1C9A5: node::NodeMainInstance::CreateMainEnvironment(int*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0xA1CB42: node::NodeMainInstance::Run() (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x9ACB67: node::Start(int, char**) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) +==4174== by 0x5BBFB96: (below main) (libc-start.c:310) +``` + +This gives us some information of where to look (node::inspector::Agent::Start) +but not where in that function. We get more information that you might expect +(or see by default with addons) because the Node.js binary exports many of +its symbols using -rdynamic so that they can be used by addons. If the stack +gives you enough information to track down where the leak is great, +otherwise the next step is to compile a Node.js binary with debugging enabled. + +To get additional information with valgrind: + +* check out the Node.js source corresponding to the release that you + want to debug. For example +```bash +git clone https://github.com/nodejs/node.git +git checkout -b v12.14.1 +``` +* compile with debug enabled (for addional info see [building a debug build](https://github.com/nodejs/node/blob/v12.14.1/BUILDING.md#building-a-debug-build) +```bash +./configure --debug +make -j4 +``` +* make sure to run with your compiled debug version of Node.js. Having used + `./configure --debug`, two binaries will have been built when make was run. + You must use the one which is in .../out/Debug. Add that version of the + binary to your path. For example: + +```bash +export PATH=/home/user1/node/out/Debug/:$PATH +``` + +Running valgrind using the Node.js binary with debug symbols enabled shows: + +```bash +==44112== 592 bytes in 1 blocks are possibly lost in loss record 26 of 27 +==44112== at 0x4C2BF79: calloc (vg_replace_malloc.c:762) +==44112== by 0x4012754: _dl_allocate_tls (in /usr/lib64/ld-2.17.so) +==44112== by 0x586287B: pthread_create@@GLIBC_2.2.5 (in /usr/lib64/libpthread-2.17.so) +==44112== by 0xFAB2D2: node::inspector::(anonymous namespace)::StartDebugSignalHandler() (inspector_agent.cc:140) +==44112== by 0xFACB10: node::inspector::Agent::Start(std::string const&, node::DebugOptions const&, std::shared_ptr, bool) (inspector_agent.cc:777) +==44112== by 0xE3A0BB: node::Environment::InitializeInspector(std::unique_ptr >) (node.cc:216) +==44112== by 0xEE8F3E: node::NodeMainInstance::CreateMainEnvironment(int*) (node_main_instance.cc:222) +==44112== by 0xEE8831: node::NodeMainInstance::Run() (node_main_instance.cc:108) +==44112== by 0xE3CDEC: node::Start(int, char**) (node.cc:996) +==44112== by 0x22D8BBF: main (node_main.cc:126) + +``` + +where we can see the specific file name and line in the Node.js code which +caused the allocation (inspector_agent.cc:140). From 3ef3a0c71874ed30e57ad852bf57231340e42fd1 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:18:25 -0500 Subject: [PATCH 02/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 9a5b9b6eae6bce..bf501c0f073e31 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -3,7 +3,7 @@ A Node.js process may run out of memory due to excessive consumption of `Native memory`. `Native Memory` is memory which is not managed by the the V8 Garbage collector and is allocated either by the Node.js runtime, it's -dependancies or native [addons](://nodejs.org/docs/latest/api/n-api.html). +dependencies or native [addons](://nodejs.org/docs/latest/api/n-api.html). This guide provides informaiton on how to use valgrind to investigate these issues. From 7728bfc81095149423908be2724ea7e5a541b861 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:18:41 -0500 Subject: [PATCH 03/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index bf501c0f073e31..91390b7bdc0872 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -5,7 +5,7 @@ A Node.js process may run out of memory due to excessive consumption of V8 Garbage collector and is allocated either by the Node.js runtime, it's dependencies or native [addons](://nodejs.org/docs/latest/api/n-api.html). -This guide provides informaiton on how to use valgrind to investigate these +This guide provides information on how to use valgrind to investigate these issues. ## valgrind From 73c5e923e4ad6baa8c690e18843a7ac1bb18d8b3 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:18:56 -0500 Subject: [PATCH 04/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 91390b7bdc0872..c344fd9eac51d1 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -21,7 +21,7 @@ To use valgrind: * Be patient, running under valgrind slows execution significantly due to the checks being performed. * Reduce your test case to the smallest reproduce. Due to the slowdown it is - important to run the minimum testcase in order to be able to do it in + important to run the minimum test case in order to be able to do it in a reasonable time. ## Installation From 590dc3fc597e2bd28b2946bf422a5c37f379a4f1 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:19:12 -0500 Subject: [PATCH 05/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index c344fd9eac51d1..57c0b5e438daf3 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -27,7 +27,7 @@ To use valgrind: ## Installation It is an optional package in most cases and must be installed. For example on -ubuntu: +Debian/Ubuntu: ```bash apt-get install valgrind From d088370373decc2b233308334102689e27733ea7 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:19:28 -0500 Subject: [PATCH 06/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 57c0b5e438daf3..3d3718313f5845 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -26,7 +26,7 @@ To use valgrind: ## Installation -It is an optional package in most cases and must be installed. For example on +It is an optional package in most cases and must be installed explicitly. For example on Debian/Ubuntu: ```bash From 50c9880f542c37c6a03c6e656cd39d794d98227e Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:19:47 -0500 Subject: [PATCH 07/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 3d3718313f5845..a9a1b9e65a569e 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -87,7 +87,7 @@ user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ valgrind node ==28993== Use --track-origins=yes to see where uninitialised values come ``` -This reports that Node.js is not `completely` clean as there is some memory +This reports that Node.js is not _completely_ clean as there is some memory that was allocated but not freed when the process shut down. It is often impractical/not worth being completely clean in this respect and Node.js does a pretty good job only leaving in the order or 6k bytes that are From 2962a9087b29691d7cd2f0a725aafc0c829c4dd7 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:20:02 -0500 Subject: [PATCH 08/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index a9a1b9e65a569e..cc63b9e6460ca1 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -90,7 +90,7 @@ user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ valgrind node This reports that Node.js is not _completely_ clean as there is some memory that was allocated but not freed when the process shut down. It is often impractical/not worth being completely clean in this respect and Node.js -does a pretty good job only leaving in the order or 6k bytes that are +does a pretty good job only leaving on the order of 6KB that are not freed on shutdown. ## An obvious memory leak From 563f2ca485d4c6f816e5b564fbe2a53e73e020b4 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:20:18 -0500 Subject: [PATCH 09/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index cc63b9e6460ca1..4c4d2645eb0075 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -96,7 +96,7 @@ not freed on shutdown. ## An obvious memory leak Leaks can be introduced in native addons and the following is a simple -example generated by using the Hello world example in +example leak based on the "Hello world" addon from [node-addon-examples](https://github.com/nodejs/node-addon-examples). In that example a loop which allocates ~1MB of memory and never frees it From 18dd67fa2a72254d9d2ccda54eeef814497528f4 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:20:32 -0500 Subject: [PATCH 10/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 4c4d2645eb0075..58ac2ba12a0e2c 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -119,7 +119,7 @@ napi_value Method(napi_env env, napi_callback_info info) { } ``` -When trying to introduce a memory leak you need to ensure that +When trying to create a memory leak you need to ensure that the compiler has not optimized out the code you've introduced to generate the leak. For example, with assigning the result of malloc to either a global variable or a variable that is read the compiler will `optimize` From 21b117afc29a8f6b014b9b1e7b3c04fa62553aab Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:20:48 -0500 Subject: [PATCH 11/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 58ac2ba12a0e2c..58d5064121f109 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -120,7 +120,7 @@ napi_value Method(napi_env env, napi_callback_info info) { ``` When trying to create a memory leak you need to ensure that -the compiler has not optimized out the code you've introduced to generate +the compiler has not optimized out the code that creates the leak. For example, with assigning the result of malloc to either a global variable or a variable that is read the compiler will `optimize` out the loop with the malloc and valgrind does not report From fa59a67acbb4b07aa0c1e5eadf5ec950dd2086a0 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:21:07 -0500 Subject: [PATCH 12/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 58d5064121f109..b4c14ce3a0d521 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -99,7 +99,7 @@ Leaks can be introduced in native addons and the following is a simple example leak based on the "Hello world" addon from [node-addon-examples](https://github.com/nodejs/node-addon-examples). -In that example a loop which allocates ~1MB of memory and never frees it +In this example, a loop which allocates ~1MB of memory and never frees it has been added: ```C++ From 096ae1852f5b3008756fa288aa693fcc86b0ec02 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:21:20 -0500 Subject: [PATCH 13/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index b4c14ce3a0d521..a9b7ef6f7f7e52 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -34,7 +34,7 @@ apt-get install valgrind ``` ## Invocation -The simplest invocation of valgrind is simply +The simplest invocation of valgrind is: ```bash valgrind node test.js From 449ce6f7c691f776579a185595144a0b42afe048 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:21:36 -0500 Subject: [PATCH 14/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index a9b7ef6f7f7e52..eb08187892a1cc 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -121,7 +121,7 @@ napi_value Method(napi_env env, napi_callback_info info) { When trying to create a memory leak you need to ensure that the compiler has not optimized out the code that creates -the leak. For example, with assigning the result of malloc to either a +the leak. For example, assigning the result of `malloc()` to either a global variable or a variable that is read the compiler will `optimize` out the loop with the malloc and valgrind does not report a problem (since once does not exist in the code being run). From bcb0cc489f5948e103cd9413361cdb37f5233973 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:21:54 -0500 Subject: [PATCH 15/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index eb08187892a1cc..c1c2f0a1570f95 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -122,7 +122,7 @@ napi_value Method(napi_env env, napi_callback_info info) { When trying to create a memory leak you need to ensure that the compiler has not optimized out the code that creates the leak. For example, assigning the result of `malloc()` to either a -global variable or a variable that is read the compiler will `optimize` +global variable or a variable that is read the compiler will optimize out the loop with the malloc and valgrind does not report a problem (since once does not exist in the code being run). From 6f619068c7a29a903112e92aaba77f30be970bff Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:22:08 -0500 Subject: [PATCH 16/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index c1c2f0a1570f95..32ba424adbe266 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -124,7 +124,7 @@ the compiler has not optimized out the code that creates the leak. For example, assigning the result of `malloc()` to either a global variable or a variable that is read the compiler will optimize out the loop with the malloc and valgrind does not report -a problem (since once does not exist in the code being run). +a problem (since one no longer exists in the code being run). Running valgrind on the example program with this addition shows the following: From 9095593170971104ff0ac819c7dc557dc81fb61c Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:22:34 -0500 Subject: [PATCH 17/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 32ba424adbe266..295b8faf4038ce 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -126,7 +126,7 @@ global variable or a variable that is read the compiler will optimize out the loop with the malloc and valgrind does not report a problem (since one no longer exists in the code being run). -Running valgrind on the example program with this addition shows the following: +Running valgrind on this code shows the following: ```bash user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ valgrind node hello.js From e5db9f0d936b67fc7da602239f6406f4f15f7227 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:22:51 -0500 Subject: [PATCH 18/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 295b8faf4038ce..662fc072675278 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -176,7 +176,7 @@ world ``` Valgrind is reporting a problem as it shows 996,604 bytes as -definitely lost and the question is how to find where that memory is +definitely lost and the question is how to find where that memory was allocated. The next step is to rerun as suggested in the output with `--leak-check=full` From 769125c0d7fbc68746ab67dcea66fa09b063dfa0 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:23:02 -0500 Subject: [PATCH 19/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 662fc072675278..73784040e9d8cb 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -178,7 +178,7 @@ world Valgrind is reporting a problem as it shows 996,604 bytes as definitely lost and the question is how to find where that memory was allocated. The next step is to rerun as suggested in the -output with `--leak-check=full` +output with `--leak-check=full`: ``` bash user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ valgrind --leak-check=full node hello.js From 92115dbd4e4198affe130226c33400bf2d740824 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:23:16 -0500 Subject: [PATCH 20/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 73784040e9d8cb..6601c1697902d9 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -344,7 +344,7 @@ line number! >>>>> ==18481== by 0x9794989: Method(napi_env__*, napi_callback_info__*) (hello.cc:13) <<<<< ==18481== by 0x98F764: v8impl::(anonymous namespace)::FunctionCallbackWrapper::Invoke(v8::FunctionCallbackInfo const&) (in /home/user1/val grind/node-v12.14.1-linux-x64/bin/node) ==18481== by 0xBA6FC8: v8::internal::MaybeHandle v8::internal::(anonymous namespace)::HandleApiCallHelper(v8::internal:: Isolate*, v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, v8::internal::Handle, v8::internal::BuiltinArguments) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) -==18481== by 0xBA8DB6: v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) (in /home/user1/valgrind/node-v12.14.1-linux- x64/bin/node) +==18481== by 0xBA8DB6: v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) ==18481== by 0x1376358: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) ==18481== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) ==18481== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) From fc4094b6b10d7b66391d5d9d4ce502fa8dbece59 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:23:31 -0500 Subject: [PATCH 21/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 6601c1697902d9..edaf7006d2764a 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -336,7 +336,7 @@ a rebuild. The next step is to run valgrind after the rebuild. This time the information for the leaking location includes the name of the source file and the -line number! +line number: ```bash ==18481== 997,000 bytes in 997 blocks are definitely lost in loss record 35 of 35 From 629191bc2aa7f2342037c30c759846fb4d30a230 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:23:46 -0500 Subject: [PATCH 22/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index edaf7006d2764a..ebf2dba1d5f17c 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -354,7 +354,7 @@ line number: ==18481== by 0x12F68A3: ??? (in /home/user1/valgrind/node-v12.14.1-linux-x64/bin/node) ``` -which shows us exactly where the leak is occuring in the file hello.cc: +This new output shows us exactly where the leak is occurring in the file `hello.cc`: ```C++ 6 void* malloc_holder = nullptr; From da9e9ae9bacaacc35680cdf8953d981fe3f8c183 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:24:02 -0500 Subject: [PATCH 23/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index ebf2dba1d5f17c..49c459385d5ebc 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -357,7 +357,7 @@ line number: This new output shows us exactly where the leak is occurring in the file `hello.cc`: ```C++ -6 void* malloc_holder = nullptr; + 6 void* malloc_holder = nullptr; 7 napi_value Method(napi_env env, napi_callback_info info) { 8 napi_status status; 9 napi_value world; From 12fb16d3e5e220248f632f990afa7a9cdee7b5e1 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:24:16 -0500 Subject: [PATCH 24/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 49c459385d5ebc..a4c9c4ee35cb9a 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -364,7 +364,7 @@ This new output shows us exactly where the leak is occurring in the file `hello. 10 status = napi_create_string_utf8(env, "world", 5, &world); 11 assert(status == napi_ok); 12 for (int i=0; i< 1000; i++) { - 13 malloc_holder = malloc(1000); <<<<<< Yup this is where we are allocating the memory that is not freed + 13 malloc_holder = malloc(1000); // <<<<<< Yup this is where we are allocating the memory that is not freed 14 } 15 return world; 16 } From 073339d8a38e43532db3b881e3ea415bc21f6b09 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:24:32 -0500 Subject: [PATCH 25/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index a4c9c4ee35cb9a..e7e19bf67b8cff 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -372,7 +372,7 @@ This new output shows us exactly where the leak is occurring in the file `hello. ### Node.js binary -If the leak is not in an addon and in the Node.js binary itself you may need +If the leak is not in an addon and is instead in the Node.js binary itself, you may need to compile node yourself and turn on debug symbols. Looking at this entry reported by valgrind, with the release binary we see: From b31e10c89b379806236267dff57e287185e42fbf Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:24:45 -0500 Subject: [PATCH 26/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index e7e19bf67b8cff..5167b537cf36f3 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -391,7 +391,7 @@ reported by valgrind, with the release binary we see: ==4174== by 0x5BBFB96: (below main) (libc-start.c:310) ``` -This gives us some information of where to look (node::inspector::Agent::Start) +This gives us some information of where to look (`node::inspector::Agent::Start`) but not where in that function. We get more information that you might expect (or see by default with addons) because the Node.js binary exports many of its symbols using -rdynamic so that they can be used by addons. If the stack From 2a79264585e809149272009b3f23aaed925e5044 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:25:06 -0500 Subject: [PATCH 27/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 5167b537cf36f3..56fc4f9dcca483 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -374,7 +374,7 @@ This new output shows us exactly where the leak is occurring in the file `hello. If the leak is not in an addon and is instead in the Node.js binary itself, you may need to compile node yourself and turn on debug symbols. Looking at this entry -reported by valgrind, with the release binary we see: +reported by valgrind, with a release binary we see: ```bash 74== 304 bytes in 1 blocks are possibly lost in loss record 27 of 35 From 47f51791d21e7033f7f986d38bf8c95ed7361e1d Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:25:48 -0500 Subject: [PATCH 28/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 56fc4f9dcca483..a69c053ab8bf25 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -392,7 +392,7 @@ reported by valgrind, with a release binary we see: ``` This gives us some information of where to look (`node::inspector::Agent::Start`) -but not where in that function. We get more information that you might expect +but not where in that function. We get more information than you might expect (or see by default with addons) because the Node.js binary exports many of its symbols using -rdynamic so that they can be used by addons. If the stack gives you enough information to track down where the leak is great, From dbd1307cf89b3675a7014bcd25e6528a67e60254 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:26:22 -0500 Subject: [PATCH 29/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index a69c053ab8bf25..c937c6cb74c071 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -394,7 +394,7 @@ reported by valgrind, with a release binary we see: This gives us some information of where to look (`node::inspector::Agent::Start`) but not where in that function. We get more information than you might expect (or see by default with addons) because the Node.js binary exports many of -its symbols using -rdynamic so that they can be used by addons. If the stack +its symbols using `-rdynamic` so that they can be used by addons. If the stack gives you enough information to track down where the leak is great, otherwise the next step is to compile a Node.js binary with debugging enabled. From b40ff7e37423004d16b655c6aec974daa4f1f214 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:27:02 -0500 Subject: [PATCH 30/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index c937c6cb74c071..b2b1bafcbf0495 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -395,7 +395,7 @@ This gives us some information of where to look (`node::inspector::Agent::Start` but not where in that function. We get more information than you might expect (or see by default with addons) because the Node.js binary exports many of its symbols using `-rdynamic` so that they can be used by addons. If the stack -gives you enough information to track down where the leak is great, +gives you enough information to track down where the leak is, that's great, otherwise the next step is to compile a Node.js binary with debugging enabled. To get additional information with valgrind: From f0d2b43e98d56e5ec0b0bdcca21029da8cc37b0e Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:27:29 -0500 Subject: [PATCH 31/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index b2b1bafcbf0495..5696f6b3c11735 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -400,7 +400,7 @@ otherwise the next step is to compile a Node.js binary with debugging enabled. To get additional information with valgrind: -* check out the Node.js source corresponding to the release that you +* Check out the Node.js source corresponding to the release that you want to debug. For example ```bash git clone https://github.com/nodejs/node.git From fa454fcc967c78ab5bef5c8b581128c72ab8f66c Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:27:51 -0500 Subject: [PATCH 32/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 5696f6b3c11735..4bbbf9d3d5303f 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -406,7 +406,7 @@ To get additional information with valgrind: git clone https://github.com/nodejs/node.git git checkout -b v12.14.1 ``` -* compile with debug enabled (for addional info see [building a debug build](https://github.com/nodejs/node/blob/v12.14.1/BUILDING.md#building-a-debug-build) +* Compile with debug enabled (for additional info see [building a debug build](https://github.com/nodejs/node/blob/v12.14.1/BUILDING.md#building-a-debug-build)). For example, on *nix: ```bash ./configure --debug make -j4 From 769a9b38a8111eee5e5644e09cc74cae478360ff Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:28:14 -0500 Subject: [PATCH 33/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 4bbbf9d3d5303f..3a01b48091a92b 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -396,7 +396,7 @@ but not where in that function. We get more information than you might expect (or see by default with addons) because the Node.js binary exports many of its symbols using `-rdynamic` so that they can be used by addons. If the stack gives you enough information to track down where the leak is, that's great, -otherwise the next step is to compile a Node.js binary with debugging enabled. +otherwise the next step is to compile a debug build of Node.js. To get additional information with valgrind: From 1e243d6941ee77b1fc9a2975ef5bcaefe8f30b19 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:28:36 -0500 Subject: [PATCH 34/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 3a01b48091a92b..87bb3b6f15d7e1 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -401,7 +401,7 @@ otherwise the next step is to compile a debug build of Node.js. To get additional information with valgrind: * Check out the Node.js source corresponding to the release that you - want to debug. For example + want to debug. For example: ```bash git clone https://github.com/nodejs/node.git git checkout -b v12.14.1 From 599f71bfe7f15b7b0f09ebb1e031d588c7631717 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:28:58 -0500 Subject: [PATCH 35/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 87bb3b6f15d7e1..f9b3822d51ea76 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -411,7 +411,7 @@ git checkout -b v12.14.1 ./configure --debug make -j4 ``` -* make sure to run with your compiled debug version of Node.js. Having used +* Make sure to run with your compiled debug version of Node.js. Having used `./configure --debug`, two binaries will have been built when make was run. You must use the one which is in .../out/Debug. Add that version of the binary to your path. For example: From 73842f0434a7a9baa04e96e644dcf2cce7e16839 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:29:19 -0500 Subject: [PATCH 36/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index f9b3822d51ea76..1707fbb4b4a87c 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -412,7 +412,7 @@ git checkout -b v12.14.1 make -j4 ``` * Make sure to run with your compiled debug version of Node.js. Having used - `./configure --debug`, two binaries will have been built when make was run. + `./configure --debug`, two binaries will have been built when `make` was run. You must use the one which is in .../out/Debug. Add that version of the binary to your path. For example: From 74f9fe20e0002d8050ca64fee45a02e7cb1101b0 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:29:37 -0500 Subject: [PATCH 37/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 1707fbb4b4a87c..aa85d68c19c3d6 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -413,7 +413,7 @@ make -j4 ``` * Make sure to run with your compiled debug version of Node.js. Having used `./configure --debug`, two binaries will have been built when `make` was run. - You must use the one which is in .../out/Debug. Add that version of the + You must use the one which is in `out/Debug`. Add that version of the binary to your path. For example: ```bash From 1934ca012ef0044a48f29d9f107f960a4a3bd149 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:29:59 -0500 Subject: [PATCH 38/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index aa85d68c19c3d6..0cfb638d0d83da 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -420,7 +420,7 @@ make -j4 export PATH=/home/user1/node/out/Debug/:$PATH ``` -Running valgrind using the Node.js binary with debug symbols enabled shows: +Running valgrind using the debug build of Node.js shows: ```bash ==44112== 592 bytes in 1 blocks are possibly lost in loss record 26 of 27 From a2d4c2c2b8af82fce57b3e1d083405441f95bddf Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:30:45 -0500 Subject: [PATCH 39/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 0cfb638d0d83da..f174e4549d3b28 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -437,5 +437,5 @@ Running valgrind using the debug build of Node.js shows: ``` -where we can see the specific file name and line in the Node.js code which +Now we can see the specific file name and line in the Node.js code which caused the allocation (inspector_agent.cc:140). From 2001796eb5141419c08f93c6e34283dd696b73f2 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:31:27 -0500 Subject: [PATCH 40/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: mscdex --- doc/guides/investigating_native_memory_leak.md | 1 - 1 file changed, 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index f174e4549d3b28..c69eda75abbfbf 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -434,7 +434,6 @@ Running valgrind using the debug build of Node.js shows: ==44112== by 0xEE8831: node::NodeMainInstance::Run() (node_main_instance.cc:108) ==44112== by 0xE3CDEC: node::Start(int, char**) (node.cc:996) ==44112== by 0x22D8BBF: main (node_main.cc:126) - ``` Now we can see the specific file name and line in the Node.js code which From 2f61dfcf30e9c58ce42a0e03d1a4e46e92b33801 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:32:55 -0500 Subject: [PATCH 41/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Anna Henningsen --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index c69eda75abbfbf..8f300e46d65a86 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -1,7 +1,7 @@ # Investigating Memory Leaks with valgrind A Node.js process may run out of memory due to excessive consumption of -`Native memory`. `Native Memory` is memory which is not managed by the the +native memory. Native Memory is memory which is not managed by the V8 Garbage collector and is allocated either by the Node.js runtime, it's dependencies or native [addons](://nodejs.org/docs/latest/api/n-api.html). From 627691cc8d1fcbd96993d127a7dbd406efae6b1c Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:33:31 -0500 Subject: [PATCH 42/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Rich Trott --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 8f300e46d65a86..f548ced8cc70b3 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -422,7 +422,7 @@ export PATH=/home/user1/node/out/Debug/:$PATH Running valgrind using the debug build of Node.js shows: -```bash +```console ==44112== 592 bytes in 1 blocks are possibly lost in loss record 26 of 27 ==44112== at 0x4C2BF79: calloc (vg_replace_malloc.c:762) ==44112== by 0x4012754: _dl_allocate_tls (in /usr/lib64/ld-2.17.so) From 0a5c751ce5366fc77f13d9b5cf8aa95f948bd00c Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:34:03 -0500 Subject: [PATCH 43/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Anna Henningsen --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index f548ced8cc70b3..53f296e2145bfb 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -2,7 +2,7 @@ A Node.js process may run out of memory due to excessive consumption of native memory. Native Memory is memory which is not managed by the -V8 Garbage collector and is allocated either by the Node.js runtime, it's +V8 Garbage collector and is allocated either by the Node.js runtime, its dependencies or native [addons](://nodejs.org/docs/latest/api/n-api.html). This guide provides information on how to use valgrind to investigate these From 323947c37c6165d61dafc3b90225b668d8a447c8 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:34:53 -0500 Subject: [PATCH 44/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Anna Henningsen --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 53f296e2145bfb..3ebde1b52f191e 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -11,7 +11,7 @@ issues. ## valgrind [Valgrind](https://valgrind.org/docs/manual/quick-start.html) is a -tool available on linux distributions which can be used to investigate +tool available on Linux distributions which can be used to investigate memory usage including identifying memory leaks (memory which is allocated and not freed) and other memory related problems like double freeing memory. From 0a29d6bc02f4767620f18252acee7fcf03db883e Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:35:31 -0500 Subject: [PATCH 45/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Anna Henningsen --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 3ebde1b52f191e..20499cb04da381 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -111,7 +111,7 @@ napi_value Method(napi_env env, napi_callback_info info) { assert(status == napi_ok); // NEW LEAK HERE - for (int i=0; i< 1024; i++) { + for (int i=0; i < 1024; i++) { malloc_holder = malloc(1024); } From e87c6a9b625986a42167ec904e6b8839e886850c Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:35:54 -0500 Subject: [PATCH 46/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Anna Henningsen --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 20499cb04da381..84eab542da9c4c 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -319,7 +319,7 @@ information. Leaks may be either in addons or Node.js itself. The sections which follow cover the steps needed to enable debug symbols to get more info. -### addons +### Native addons To enable debug symbols for all of your addons that are compiled on install simply use: From 819ca8136b7fe093ac209ef1125aa64ed0367a54 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:36:23 -0500 Subject: [PATCH 47/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Anna Henningsen --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 84eab542da9c4c..073bd3c0301481 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -377,7 +377,7 @@ to compile node yourself and turn on debug symbols. Looking at this entry reported by valgrind, with a release binary we see: ```bash -74== 304 bytes in 1 blocks are possibly lost in loss record 27 of 35 + ==4174== 304 bytes in 1 blocks are possibly lost in loss record 27 of 35 ==4174== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==4174== by 0x40134A6: allocate_dtv (dl-tls.c:286) ==4174== by 0x40134A6: _dl_allocate_tls (dl-tls.c:530) From 52eff0e8b34cd8438ff8e4d9902bfb3748aa94a0 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:37:37 -0500 Subject: [PATCH 48/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Rich Trott --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 073bd3c0301481..d9cf6e4c27f8e5 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -29,7 +29,7 @@ To use valgrind: It is an optional package in most cases and must be installed explicitly. For example on Debian/Ubuntu: -```bash +```console apt-get install valgrind ``` From 18f4ac1f6cbb598f8c798fccd15ceadb52b16777 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:38:03 -0500 Subject: [PATCH 49/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Rich Trott --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index d9cf6e4c27f8e5..0c141115953ae2 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -36,7 +36,7 @@ apt-get install valgrind ## Invocation The simplest invocation of valgrind is: -```bash +```console valgrind node test.js ``` From 7172ef56a9bcca767ec713cb35981c7e004b0456 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:38:25 -0500 Subject: [PATCH 50/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Rich Trott --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 0c141115953ae2..fd398f59bc2fc8 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -42,7 +42,7 @@ valgrind node test.js with the output being: -```bash +```console user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ valgrind test.js valgrind: test.js: command not found user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ valgrind node test.js From c5e0150430a96a00415df03d791d1c0571995b06 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:38:49 -0500 Subject: [PATCH 51/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Rich Trott --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index fd398f59bc2fc8..19fa8b2c7ff0e6 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -128,7 +128,7 @@ a problem (since one no longer exists in the code being run). Running valgrind on this code shows the following: -```bash +```console user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ valgrind node hello.js ==1504== Memcheck, a memory error detector ==1504== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. From 7fd1928c635a33bb045be012e27d23494c971412 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:39:09 -0500 Subject: [PATCH 52/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Rich Trott --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 19fa8b2c7ff0e6..3fdbd093bb80ca 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -287,7 +287,7 @@ user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ This is the most interesting report: -```bash +```console ==4174== 997,000 bytes in 997 blocks are definitely lost in loss record 35 of 35 ==4174== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==4174== by 0x9794979: Method(napi_env__*, napi_callback_info__*) (in /home/user1/valgrind/node-addon-examples/1_hello_world/napi/build/Release/hello.node) From e961ea191870e46914d6531ca0a363919ae7c06e Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:39:31 -0500 Subject: [PATCH 53/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Rich Trott --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 3fdbd093bb80ca..9b2bca5facd4d7 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -305,7 +305,7 @@ This is the most interesting report: From the stack trace we can tell that the leak came from a native addon: -```bash +```console ==4174== by 0x9794979: Method(napi_env__*, napi_callback_info__*) (in /home/user1/valgrind/node-addon-examples/1_hello_world/napi/build/Release/hello.node) ``` From e7392f5953eb3cecc5e2ee8565f42d5eca17fae4 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:39:53 -0500 Subject: [PATCH 54/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Rich Trott --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 9b2bca5facd4d7..e24ba0113a7e9a 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -324,7 +324,7 @@ follow cover the steps needed to enable debug symbols to get more info. To enable debug symbols for all of your addons that are compiled on install simply use: -```bash +```console npm install --debug ``` From fa2c7a307c852fdf6b982e39c710990e4f16a66f Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:40:17 -0500 Subject: [PATCH 55/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Rich Trott --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index e24ba0113a7e9a..6c0afd9bb875cb 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -338,7 +338,7 @@ The next step is to run valgrind after the rebuild. This time the information for the leaking location includes the name of the source file and the line number: -```bash +```console ==18481== 997,000 bytes in 997 blocks are definitely lost in loss record 35 of 35 ==18481== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) >>>>> ==18481== by 0x9794989: Method(napi_env__*, napi_callback_info__*) (hello.cc:13) <<<<< From 1370c461166ed840b4e2cfbf7b73a856dc2222ce Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:40:38 -0500 Subject: [PATCH 56/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Rich Trott --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 6c0afd9bb875cb..96eb60c0d58e9c 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -376,7 +376,7 @@ If the leak is not in an addon and is instead in the Node.js binary itself, you to compile node yourself and turn on debug symbols. Looking at this entry reported by valgrind, with a release binary we see: -```bash +```console ==4174== 304 bytes in 1 blocks are possibly lost in loss record 27 of 35 ==4174== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==4174== by 0x40134A6: allocate_dtv (dl-tls.c:286) From d925433f2f60912c0ca79b8f99707eb281b25aac Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:40:57 -0500 Subject: [PATCH 57/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Rich Trott --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 96eb60c0d58e9c..83ef104284131b 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -402,7 +402,7 @@ To get additional information with valgrind: * Check out the Node.js source corresponding to the release that you want to debug. For example: -```bash +```console git clone https://github.com/nodejs/node.git git checkout -b v12.14.1 ``` From a39456f4676265656f27e11f31836ff9a54cc6c8 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:41:18 -0500 Subject: [PATCH 58/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Rich Trott --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 83ef104284131b..d513137fd3da78 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -407,7 +407,7 @@ git clone https://github.com/nodejs/node.git git checkout -b v12.14.1 ``` * Compile with debug enabled (for additional info see [building a debug build](https://github.com/nodejs/node/blob/v12.14.1/BUILDING.md#building-a-debug-build)). For example, on *nix: -```bash +```console ./configure --debug make -j4 ``` From 8d901397e636daeb55408d8bc9fd151275e632f4 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:41:40 -0500 Subject: [PATCH 59/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Rich Trott --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index d513137fd3da78..783a73bc4e3c32 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -416,7 +416,7 @@ make -j4 You must use the one which is in `out/Debug`. Add that version of the binary to your path. For example: -```bash +```console export PATH=/home/user1/node/out/Debug/:$PATH ``` From ff189a20a24194cf0012b182303012ba6a8d70c3 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:42:23 -0500 Subject: [PATCH 60/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Colin Ihrig --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 783a73bc4e3c32..a4f3737dedec62 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -322,7 +322,7 @@ follow cover the steps needed to enable debug symbols to get more info. ### Native addons To enable debug symbols for all of your addons that are compiled on -install simply use: +install use: ```console npm install --debug From 1b73672fd8d078de7da6d075f1d66ce0a3dfdcca Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:45:09 -0500 Subject: [PATCH 61/73] squash: address comments --- doc/guides/investigating_native_memory_leak.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index a4f3737dedec62..63aeb67ffea287 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -43,8 +43,6 @@ valgrind node test.js with the output being: ```console -user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ valgrind test.js -valgrind: test.js: command not found user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ valgrind node test.js ==28993== Memcheck, a memory error detector ==28993== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. @@ -282,7 +280,6 @@ world ==4174== For counts of detected and suppressed errors, rerun with: -v ==4174== Use --track-origins=yes to see where uninitialised values come from ==4174== ERROR SUMMARY: 5 errors from 5 contexts (suppressed: 0 from 0) -user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ ``` This is the most interesting report: @@ -364,7 +361,7 @@ This new output shows us exactly where the leak is occurring in the file `hello. 10 status = napi_create_string_utf8(env, "world", 5, &world); 11 assert(status == napi_ok); 12 for (int i=0; i< 1000; i++) { - 13 malloc_holder = malloc(1000); // <<<<<< Yup this is where we are allocating the memory that is not freed + 13 malloc_holder = malloc(1000); // <<<<<< This is where we are allocating the memory that is not freed 14 } 15 return world; 16 } From 198334677e7b27572643f4bfcf8b9d7237701321 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:47:32 -0500 Subject: [PATCH 62/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Anna Henningsen --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 63aeb67ffea287..9d6b09facb452d 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -401,7 +401,7 @@ To get additional information with valgrind: want to debug. For example: ```console git clone https://github.com/nodejs/node.git -git checkout -b v12.14.1 +git checkout v12.14.1 ``` * Compile with debug enabled (for additional info see [building a debug build](https://github.com/nodejs/node/blob/v12.14.1/BUILDING.md#building-a-debug-build)). For example, on *nix: ```console From 09d1ea6bcaf84043ca8b1ef446d3a3d2ab0c8494 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 13:50:18 -0500 Subject: [PATCH 63/73] squash: address comments --- doc/guides/investigating_native_memory_leak.md | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 9d6b09facb452d..46f56d421b87dd 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -410,12 +410,7 @@ make -j4 ``` * Make sure to run with your compiled debug version of Node.js. Having used `./configure --debug`, two binaries will have been built when `make` was run. - You must use the one which is in `out/Debug`. Add that version of the - binary to your path. For example: - -```console -export PATH=/home/user1/node/out/Debug/:$PATH -``` + You must use the one which is in `out/Debug`. Running valgrind using the debug build of Node.js shows: From 3252c83b7d5a213e17b6bf63c6c5db0471d6ea2d Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Mon, 27 Jan 2020 14:01:31 -0500 Subject: [PATCH 64/73] squash: fix linter issues introduced by comments --- doc/guides/investigating_native_memory_leak.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 46f56d421b87dd..021fc4caecfe07 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -18,16 +18,16 @@ like double freeing memory. To use valgrind: -* Be patient, running under valgrind slows execution significantly due - to the checks being performed. +* Be patient, running under valgrind slows execution significantly + due to the checks being performed. * Reduce your test case to the smallest reproduce. Due to the slowdown it is important to run the minimum test case in order to be able to do it in a reasonable time. ## Installation -It is an optional package in most cases and must be installed explicitly. For example on -Debian/Ubuntu: +It is an optional package in most cases and must be installed explicitly. +For example on Debian/Ubuntu: ```console apt-get install valgrind @@ -369,9 +369,9 @@ This new output shows us exactly where the leak is occurring in the file `hello. ### Node.js binary -If the leak is not in an addon and is instead in the Node.js binary itself, you may need -to compile node yourself and turn on debug symbols. Looking at this entry -reported by valgrind, with a release binary we see: +If the leak is not in an addon and is instead in the Node.js binary itself, +you may need to compile node yourself and turn on debug symbols. Looking at +this entry reported by valgrind, with a release binary we see: ```console ==4174== 304 bytes in 1 blocks are possibly lost in loss record 27 of 35 @@ -403,7 +403,9 @@ To get additional information with valgrind: git clone https://github.com/nodejs/node.git git checkout v12.14.1 ``` -* Compile with debug enabled (for additional info see [building a debug build](https://github.com/nodejs/node/blob/v12.14.1/BUILDING.md#building-a-debug-build)). For example, on *nix: +* Compile with debug enabled (for additional info see +[building a debug build](https://github.com/nodejs/node/blob/v12.14.1/BUILDING.md#building-a-debug-build)). +For example, on *nix: ```console ./configure --debug make -j4 From e7ad36a260ee1b97b8a2f9d8e04328b17dc45d3e Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Thu, 30 Jan 2020 18:04:09 -0500 Subject: [PATCH 65/73] squash: address comments --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 021fc4caecfe07..055390765744a0 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -3,7 +3,7 @@ A Node.js process may run out of memory due to excessive consumption of native memory. Native Memory is memory which is not managed by the V8 Garbage collector and is allocated either by the Node.js runtime, its -dependencies or native [addons](://nodejs.org/docs/latest/api/n-api.html). +dependencies or native [addons](https://nodejs.org/docs/latest/api/n-api.html). This guide provides information on how to use valgrind to investigate these issues. From aeb5b9374744718040f3fe7c37ad37baba041fb8 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Fri, 31 Jan 2020 16:37:44 -0500 Subject: [PATCH 66/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Denys Otrishko <9109612+lundibundi@users.noreply.github.com> --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 055390765744a0..7bd48b25439a48 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -408,7 +408,7 @@ git checkout v12.14.1 For example, on *nix: ```console ./configure --debug -make -j4 +make -j4 ``` * Make sure to run with your compiled debug version of Node.js. Having used `./configure --debug`, two binaries will have been built when `make` was run. From 70b9c40321ad726edc864bef9b64a06d4bbf75c7 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Fri, 31 Jan 2020 16:38:10 -0500 Subject: [PATCH 67/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Denys Otrishko <9109612+lundibundi@users.noreply.github.com> --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 7bd48b25439a48..cc6931db03c62c 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -282,7 +282,7 @@ world ==4174== ERROR SUMMARY: 5 errors from 5 contexts (suppressed: 0 from 0) ``` -This is the most interesting report: +This is the most interesting part of the report: ```console ==4174== 997,000 bytes in 997 blocks are definitely lost in loss record 35 of 35 From eafd4287c7a32a7a2f12c5475d9d8827bb883e73 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Fri, 31 Jan 2020 16:38:46 -0500 Subject: [PATCH 68/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Denys Otrishko <9109612+lundibundi@users.noreply.github.com> --- doc/guides/investigating_native_memory_leak.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index cc6931db03c62c..05fe0a200cecd5 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -331,6 +331,11 @@ results in the addons being compiled with the debug option. If the native addon contains pre-built binaries you will need to force a rebuild. +```console +npm install --debug +npm rebuild +``` + The next step is to run valgrind after the rebuild. This time the information for the leaking location includes the name of the source file and the line number: From 089d906a2bf4d0693f5051d4d2a8202d27ce9fbe Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Fri, 31 Jan 2020 16:38:57 -0500 Subject: [PATCH 69/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Denys Otrishko <9109612+lundibundi@users.noreply.github.com> --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 05fe0a200cecd5..4acdb63d443e7e 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -122,7 +122,7 @@ the compiler has not optimized out the code that creates the leak. For example, assigning the result of `malloc()` to either a global variable or a variable that is read the compiler will optimize out the loop with the malloc and valgrind does not report -a problem (since one no longer exists in the code being run). +a problem (since it no longer exists in the code being run). Running valgrind on this code shows the following: From b5466807da664c162c2361f0cd05fa7fe6f06f05 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Fri, 31 Jan 2020 16:46:00 -0500 Subject: [PATCH 70/73] squash: address comments --- .../investigating_native_memory_leak.md | 22 +++++++++++++------ 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 4acdb63d443e7e..c9550db72e1e8e 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -87,9 +87,12 @@ user1@minikube1:~/valgrind/node-addon-examples/1_hello_world/napi$ valgrind node This reports that Node.js is not _completely_ clean as there is some memory that was allocated but not freed when the process shut down. It is often -impractical/not worth being completely clean in this respect and Node.js -does a pretty good job only leaving on the order of 6KB that are -not freed on shutdown. +impractical/not worth being completely clean in this respect. Modern +operating systems will clean up the memory of the process after the +shutdown while attemping to fre all memory to get a clean +report may have a negative impact on the code complexity and +shutdown times. Node.js does a pretty good job only leaving on +the order of 6KB that are not freed on shutdown. ## An obvious memory leak @@ -119,10 +122,15 @@ napi_value Method(napi_env env, napi_callback_info info) { When trying to create a memory leak you need to ensure that the compiler has not optimized out the code that creates -the leak. For example, assigning the result of `malloc()` to either a -global variable or a variable that is read the compiler will optimize -out the loop with the malloc and valgrind does not report -a problem (since it no longer exists in the code being run). +the leak. For example, by assigning the result of the allocation +to either a global variable or a variable that will be read +afterwards the compiler will not optimize it out along with +the malloc and Valgrind will properly report the memory leak. +If `malloc_holder` in the example above is made into a +local variable then the compiler may freely remove +it along with the allocations (since it is not used) +and Valgrind will not find any leaks since they +will no longer exist in the code being run. Running valgrind on this code shows the following: From 70e831ce666ea4c49a1880d7e5026a7027f5b783 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Fri, 31 Jan 2020 16:49:12 -0500 Subject: [PATCH 71/73] squash: fix linting issues introduced by suggestions --- doc/guides/investigating_native_memory_leak.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index c9550db72e1e8e..04c5bfbb3baa31 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -92,7 +92,7 @@ operating systems will clean up the memory of the process after the shutdown while attemping to fre all memory to get a clean report may have a negative impact on the code complexity and shutdown times. Node.js does a pretty good job only leaving on -the order of 6KB that are not freed on shutdown. +the order of 6KB that are not freed on shutdown. ## An obvious memory leak @@ -125,7 +125,7 @@ the compiler has not optimized out the code that creates the leak. For example, by assigning the result of the allocation to either a global variable or a variable that will be read afterwards the compiler will not optimize it out along with -the malloc and Valgrind will properly report the memory leak. +the malloc and Valgrind will properly report the memory leak. If `malloc_holder` in the example above is made into a local variable then the compiler may freely remove it along with the allocations (since it is not used) From 7f8ddeba4c3b0adef9e7d8c8656740024aea499a Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Wed, 12 Feb 2020 12:34:41 -0500 Subject: [PATCH 72/73] Update doc/guides/investigating_native_memory_leak.md Co-Authored-By: Denys Otrishko --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 04c5bfbb3baa31..190ec2f2a5587f 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -89,7 +89,7 @@ This reports that Node.js is not _completely_ clean as there is some memory that was allocated but not freed when the process shut down. It is often impractical/not worth being completely clean in this respect. Modern operating systems will clean up the memory of the process after the -shutdown while attemping to fre all memory to get a clean +shutdown while attempting to free all memory to get a clean report may have a negative impact on the code complexity and shutdown times. Node.js does a pretty good job only leaving on the order of 6KB that are not freed on shutdown. From d1773b2f83dd770a9cc7167d4dbb6f5bf1eac0a1 Mon Sep 17 00:00:00 2001 From: Michael Dawson Date: Fri, 14 Feb 2020 16:43:12 -0500 Subject: [PATCH 73/73] Update doc/guides/investigating_native_memory_leak.md --- doc/guides/investigating_native_memory_leak.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guides/investigating_native_memory_leak.md b/doc/guides/investigating_native_memory_leak.md index 190ec2f2a5587f..366cc2917f6a4c 100644 --- a/doc/guides/investigating_native_memory_leak.md +++ b/doc/guides/investigating_native_memory_leak.md @@ -6,7 +6,7 @@ V8 Garbage collector and is allocated either by the Node.js runtime, its dependencies or native [addons](https://nodejs.org/docs/latest/api/n-api.html). This guide provides information on how to use valgrind to investigate these -issues. +issues on Linux platforms. ## valgrind