Memory leaks in WASM

[oblique]

Currently there are memory leaks in WASM and it looks like a limitation of WASM, but we need further investigation.

Useful information:

• Firefox has a about:memory which shows some extra information about the worker
• Firefox Nightly has a DMD mode, which can be used for memory profiling. Run DTD=1 ./firefox in nightly's directory.
• Stack Overflow: How can I free memory allocated by Rust code exposed in WebAssembly?
• Wasm needs a better memory management story WebAssembly/design#1397
• Check wasm-pack's --weak-refs and --reference-types
• wasm-bindgen - Internal Design - JS Objects in Rust

[oblique]

I just ruled out that this could be a memory fragmentation on the Rust side.

Rust allocates a single continuous memory region my using memory_grow instruction. That memory region is then used by dlmalloc to provide Rust's global allocation. When dlmalloc uses all the memory of that region, it calls memory_grow again to allocate more space. Whatever is allocated by memory_grow can not be freed. I used memory_size to check how much memory was purely allocated for Rust.

After an hour or so, browser's task manager was showing 3GB of memory consumption by Lumina tab. However the memory region that was used by Rust was at 103mb, almost from the beginning of the syncing. This concludes that memory leaks are tightly related to JavaScript's types.

[oblique]

After reading JS Objects in Rust from wasm-bindgen's internal design documentation I ruled out the posibility that memory leak is happening because of Rust not freeing JS objects.

I was printing wasm_bindgen::externref_heap_live_count every 1 minute. In case of memory leak, that value should have constantly increasing to a huge number over time, which didn't happen.

---

After ruling out the above, I couldn't understand why memory consumption of Lumina's tab was increasing and even becoming more than 3GB.

Throughout all my investigation I was using Allocation instrumentation on timeline of Chrome, trying to pin point the leak. While doing that I was ignoring Performance, PerformanceMark, and PerformanceMeasure from the allocations, thinking that they existed because of the instrumentation.

I started checking random things to find any other clues about the leak, including of reading code of wasm-bindgen generated file lumina_node_wasm.js. Then I notice this:

    imports.wbg.__wbg_mark_6045ef1772587264 = function() { return handleError(function (arg0, arg1, arg2) {
        getObject(arg0).mark(getStringFromWasm0(arg1, arg2));
    }, arguments) };
    imports.wbg.__wbg_mark_bad820680b8580c2 = function() { return handleError(function (arg0, arg1, arg2, arg3) {
        getObject(arg0).mark(getStringFromWasm0(arg1, arg2), getObject(arg3));
    }, arguments) };
    imports.wbg.__wbg_measure_1d846b814d43d7e1 = function() { return handleError(function (arg0, arg1, arg2, arg3, arg4, arg5, arg6) {
        getObject(arg0).measure(getStringFromWasm0(arg1, arg2), getStringFromWasm0(arg3, arg4), getStringFromWasm0(arg5, arg6));
    }, arguments) };
    imports.wbg.__wbg_measure_7ca0e5cfef892340 = function() { return handleError(function (arg0, arg1, arg2, arg3) {
        getObject(arg0).measure(getStringFromWasm0(arg1, arg2), getObject(arg3));
    }, arguments) };
    imports.wbg.__wbg_performance_72f95fe5952939b5 = function() {
        const ret = globalThis.performance;
        return addHeapObject(ret);
    };

Which indicates that our Rust code at some point is using Performance.mark() and Performance.measure(). I started looking on our dependencies and found that tracing-web is using it in its performance layer. After checking our initialization of tracing-web, I realized that we are actually enabling the performance layer, so I commented it out.

After doing that, memory consumption stopped increasing to that huge numbers! 🥳

Note that this is not really a memory leak, but it is how performance marks work.