asynchronous drawImage speeds up execution but causes serious memory leaks. #890
Comments
|
After several tests on my side and after proceeding by elimination, I confirm I have the same issue. |
|
I have the same issue |
|
Lines 417 to 424 in 8ca3a9e // 'static pointer
let self_mut = unsafe { Box::leak(Box::from_raw(image_ptr.cast::<Image>())) }; |
@Brooooooklyn since I have limited understanding of #[napi(setter)]
pub fn set_src(&mut self, env: Env, this: This, data: Uint8Array) -> Result<()> {
let length = data.len();
if length <= 2 {
self.src = Some(data);
+ self.bitmap = None;
+ self.width = -1.0;
+ self.height = -1.0;Of course, I have my own motives, as it just happens to meet another need of mine. #868 |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
|
@Brooooooklyn Hi, are there any progress on this issue, it's a serious problem on server, after a few requests, the server memory usage is full. |
|
@Brooooooklyn I tested and found that the memory usage has not decreased for a long time. This is my test code. import { clearAllCache, createCanvas, loadImage } from "@napi-rs/canvas";
const imgBuf = await fetch(
"https://cdn-ms.17zuoye.cn/zx-ptqlm/testing_2023_10_27/pdf_6d1e8d355613ef824d_0.webp",
).then((a) => a.arrayBuffer());
const len = ~~(process.argv[2]??50);
console.time("drawImage");
const res = await Promise.all(
Array(len)
.fill(0)
.map(async () => {
// console.time(i);
const img = await loadImage(imgBuf);
const canvas = createCanvas(img.width, img.height);
canvas.getContext("2d").drawImage(img, 0, 0, img.width, img.height);
await canvas.encode("webp", 80);
// console.timeEnd(i);
}),
);
console.timeEnd("drawImage");
setInterval(() => {
global.gc?.();
clearAllCache();
console.log(process.memoryUsage.rss() / 1024 / 1024);
// decreasing array length
res.splice(0, 10);
}, 1000);
|
|
If I reuse the script from @rambo-panda , the fix doesn't seem to work |
Why do you think it doesn't work? The |
Because in my script I generate GIFs (around 5 each time) However, the script is written in TypeScript, so the promises are properly resolved But in Chrome's inspector, when using the |
|
I had also modified Panda's script by logging the RSS by default before logging it at intervals |
|
I tested |
|
@bingtsingw can't reproduce the memory leak with this: import { whiteBright, red, green, gray } from 'colorette'
import prettyBytes from 'pretty-bytes'
import { table } from 'table'
import { clearAllCache, createCanvas, loadImage } from "./index.js";
const imgBuf = await fetch(
"https://cdn-ms.17zuoye.cn/zx-ptqlm/testing_2023_10_27/pdf_6d1e8d355613ef824d_0.webp",
).then((a) => a.arrayBuffer());
const initialMemoryUsage = process.memoryUsage()
const len = ~~(process.argv[2] ?? 50);
console.time("drawImage");
const res = await Promise.all(
Array(len)
.fill(0)
.map(async () => {
// console.time(i);
const img = await loadImage(imgBuf);
const canvas = createCanvas(img.width, img.height);
canvas.getContext("2d").drawImage(img, 0, 0, img.width, img.height);
await canvas.encode("webp", 80);
// console.timeEnd(i);
}),
);
console.timeEnd("drawImage");
function displayMemoryUsageFromNode(initialMemoryUsage) {
const finalMemoryUsage = process.memoryUsage()
const titles = Object.keys(initialMemoryUsage).map((k) => whiteBright(k))
const tableData = [titles]
const diffColumn = []
for (const [key, value] of Object.entries(initialMemoryUsage)) {
const diff = finalMemoryUsage[key] - value
const prettyDiff = prettyBytes(diff, { signed: true })
if (diff > 0) {
diffColumn.push(red(prettyDiff))
} else if (diff < 0) {
diffColumn.push(green(prettyDiff))
} else {
diffColumn.push(gray(prettyDiff))
}
}
tableData.push(diffColumn)
console.info(table(tableData))
}
setInterval(() => {
global.gc?.();
clearAllCache();
displayMemoryUsageFromNode(initialMemoryUsage);
// decreasing array length
res.splice(0, 10);
}, 1000);And the output: |
|
@Brooooooklyn Thank you for you test script, I'll use this method to test my image. |
|
I have the same issue, noticed it while periodically checking |
|
when I run it in a container on Ubuntu 22 LTS, there is a severe memory leak. -import { clearAllCache, createCanvas, loadImage } from "./index.js";
+import { clearAllCache, createCanvas, loadImage } from '@napi-rs/canvas';
+
const imgBuf = await fetch(
"https://cdn-ms.17zuoye.cn/zx-ptqlm/testing_2023_10_27/pdf_6d1e8d355613ef824d_0.webp",
-).then((a) => a.arrayBuffer());
+).then((a) => a.arrayBuffer()).then(Buffer.from);
const initialMemoryUsage = process.memoryUsage()
@@ -45,6 +46,7 @@ function displayMemoryUsageFromNode(initialMemoryUsage) {
}
tableData.push(diffColumn)
console.info(table(tableData))
+ console.info("current RSS value", process.memoryUsage.rss());
}
after 1min |
|
@rambo-panda your memory leak was caused by |
@Brooooooklyn sorry, I didn't understand. Even if I try to download the image locally and then use |
@Brooooooklyn This script is what I used to test it, I still get a severe memory leak with this, in my comment earlier I loaded a different image via import { whiteBright, red, green, gray } from 'colorette';
import prettyBytes from 'pretty-bytes';
import { table } from 'table';
import { clearAllCache, createCanvas, loadImage } from "@napi-rs/canvas";
const imgBuf = await fetch(
"https://cdn-ms.17zuoye.cn/zx-ptqlm/testing_2023_10_27/pdf_6d1e8d355613ef824d_0.webp",
).then((a) => a.arrayBuffer());
// Tried this too, same result
// import { readFile } from 'node:fs/promises';
// const imgBuf = await readFile("pdf_6d1e8d355613ef824d_0.webp");
const initialMemoryUsage = process.memoryUsage();
const len = ~~(process.argv[2] ?? 50);
console.time("drawImage");
const res = await Promise.all(
Array(len)
.fill(0)
.map(async () => {
// console.time(i);
const img = await loadImage(imgBuf);
const canvas = createCanvas(img.width, img.height);
canvas.getContext("2d").drawImage(img, 0, 0, img.width, img.height);
await canvas.encode("webp", 80);
// console.timeEnd(i);
}),
);
console.timeEnd("drawImage");
function displayMemoryUsageFromNode(initialMemoryUsage) {
const finalMemoryUsage = process.memoryUsage();
const titles = Object.keys(initialMemoryUsage).map((k) => whiteBright(k));
const tableData = [titles];
const diffColumn = [];
for (const [key, value] of Object.entries(initialMemoryUsage)) {
const diff = finalMemoryUsage[key] - value;
const prettyDiff = prettyBytes(diff, { signed: true });
if (diff > 0) {
diffColumn.push(red(prettyDiff));
} else if (diff < 0) {
diffColumn.push(green(prettyDiff));
} else {
diffColumn.push(gray(prettyDiff));
}
}
tableData.push(diffColumn);
console.info(table(tableData));
}
setInterval(() => {
global.gc?.();
clearAllCache();
displayMemoryUsageFromNode(initialMemoryUsage);
// decreasing array length
res.splice(0, 10);
}, 1000);
I let the code run for a while, nothing really changed. |
The memory does not automatically drop to normal levels, even though I have enabled GC and cleared all cache.
The text was updated successfully, but these errors were encountered: