Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does node 20.x fully supports cgroups v2 ? facing memory celling problem Kubernetes cluster #52478

Open
PedroFonsecaDEV opened this issue Apr 11, 2024 · 10 comments
Labels
question Issues that look for answers. v20.x v20.x Issues that can be reproduced on v20.x or PRs targeting the v20.x-staging branch.

Comments

@PedroFonsecaDEV
Copy link

Version

20.x

Platform

No response

Subsystem

No response

What steps will reproduce the bug?

Is #47259 fixed on node js 20.x ?

How often does it reproduce? Is there a required condition?

No response

What is the expected behavior? Why is that the expected behavior?

Node js being aware of memory and CPU available via cgroups v2

What do you see instead?

Pods running node js on my cluster running out of memory.

Additional information

No response

@PedroFonsecaDEV PedroFonsecaDEV changed the title Does node 20.x fully supports cgroups v2 ? is the memory celling working on Kubernetes ? Does node 20.x fully supports cgroups v2 ? facing memory celling problem Kubernetes cluster Apr 11, 2024
@mcollina
Copy link
Member

In theory yes.

@PedroFonsecaDEV
Copy link
Author

Hi @mcollina,
could you please expand on that ?

@mcollina
Copy link
Member

Node v20.x includes the libuv version with the necessary fix as far as I understand #47259. However I have not tested it.

@RedYetiDev RedYetiDev added question Issues that look for answers. v20.x v20.x Issues that can be reproduced on v20.x or PRs targeting the v20.x-staging branch. labels Apr 27, 2024
@ben-bertrands-hs
Copy link

We have the same issue and are switching back to cgroup v1

@mcollina
Copy link
Member

cc @santigimeno

@jonesbusy
Copy link

After upgrading on cgroup v2 on our OKD cluster we don't have any issue when running Node 20.13.1 apps. Only Node 18 cause issue that were mitigated using max_old_space_size

@ben-bertrands-hs
Copy link

ben-bertrands-hs commented May 17, 2024

Hi.
Below output for cgroupv1 vs cgroupv2 for the same container image and both with a memory limit of 320Mi

This is on a node using cgroupv1

/app $ node -e "console.log(v8.getHeapStatistics())"
{
total_heap_size: 4407296,
total_heap_size_executable: 262144,
total_physical_size: 3936256,
total_available_size: 268868568,
used_heap_size: 3652248,
heap_size_limit: 271581184,
malloced_memory: 262296,
peak_malloced_memory: 108144,
does_zap_garbage: 0,
number_of_native_contexts: 1,
number_of_detached_contexts: 0,
total_global_handles_size: 8192,
used_global_handles_size: 2240,
external_memory: 1342981
}
/app $ node -v
v20.13.1
/app $ node -e "console.log(process.availableMemory())"
204288000

while this is on a cgroupv2 node

/app $ node -e "console.log(v8.getHeapStatistics())"
{
total_heap_size: 4407296,
total_heap_size_executable: 262144,
total_physical_size: 3936256,
total_available_size: 2195102632,
used_heap_size: 3652280,
heap_size_limit: 2197815296,
malloced_memory: 262296,
peak_malloced_memory: 108144,
does_zap_garbage: 0,
number_of_native_contexts: 1,
number_of_detached_contexts: 0,
total_global_handles_size: 8192,
used_global_handles_size: 2240,
external_memory: 1342981
}
/app $ node -v
v20.13.1
/app $ node -e "console.log(process.availableMemory())"
8880730112

Is something else going on here?

@ben-bertrands-hs
Copy link

@mcollina can you reopen this issue?

@rescomms-tech
Copy link

@ben-bertrands-hs By a chance, are you running on alpine linux? AFAIK its node.js bulid links on alpine-supplied libuv, which, until recently, was fairly old one.

@ben-bertrands-hs
Copy link

ben-bertrands-hs commented Sep 16, 2024

@rescomms-tech yes, we are running alpine linux (node:20.17.0-alpine)
I just checked and we still have this issue when switching to cgroupv2

doing a find on the container only returns these:

~ $ find / -type f -regex '.*uv.*'
find: /proc/tty/driver: Permission denied
find: /root: Permission denied
/usr/local/include/node/uv/aix.h
/usr/local/include/node/uv/bsd.h
/usr/local/include/node/uv/darwin.h
/usr/local/include/node/uv/errno.h
/usr/local/include/node/uv/linux.h
/usr/local/include/node/uv/os390.h
/usr/local/include/node/uv/posix.h
/usr/local/include/node/uv/sunos.h
/usr/local/include/node/uv/threadpool.h
/usr/local/include/node/uv/tree.h
/usr/local/include/node/uv/unix.h
/usr/local/include/node/uv/version.h
/usr/local/include/node/uv/win.h
/usr/local/include/node/uv.h

Checking /usr/local/include/node/uv/version.h gives me version 1.46.0.
Every version from 1.45 on should support cgroupv2 right?

@mcollina mcollina reopened this Sep 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Issues that look for answers. v20.x v20.x Issues that can be reproduced on v20.x or PRs targeting the v20.x-staging branch.
Projects
None yet
Development

No branches or pull requests

6 participants