[17322.406998] Memory cgroup stats for /kubepods.slice/kubepods-pod36f770ce_9543_4d89_9a49_c13ea804cf3b.slice: [17322.407018] anon 19869696 file 0 kernel_stack 2101248 slab 38920192 sock 0 shmem 0 file_mapped 0 file_dirty 0 file_writeback 0 anon_thp 0 inactive_anon 0 active_anon 21221376 inactive_file 0 active_file 0 unevictable 0 slab_reclaimable 9027584 slab_unreclaimable 29892608 pgfault 4123709 pgmajfault 3279 workingset_refault 61051 workingset_activate 1037 workingset_nodereclaim 0 pgrefill 13006 pgscan 110965 pgsteal 42350 pgactivate 8398 pgdeactivate 12357 pglazyfree 0 pglazyfreed 0 thp_fault_alloc 0 thp_collapse_alloc 0 [17322.407019] Tasks state (memory values in pages): [17322.407021] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [17322.407029] [ 480740] 0 480740 241 1 28672 0 -998 pause [17322.407035] [1753645] 0 1753645 181096 7208 163840 0 -997 nsmgr [17322.407039] [1780226] 0 1780226 178142 2808 110592 0 -997 grpc-health-pro [17322.407042] [1780297] 0 1780297 177918 2473 106496 0 -997 grpc-health-pro [17322.407045] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=docker-13df99ee6a7c347d09fd883470dc7a8a25048ebb41f561bc575416c05d15255c.scope,mems_allowed=0-1,oom_memcg=/kubepods.slice/kubepods-pod36f770ce_9543_4d89_9a49_c13ea804cf3b.slice,task_memcg=/kubepods.slice/kubepods-pod36f770ce_9543_4d89_9a49_c13ea804cf3b.slice/docker-13df99ee6a7c347d09fd883470dc7a8a25048ebb41f561bc575416c05d15255c.scope,task=grpc-health-pro,pid=1780297,uid=0 [17322.407095] Memory cgroup out of memory: Killed process 1780297 (grpc-health-pro) total-vm:711672kB, anon-rss:2564kB, file-rss:7784kB, shmem-rss:0kB, UID:0 pgtables:104kB oom_score_adj:-997 [17322.415380] oom_reaper: reaped process 1780297 (grpc-health-pro), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB [17332.306107] grpc-health-pro invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=-997 [17332.306113] CPU: 7 PID: 1781675 Comm: grpc-health-pro Not tainted 5.4.0-65-generic #73-Ubuntu [17332.306116] Hardware name: Dell Inc. PowerEdge R750/0WMWCR, BIOS 1.1.3 04/27/2021 [17332.306117] Call Trace: [17332.306130] dump_stack+0x6d/0x9a [17332.306137] dump_header+0x4f/0x1eb [17332.306140] oom_kill_process.cold+0xb/0x10 [17332.306144] out_of_memory.part.0+0x1df/0x3d0 [17332.306147] out_of_memory+0x6d/0xd0 [17332.306155] mem_cgroup_out_of_memory+0xbd/0xe0 [17332.306160] try_charge+0x77c/0x810 [17332.306168] ? __alloc_pages_nodemask+0x173/0x320 [17332.306173] mem_cgroup_try_charge+0x71/0x190 [17332.306177] mem_cgroup_try_charge_delay+0x22/0x50 [17332.306183] do_anonymous_page+0x13c/0x650 [17332.306186] ? do_fault+0x1c2/0x640 [17332.306190] __handle_mm_fault+0x760/0x7a0 [17332.306195] handle_mm_fault+0xca/0x200 [17332.306202] do_user_addr_fault+0x1f9/0x450 [17332.306208] __do_page_fault+0x58/0x90 [17332.306215] ? syscall_slow_exit_work+0x123/0x150 [17332.306219] do_page_fault+0x2c/0xe0 [17332.306224] page_fault+0x34/0x40 [17332.306228] RIP: 0033:0x44e80c [17332.306234] Code: d2 0f 85 ce 00 00 00 0f b6 4c 24 48 ba 00 08 00 00 48 d3 e2 48 89 50 68 31 c9 eb 22 48 8b 50 18 48 01 ca 48 89 d3 48 8b 70 28 <48> 89 32 48 89 58 28 48 8b 50 68 48 01 d1 66 0f 1f 44 00 00 48 81 [17332.306236] RSP: 002b:000000c000189b78 EFLAGS: 00010206 [17332.306240] RAX: 00007fc6ceb85328 RBX: 000000c000334000 RCX: 0000000000000000 [17332.306242] RDX: 000000c000334000 RSI: 0000000000000000 RDI: 0000800000000000 [17332.306244] RBP: 000000c000189bb0 R08: 000000c000334000 R09: 00007fc6ceb85328 [17332.306245] R10: 0000000000000004 R11: 000000000000019d R12: 0000000000000003 [17332.306247] R13: 0000000000000001 R14: 0000000000000036 R15: ffffffffffffffff [17332.306253] memory: usage 61440kB, limit 61440kB, failcnt 48009 [17332.306255] memory+swap: usage 0kB, limit 9007199254740988kB, failcnt 0 [17332.306257] kmem: usage 38112kB, limit 9007199254740988kB, failcnt 0 [17332.306258] Memory cgroup stats for /kubepods.slice/kubepods-pod36f770ce_9543_4d89_9a49_c13ea804cf3b.slice: [17332.306280] anon 19427328 file 0 kernel_stack 1990656 slab 39370752 sock 0 shmem 0 file_mapped 0 file_dirty 0 file_writeback 0 anon_thp 0 inactive_anon 0 active_anon 20815872 inactive_file 0 active_file 0 unevictable 0 slab_reclaimable 9166848 slab_unreclaimable 30203904 pgfault 4127207 pgmajfault 3279 workingset_refault 61084 workingset_activate 1037 workingset_nodereclaim 0 pgrefill 13006 pgscan 111836 pgsteal 42383 pgactivate 8398 pgdeactivate 12357 pglazyfree 0 pglazyfreed 0 thp_fault_alloc 0 thp_collapse_alloc 0 [17332.306282] Tasks state (memory values in pages): [17332.306283] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [17332.306291] [ 480740] 0 480740 241 1 28672 0 -998 pause [17332.306297] [1753645] 0 1753645 181096 7202 163840 0 -997 nsmgr [17332.306301] [1781612] 0 1781612 178270 2722 102400 0 -997 grpc-health-pro [17332.306305] [1781659] 0 1781659 177854 2430 102400 0 -997 grpc-health-pro [17332.306307] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=docker-13df99ee6a7c347d09fd883470dc7a8a25048ebb41f561bc575416c05d15255c.scope,mems_allowed=0-1,oom_memcg=/kubepods.slice/kubepods-pod36f770ce_9543_4d89_9a49_c13ea804cf3b.slice,task_memcg=/kubepods.slice/kubepods-pod36f770ce_9543_4d89_9a49_c13ea804cf3b.slice/docker-13df99ee6a7c347d09fd883470dc7a8a25048ebb41f561bc575416c05d15255c.scope,task=grpc-health-pro,pid=1781659,uid=0 [17332.306360] Memory cgroup out of memory: Killed process 1781659 (grpc-health-pro) total-vm:711416kB, anon-rss:2628kB, file-rss:7092kB, shmem-rss:0kB, UID:0 pgtables:100kB oom_score_adj:-997 [17332.315285] oom_reaper: reaped process 1781659 (grpc-health-pro), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB [17342.212192] grpc-health-pro invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=-997 [17342.212200] CPU: 62 PID: 1782726 Comm: grpc-health-pro Not tainted 5.4.0-65-generic #73-Ubuntu [17342.212202] Hardware name: Dell Inc. PowerEdge R750/0WMWCR, BIOS 1.1.3 04/27/2021 [17342.212204] Call Trace: [17342.212222] dump_stack+0x6d/0x9a [17342.212230] dump_header+0x4f/0x1eb [17342.212233] oom_kill_process.cold+0xb/0x10 [17342.212237] out_of_memory.part.0+0x1df/0x3d0 [17342.212240] out_of_memory+0x6d/0xd0 [17342.212250] mem_cgroup_out_of_memory+0xbd/0xe0 [17342.212255] try_charge+0x77c/0x810 [17342.212265] ? __alloc_pages_nodemask+0x173/0x320 [17342.212269] mem_cgroup_try_charge+0x71/0x190 [17342.212274] mem_cgroup_try_charge_delay+0x22/0x50 [17342.212281] do_anonymous_page+0x13c/0x650 [17342.212285] __handle_mm_fault+0x760/0x7a0 [17342.212290] handle_mm_fault+0xca/0x200 [17342.212299] do_user_addr_fault+0x1f9/0x450 [17342.212304] __do_page_fault+0x58/0x90 [17342.212309] do_page_fault+0x2c/0xe0 [17342.212314] page_fault+0x34/0x40 [17342.212320] RIP: 0033:0x46ca73 [17342.212325] Code: 00 48 81 eb 00 01 00 00 48 81 c7 00 01 00 00 48 81 fb 00 01 00 00 73 82 e9 07 ff ff ff c5 fd ef c0 48 81 fb 00 00 00 02 73 46 fe 7f 07 c5 fe 7f 47 20 c5 fe 7f 47 40 c5 fe 7f 47 60 48 81 eb [17342.212327] RSP: 002b:000000c00030d8c8 EFLAGS: 00010287 [17342.212332] RAX: 0000000000000000 RBX: 0000000000002000 RCX: 000000c00027e000 [17342.212334] RDX: 00000000000000eb RSI: 0000000000000080 RDI: 000000c00027e000 [17342.212335] RBP: 000000c00030d910 R08: 000000c00027e000 R09: 00007f517da53448 [17342.212337] R10: 0000000000000001 R11: 000000000000013f R12: 0000000000000000 [17342.212339] R13: 0000000000000040 R14: 0000000000000040 R15: 0000000000000000 [17342.212344] memory: usage 61440kB, limit 61440kB, failcnt 48119 [17342.212347] memory+swap: usage 0kB, limit 9007199254740988kB, failcnt 0 [17342.212349] kmem: usage 38348kB, limit 9007199254740988kB, failcnt 0 [17342.212350] Memory cgroup stats for /kubepods.slice/kubepods-pod36f770ce_9543_4d89_9a49_c13ea804cf3b.slice: [17342.212372] anon 19058688 file 0 kernel_stack 2027520 slab 39645184 sock 0 shmem 0 file_mapped 0 file_dirty 0 file_writeback 0 anon_thp 0 inactive_anon 0 active_anon 20410368 inactive_file 0 active_file 0 unevictable 0 slab_reclaimable 9306112 slab_unreclaimable 30339072 pgfault 4130540 pgmajfault 3279 workingset_refault 61117 workingset_activate 1037 workingset_nodereclaim 0 pgrefill 13006 pgscan 111924 pgsteal 42423 pgactivate 8398 pgdeactivate 12357 pglazyfree 0 pglazyfreed 0 thp_fault_alloc 0 thp_collapse_alloc 0 [17342.212374] Tasks state (memory values in pages): [17342.212375] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [17342.212382] [ 480740] 0 480740 241 1 28672 0 -998 pause [17342.212388] [1753645] 0 1753645 181096 7198 163840 0 -997 nsmgr [17342.212393] [1782687] 0 1782687 178270 2717 106496 0 -997 grpc-health-pro [17342.212396] [1782726] 0 1782726 177726 1900 98304 0 -997 grpc-health-pro [17342.212399] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=docker-13df99ee6a7c347d09fd883470dc7a8a25048ebb41f561bc575416c05d15255c.scope,mems_allowed=0-1,oom_memcg=/kubepods.slice/kubepods-pod36f770ce_9543_4d89_9a49_c13ea804cf3b.slice,task_memcg=/kubepods.slice/kubepods-pod36f770ce_9543_4d89_9a49_c13ea804cf3b.slice/docker-13df99ee6a7c347d09fd883470dc7a8a25048ebb41f561bc575416c05d15255c.scope,task=grpc-health-pro,pid=1782726,uid=0 [17342.212436] Memory cgroup out of memory: Killed process 1782726 (grpc-health-pro) total-vm:710904kB, anon-rss:2364kB, file-rss:5236kB, shmem-rss:0kB, UID:0 pgtables:96kB oom_score_adj:-997 [17342.218089] oom_reaper: reaped process 1782726 (grpc-health-pro), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB [17413.749713] vpp invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=999 [17413.749717] CPU: 1 PID: 1789734 Comm: vpp Not tainted 5.4.0-65-generic #73-Ubuntu [17413.749718] Hardware name: Dell Inc. PowerEdge R750/0WMWCR, BIOS 1.1.3 04/27/2021 [17413.749719] Call Trace: [17413.749728] dump_stack+0x6d/0x9a [17413.749732] dump_header+0x4f/0x1eb [17413.749734] oom_kill_process.cold+0xb/0x10 [17413.749735] out_of_memory.part.0+0x1df/0x3d0 [17413.749736] out_of_memory+0x6d/0xd0 [17413.749740] mem_cgroup_out_of_memory+0xbd/0xe0 [17413.749742] try_charge+0x77c/0x810 [17413.749744] mem_cgroup_try_charge+0x71/0x190 [17413.749746] mem_cgroup_try_charge_delay+0x22/0x50 [17413.749749] shmem_getpage_gfp+0x1d7/0x940 [17413.749751] shmem_fault+0x79/0x210 [17413.749754] ? file_update_time+0x62/0x140 [17413.749757] __do_fault+0x3c/0x130 [17413.749759] do_fault+0x38b/0x640 [17413.749763] ? __switch_to_asm+0x40/0x70 [17413.749765] __handle_mm_fault+0x4c5/0x7a0 [17413.749766] handle_mm_fault+0xca/0x200 [17413.749771] do_user_addr_fault+0x1f9/0x450 [17413.749773] __do_page_fault+0x58/0x90 [17413.749774] do_page_fault+0x2c/0xe0 [17413.749776] page_fault+0x34/0x40 [17413.749778] RIP: 0033:0x7fe6baa8aad1 [17413.749781] Code: 01 00 00 48 83 fa 40 77 77 c5 fe 7f 44 17 e0 c5 fe 7f 07 c5 f8 77 c3 66 90 f3 0f 1e fa c5 f8 77 48 89 d1 40 0f b6 c6 48 89 fa aa 48 89 d0 c3 66 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 48 39 d1 [17413.749782] RSP: 002b:00007fe677957af8 EFLAGS: 00010206 [17413.749784] RAX: 0000000000000000 RBX: 00007fe67abd0208 RCX: 0000000002d14000 [17413.749784] RDX: 0000001000000000 RSI: 0000000000000000 RDI: 00000010052ec000 [17413.749785] RBP: 00007fe67ae5e040 R08: 0000000000000005 R09: 0000000000000000 [17413.749786] R10: 0000000000000011 R11: 0000000000000206 R12: 0000000000000000 [17413.749786] R13: 0000001000000000 R14: 0000000008000000 R15: 0000000000000000 [17413.749788] memory: usage 409600kB, limit 409600kB, failcnt 134880 [17413.749789] memory+swap: usage 0kB, limit 9007199254740988kB, failcnt 0 [17413.749790] kmem: usage 8048kB, limit 9007199254740988kB, failcnt 0 [17413.749790] Memory cgroup stats for /kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34486a26_2d41_41e0_8f42_590e2ba8e868.slice: [17413.749801] anon 50675712 file 355033088 kernel_stack 221184 slab 14548992 sock 0 shmem 355766272 file_mapped 85966848 file_dirty 0 file_writeback 0 anon_thp 0 inactive_anon 353464320 active_anon 53121024 inactive_file 0 active_file 0 unevictable 0 slab_reclaimable 8536064 slab_unreclaimable 6012928 pgfault 2527035 pgmajfault 6745 workingset_refault 213214 workingset_activate 33 workingset_nodereclaim 660 pgrefill 125926 pgscan 440231 pgsteal 391275 pgactivate 38876 pgdeactivate 118047 pglazyfree 0 pglazyfreed 0 thp_fault_alloc 0 thp_collapse_alloc 0 [17413.749802] Tasks state (memory values in pages): [17413.749803] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [17413.749806] [ 517133] 0 517133 241 1 28672 0 -998 pause [17413.749809] [1789689] 0 1789689 180596 3320 139264 0 999 app [17413.749810] [1789734] 0 1789734 4482778 35099 393216 0 999 vpp [17413.749811] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=docker-1ee48d7e0270d6f460982bdd1a861784e7b9fff08c5e6287185828156938031e.scope,mems_allowed=0-1,oom_memcg=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34486a26_2d41_41e0_8f42_590e2ba8e868.slice,task_memcg=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34486a26_2d41_41e0_8f42_590e2ba8e868.slice/docker-1ee48d7e0270d6f460982bdd1a861784e7b9fff08c5e6287185828156938031e.scope,task=vpp,pid=1789734,uid=0 [17413.749823] Memory cgroup out of memory: Killed process 1789734 (vpp) total-vm:17931112kB, anon-rss:48100kB, file-rss:7432kB, shmem-rss:84864kB, UID:0 pgtables:384kB oom_score_adj:999 [17413.764565] oom_reaper: reaped process 1789734 (vpp), now anon-rss:0kB, file-rss:0kB, shmem-rss:84864kB [17455.921190] vpp invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=999 [17455.921194] CPU: 1 PID: 1794318 Comm: vpp Not tainted 5.4.0-65-generic #73-Ubuntu [17455.921195] Hardware name: Dell Inc. PowerEdge R750/0WMWCR, BIOS 1.1.3 04/27/2021 [17455.921196] Call Trace: [17455.921205] dump_stack+0x6d/0x9a [17455.921209] dump_header+0x4f/0x1eb [17455.921210] oom_kill_process.cold+0xb/0x10 [17455.921211] out_of_memory.part.0+0x1df/0x3d0 [17455.921213] out_of_memory+0x6d/0xd0 [17455.921217] mem_cgroup_out_of_memory+0xbd/0xe0 [17455.921219] try_charge+0x77c/0x810 [17455.921221] mem_cgroup_try_charge+0x71/0x190 [17455.921223] mem_cgroup_try_charge_delay+0x22/0x50 [17455.921226] shmem_getpage_gfp+0x1d7/0x940 [17455.921228] shmem_fault+0x79/0x210 [17455.921231] ? file_update_time+0x62/0x140 [17455.921235] __do_fault+0x3c/0x130 [17455.921236] do_fault+0x38b/0x640 [17455.921240] ? __switch_to_asm+0x40/0x70 [17455.921241] __handle_mm_fault+0x4c5/0x7a0 [17455.921243] handle_mm_fault+0xca/0x200 [17455.921247] do_user_addr_fault+0x1f9/0x450 [17455.921249] __do_page_fault+0x58/0x90 [17455.921250] do_page_fault+0x2c/0xe0 [17455.921253] page_fault+0x34/0x40 [17455.921255] RIP: 0033:0x7fa9a678fad1 [17455.921258] Code: 01 00 00 48 83 fa 40 77 77 c5 fe 7f 44 17 e0 c5 fe 7f 07 c5 f8 77 c3 66 90 f3 0f 1e fa c5 f8 77 48 89 d1 40 0f b6 c6 48 89 fa aa 48 89 d0 c3 66 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 48 39 d1 [17455.921259] RSP: 002b:00007fa96365caf8 EFLAGS: 00010206 [17455.921261] RAX: 0000000000000000 RBX: 00007fa9668cd328 RCX: 0000000002cf5000 [17455.921262] RDX: 0000001000000000 RSI: 0000000000000000 RDI: 000000100530b000 [17455.921262] RBP: 00007fa966b61740 R08: 0000000000000005 R09: 0000000000000000 [17455.921263] R10: 0000000000000011 R11: 0000000000000206 R12: 0000000000000000 [17455.921264] R13: 0000001000000000 R14: 0000000008000000 R15: 0000000000000000 [17455.921266] memory: usage 409600kB, limit 409600kB, failcnt 132765 [17455.921266] memory+swap: usage 0kB, limit 9007199254740988kB, failcnt 0 [17455.921267] kmem: usage 7664kB, limit 9007199254740988kB, failcnt 0 [17455.921268] Memory cgroup stats for /kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25dac1b_9572_419e_b020_a736be854cb8.slice: [17455.921280] anon 48680960 file 353484800 kernel_stack 405504 slab 8802304 sock 0 shmem 356081664 file_mapped 83939328 file_dirty 0 file_writeback 0 anon_thp 0 inactive_anon 353599488 active_anon 50958336 inactive_file 0 active_file 0 unevictable 0 slab_reclaimable 7229440 slab_unreclaimable 1572864 pgfault 2546375 pgmajfault 6850 workingset_refault 267907 workingset_activate 155 workingset_nodereclaim 660 pgrefill 193725 pgscan 512275 pgsteal 388522 pgactivate 112195 pgdeactivate 187556 pglazyfree 0 pglazyfreed 0 thp_fault_alloc 0 thp_collapse_alloc 0 [17455.921281] Tasks state (memory values in pages): [17455.921281] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [17455.921284] [ 517157] 0 517157 241 1 28672 0 -998 pause [17455.921288] [1793979] 0 1793979 180532 3263 147456 0 999 app [17455.921289] [1794318] 0 1794318 4482778 35105 405504 0 999 vpp [17455.921291] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=docker-8d73828594e038fe61e8f0d8f60e4abc09a3c1c9096981d5483c98378296955b.scope,mems_allowed=0-1,oom_memcg=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25dac1b_9572_419e_b020_a736be854cb8.slice,task_memcg=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25dac1b_9572_419e_b020_a736be854cb8.slice/docker-8d73828594e038fe61e8f0d8f60e4abc09a3c1c9096981d5483c98378296955b.scope,task=vpp,pid=1794318,uid=0 [17455.921303] Memory cgroup out of memory: Killed process 1794318 (vpp) total-vm:17931112kB, anon-rss:48100kB, file-rss:7328kB, shmem-rss:84992kB, UID:0 pgtables:396kB oom_score_adj:999 [17455.932409] oom_reaper: reaped process 1794318 (vpp), now anon-rss:0kB, file-rss:0kB, shmem-rss:84992kB [17724.909253] vpp invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=999 [17724.909259] CPU: 1 PID: 1820899 Comm: vpp Not tainted 5.4.0-65-generic #73-Ubuntu [17724.909260] Hardware name: Dell Inc. PowerEdge R750/0WMWCR, BIOS 1.1.3 04/27/2021 [17724.909262] Call Trace: [17724.909275] dump_stack+0x6d/0x9a [17724.909280] dump_header+0x4f/0x1eb [17724.909283] oom_kill_process.cold+0xb/0x10 [17724.909285] out_of_memory.part.0+0x1df/0x3d0 [17724.909288] out_of_memory+0x6d/0xd0 [17724.909294] mem_cgroup_out_of_memory+0xbd/0xe0 [17724.909298] try_charge+0x77c/0x810 [17724.909303] mem_cgroup_try_charge+0x71/0x190 [17724.909307] mem_cgroup_try_charge_delay+0x22/0x50 [17724.909311] shmem_getpage_gfp+0x1d7/0x940 [17724.909315] shmem_fault+0x79/0x210 [17724.909321] ? file_update_time+0x62/0x140 [17724.909326] __do_fault+0x3c/0x130 [17724.909329] do_fault+0x38b/0x640 [17724.909336] ? __switch_to_asm+0x34/0x70 [17724.909339] __handle_mm_fault+0x4c5/0x7a0 [17724.909343] handle_mm_fault+0xca/0x200 [17724.909349] do_user_addr_fault+0x1f9/0x450 [17724.909353] __do_page_fault+0x58/0x90 [17724.909356] do_page_fault+0x2c/0xe0 [17724.909360] page_fault+0x34/0x40 [17724.909363] RIP: 0033:0x7f9b9ef66ad1 [17724.909368] Code: 01 00 00 48 83 fa 40 77 77 c5 fe 7f 44 17 e0 c5 fe 7f 07 c5 f8 77 c3 66 90 f3 0f 1e fa c5 f8 77 48 89 d1 40 0f b6 c6 48 89 fa aa 48 89 d0 c3 66 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 48 39 d1 [17724.909370] RSP: 002b:00007f9b5be33af8 EFLAGS: 00010206 [17724.909374] RAX: 0000000000000000 RBX: 00007f9b5f0ac208 RCX: 0000000002f0c000 [17724.909375] RDX: 0000001000000000 RSI: 0000000000000000 RDI: 00000010050f4000 [17724.909377] RBP: 00007f9b5f33a040 R08: 0000000000000005 R09: 0000000000000000 [17724.909378] R10: 0000000000000011 R11: 0000000000000206 R12: 0000000000000000 [17724.909379] R13: 0000001000000000 R14: 0000000008000000 R15: 0000000000000000 [17724.909384] memory: usage 409600kB, limit 409600kB, failcnt 139266 [17724.909386] memory+swap: usage 0kB, limit 9007199254740988kB, failcnt 0 [17724.909387] kmem: usage 8244kB, limit 9007199254740988kB, failcnt 0 [17724.909388] Memory cgroup stats for /kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34486a26_2d41_41e0_8f42_590e2ba8e868.slice: [17724.909406] anon 51523584 file 352694272 kernel_stack 294912 slab 12468224 sock 0 shmem 353832960 file_mapped 83263488 file_dirty 0 file_writeback 0 anon_thp 0 inactive_anon 351436800 active_anon 54067200 inactive_file 0 active_file 0 unevictable 0 slab_reclaimable 7319552 slab_unreclaimable 5148672 pgfault 2587656 pgmajfault 6943 workingset_refault 213247 workingset_activate 33 workingset_nodereclaim 660 pgrefill 131209 pgscan 455147 pgsteal 403605 pgactivate 41153 pgdeactivate 123064 pglazyfree 0 pglazyfreed 0 thp_fault_alloc 0 thp_collapse_alloc 0 [17724.909408] Tasks state (memory values in pages): [17724.909409] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [17724.909415] [ 517133] 0 517133 241 1 28672 0 -998 pause [17724.909420] [1820741] 0 1820741 180676 3882 147456 0 999 app [17724.909423] [1820899] 0 1820899 4482778 34605 389120 0 999 vpp [17724.909425] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=docker-0cb54a8a8954cd1463ae25e1de917f54a7a08ba1836fa05f4d263b47785520a8.scope,mems_allowed=0-1,oom_memcg=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34486a26_2d41_41e0_8f42_590e2ba8e868.slice,task_memcg=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34486a26_2d41_41e0_8f42_590e2ba8e868.slice/docker-0cb54a8a8954cd1463ae25e1de917f54a7a08ba1836fa05f4d263b47785520a8.scope,task=vpp,pid=1820899,uid=0 [17724.909444] Memory cgroup out of memory: Killed process 1820899 (vpp) total-vm:17931112kB, anon-rss:48104kB, file-rss:7428kB, shmem-rss:82888kB, UID:0 pgtables:380kB oom_score_adj:999 [17724.922181] oom_reaper: reaped process 1820899 (vpp), now anon-rss:0kB, file-rss:0kB, shmem-rss:82916kB [17765.026435] vpp invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=999 [17765.026439] CPU: 1 PID: 1825557 Comm: vpp Not tainted 5.4.0-65-generic #73-Ubuntu [17765.026440] Hardware name: Dell Inc. PowerEdge R750/0WMWCR, BIOS 1.1.3 04/27/2021 [17765.026441] Call Trace: [17765.026450] dump_stack+0x6d/0x9a [17765.026454] dump_header+0x4f/0x1eb [17765.026455] oom_kill_process.cold+0xb/0x10 [17765.026457] out_of_memory.part.0+0x1df/0x3d0 [17765.026458] out_of_memory+0x6d/0xd0 [17765.026462] mem_cgroup_out_of_memory+0xbd/0xe0 [17765.026464] try_charge+0x77c/0x810 [17765.026466] mem_cgroup_try_charge+0x71/0x190 [17765.026468] mem_cgroup_try_charge_delay+0x22/0x50 [17765.026471] shmem_getpage_gfp+0x1d7/0x940 [17765.026473] shmem_fault+0x79/0x210 [17765.026477] ? file_update_time+0x62/0x140 [17765.026480] __do_fault+0x3c/0x130 [17765.026482] do_fault+0x38b/0x640 [17765.026485] ? __switch_to_asm+0x34/0x70 [17765.026487] __handle_mm_fault+0x4c5/0x7a0 [17765.026489] handle_mm_fault+0xca/0x200 [17765.026493] do_user_addr_fault+0x1f9/0x450 [17765.026495] __do_page_fault+0x58/0x90 [17765.026497] do_page_fault+0x2c/0xe0 [17765.026498] page_fault+0x34/0x40 [17765.026500] RIP: 0033:0x7f2a52b2cad1 [17765.026504] Code: 01 00 00 48 83 fa 40 77 77 c5 fe 7f 44 17 e0 c5 fe 7f 07 c5 f8 77 c3 66 90 f3 0f 1e fa c5 f8 77 48 89 d1 40 0f b6 c6 48 89 fa aa 48 89 d0 c3 66 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 48 39 d1 [17765.026504] RSP: 002b:00007f2a0f9f9af8 EFLAGS: 00010206 [17765.026507] RAX: 0000000000000000 RBX: 00007f2a12c72208 RCX: 0000000002e80000 [17765.026507] RDX: 0000001000000000 RSI: 0000000000000000 RDI: 0000001005180000 [17765.026508] RBP: 00007f2a12f00040 R08: 0000000000000005 R09: 0000000000000000 [17765.026509] R10: 0000000000000011 R11: 0000000000000206 R12: 0000000000000000 [17765.026509] R13: 0000001000000000 R14: 0000000008000000 R15: 0000000000000000 [17765.026511] memory: usage 409600kB, limit 409600kB, failcnt 136714 [17765.026512] memory+swap: usage 0kB, limit 9007199254740988kB, failcnt 0 [17765.026513] kmem: usage 7728kB, limit 9007199254740988kB, failcnt 0 [17765.026513] Memory cgroup stats for /kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25dac1b_9572_419e_b020_a736be854cb8.slice: [17765.026525] anon 50388992 file 351813632 kernel_stack 516096 slab 9224192 sock 0 shmem 354459648 file_mapped 82317312 file_dirty 0 file_writeback 0 anon_thp 0 inactive_anon 351977472 active_anon 52445184 inactive_file 0 active_file 0 unevictable 0 slab_reclaimable 7229440 slab_unreclaimable 1994752 pgfault 2607580 pgmajfault 7152 workingset_refault 267907 workingset_activate 155 workingset_nodereclaim 693 pgrefill 205782 pgscan 534098 pgsteal 400982 pgactivate 121200 pgdeactivate 199345 pglazyfree 0 pglazyfreed 0 thp_fault_alloc 0 thp_collapse_alloc 0 [17765.026526] Tasks state (memory values in pages): [17765.026526] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [17765.026529] [ 517157] 0 517157 241 1 28672 0 -998 pause [17765.026533] [1825494] 0 1825494 180532 3648 131072 0 999 app [17765.026534] [1825557] 0 1825557 4482778 34710 397312 0 999 vpp [17765.026535] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=docker-7c9c486b15ffd517117fcc6622de936a6da9df23e0366d999df01280ddc9b133.scope,mems_allowed=0-1,oom_memcg=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25dac1b_9572_419e_b020_a736be854cb8.slice,task_memcg=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25dac1b_9572_419e_b020_a736be854cb8.slice/docker-7c9c486b15ffd517117fcc6622de936a6da9df23e0366d999df01280ddc9b133.scope,task=vpp,pid=1825557,uid=0 [17765.026546] Memory cgroup out of memory: Killed process 1825557 (vpp) total-vm:17931112kB, anon-rss:48104kB, file-rss:7324kB, shmem-rss:83412kB, UID:0 pgtables:388kB oom_score_adj:999 [17765.037364] oom_reaper: reaped process 1825557 (vpp), now anon-rss:0kB, file-rss:0kB, shmem-rss:83412kB [17781.848105] runc:[2:INIT] invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=999 [17781.848110] CPU: 51 PID: 1828070 Comm: runc:[2:INIT] Not tainted 5.4.0-65-generic #73-Ubuntu [17781.848111] Hardware name: Dell Inc. PowerEdge R750/0WMWCR, BIOS 1.1.3 04/27/2021 [17781.848112] Call Trace: [17781.848122] dump_stack+0x6d/0x9a [17781.848128] dump_header+0x4f/0x1eb [17781.848130] oom_kill_process.cold+0xb/0x10 [17781.848132] out_of_memory.part.0+0x1df/0x3d0 [17781.848133] out_of_memory+0x6d/0xd0 [17781.848139] mem_cgroup_out_of_memory+0xbd/0xe0 [17781.848142] try_charge+0x77c/0x810 [17781.848147] ? __alloc_pages_nodemask+0x173/0x320 [17781.848149] mem_cgroup_try_charge+0x71/0x190 [17781.848152] mem_cgroup_try_charge_delay+0x22/0x50 [17781.848156] do_anonymous_page+0x13c/0x650 [17781.848158] __handle_mm_fault+0x760/0x7a0 [17781.848161] handle_mm_fault+0xca/0x200 [17781.848165] do_user_addr_fault+0x1f9/0x450 [17781.848168] __do_page_fault+0x58/0x90 [17781.848173] ? syscall_slow_exit_work+0x123/0x150 [17781.848175] do_page_fault+0x2c/0xe0 [17781.848178] page_fault+0x34/0x40 [17781.848181] RIP: 0033:0x55f5661bf481 [17781.848185] Code: 0f 11 07 48 83 c7 10 0f 10 06 48 83 c6 10 0f 11 07 48 83 c7 10 0f 10 06 48 83 c6 10 0f 11 07 48 83 c7 10 0f 10 06 48 83 c6 10 <0f> 11 07 48 83 c7 10 0f 10 06 48 83 c6 10 0f 11 07 48 83 c7 10 0f [17781.848186] RSP: 002b:000000c0001db208 EFLAGS: 00010206 [17781.848188] RAX: 000000c000170700 RBX: 0000000000000004 RCX: 0000000000000011 [17781.848189] RDX: 000000c0001e4ea0 RSI: 000000c0001db630 RDI: 000000c0001ecff8 [17781.848190] RBP: 000000c0001db200 R08: 000000c0001eca80 R09: 0000000000000005 [17781.848191] R10: 0000000000000009 R11: 00000000000004a0 R12: 000055f566626979 [17781.848192] R13: 0000000944eeaa95 R14: 000055f5666af970 R15: 0000000000000025 [17781.848195] memory: usage 512000kB, limit 512000kB, failcnt 189 [17781.848197] memory+swap: usage 0kB, limit 9007199254740988kB, failcnt 0 [17781.848197] kmem: usage 68116kB, limit 9007199254740988kB, failcnt 0 [17781.848198] Memory cgroup stats for /kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e23f810_ca15_4d10_81c8_551598e00b45.slice: [17781.848211] anon 179986432 file 270204928 kernel_stack 1769472 slab 48750592 sock 0 shmem 270471168 file_mapped 270065664 file_dirty 0 file_writeback 0 anon_thp 0 inactive_anon 270471168 active_anon 181395456 inactive_file 0 active_file 49152 unevictable 0 slab_reclaimable 3047424 slab_unreclaimable 45703168 pgfault 4871823 pgmajfault 0 workingset_refault 0 workingset_activate 0 workingset_nodereclaim 0 pgrefill 94 pgscan 668 pgsteal 328 pgactivate 66 pgdeactivate 60 pglazyfree 0 pglazyfreed 0 thp_fault_alloc 0 thp_collapse_alloc 0 [17781.848212] Tasks state (memory values in pages): [17781.848213] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [17781.848217] [ 480661] 0 480661 241 1 28672 0 -998 pause [17781.848220] [ 480824] 0 480824 215596 23093 311296 0 999 forwarder [17781.848222] [ 480932] 0 480932 4521666 102394 942080 0 999 vpp_main [17781.848224] [1828070] 0 1828070 271911 3064 155648 0 999 runc:[2:INIT] [17781.848226] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=docker-5937d8c804f3b0c4870404c7e5a63b3644d44710904e2b3d8f64b31edad00cf0.scope,mems_allowed=0-1,oom_memcg=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e23f810_ca15_4d10_81c8_551598e00b45.slice,task_memcg=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e23f810_ca15_4d10_81c8_551598e00b45.slice/docker-5937d8c804f3b0c4870404c7e5a63b3644d44710904e2b3d8f64b31edad00cf0.scope,task=vpp_main,pid=480932,uid=0 [17781.848245] Memory cgroup out of memory: Killed process 480932 (vpp_main) total-vm:18086664kB, anon-rss:100612kB, file-rss:44816kB, shmem-rss:264148kB, UID:0 pgtables:920kB oom_score_adj:999 [17781.863924] oom_reaper: reaped process 480932 (vpp_main), now anon-rss:0kB, file-rss:20480kB, shmem-rss:264148kB