Use cgroup v2

[cartermckinnon]

What would you like to be added:

Support for cgroup v2, specifically for systemd.

Why is this needed:

Kubernetes recommends that, when using systemd as the cgroup driver, the "unified hierarchy" of cgroup v2 be used.

Depends on:

• Use systemd as the containerd cgroup driver #717
• systemd@244
• kernel@5.8

---

Implementation notes:

Cgroup v2 is enabled by a kernel flag:

sudo grubby \
  --update-kernel=ALL \
  --args="systemd.unified_cgroup_hierarchy=1"

Which can be added to the Packer spec. There is no support for v2 until systemd@230, and AL2 is currently at systemd@219. The recommended version (by runc) is systemd@244. A newer systemd is not expected to be available until the next major release of Amazon Linux.

[cartermckinnon]

The latest AL2022 AMI (ami-0bf3366da6649b338 in us-west-2) is using systemd@248, so we're on track to make this change once we rebase on 2022.

> systemctl --version
systemd 248 (v248.10-1.amzn2022.0.1)

[DobromirM]

Is there any progress on this? We've run into a problem when using the gVisor runtime that will possibly be solved by switching to cgroup v2. google/gvisor#8047 (comment)

[sidewinder12s]

The Kubernetes docs also recommend a 5.8+ kernel for cgroup v2.

[stevehipwell]

Kubernetes support for cgroups v2 is only stable from v1.25. There was a very good talk about this at KubeCon, the video should be on YouTube in a day or so.

[cartermckinnon]

Yep, I'll post the talk when it's up. Wish we had run into you, @stevehipwell!

We'll have the proper systemd and kernel versions once we rebase on AL2022, so we should be able to target this for 1.25 (assuming the GA dates align).

Edit: https://www.youtube.com/watch?v=WxZK-UXKvXk

[stevehipwell]

@cartermckinnon I couldn't convince work to let me come over so I've been watching remotely with major FOMO! 😢

[zekena2]

What will be the container runtime when you rebase on AL2022? runc or crun?

[cartermckinnon]

@zekena2 We don't have any plans to switch to crun at this time, but it's a cool project that we'll keep an eye on. As it gains wider adoption, It might be a better fit for Bottlerocket; I reached out to the Bottlerocket team and they recommended you open an issue to track it: https://github.com/bottlerocket-os/bottlerocket

[dims]

xref: aws/containers-roadmap#1953

[cartermckinnon]

We'll have the proper systemd and kernel versions once we rebase on AL2022, so we should be able to target this for 1.25 (assuming the GA dates align).

The GA dates haven't aligned, unfortunately. My current understanding is that AL2022 (now known as AL2023) is preparing to cut an initial release candidate, so we're still a few steps away from GA.

[stevehipwell]

@cartermckinnon Bottlerocket is moving to cgroup v2 for all new variants from v1.13.0 which effectively means EKS v1.26. Is this still blocked on AL2023 or could it be introduced in AL2?

[dims]

Also Default to cgroup v2 for dev and k8s-1.26 variants

[stewartsmith]

group v2 on Amazon Linux 2 would be a "you get to keep both pieces" kind of environment. Moving to Amazon Linux 2023 is where this would be supported by the Amazon Linux team. #1340 appears to have been merged, so there's at least a WIP / PoC there.

[stevehipwell]

Can we get runc v1.1.9 as a dependency here due to an incorrect reporting of resource usage prior to this version.

[cartermckinnon]

I'm going to close this, our first AL2023-based AMIs are available 👍