Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cgroups v2 support #60

Closed
dtrudg opened this issue May 27, 2021 · 1 comment · Fixed by #292
Closed

cgroups v2 support #60

dtrudg opened this issue May 27, 2021 · 1 comment · Fixed by #292
Assignees
@dtrudg @tri-adam
Labels
enhancement New feature or request roadmap Features / changes that are scheduled to be implemented techdebt
Milestone
SingularityCE 3.9.0

Comments

@dtrudg
Copy link
Member

dtrudg commented May 27, 2021

Is your feature request related to a problem? Please describe.

SingularityCE will not work completely out of the box (OCI mode / cgroups settings) on distributions that are using the cgroups v2 hierarchy.

Fedora is using cgroups v2. Also Debian bullseye (the next release, currently testing). We can reasonably expect the next Ubuntu LTS and the next EL release to do so also.

Describe the solution you'd like

SingularityCE should support the cgroups v2 hierarchy.
@dtrudg dtrudg added enhancement New feature or request techdebt labels May 27, 2021
@dtrudg
Copy link
Member Author

dtrudg commented Jun 2, 2021

We use a TOML cgroups config format that is defined here: https://github.com/sylabs/singularity/blob/master/internal/pkg/cgroups/config_linux.go

This is closely related to the opencontainers/runtime-spec/specs-go constructs: https://github.com/opencontainers/runtime-spec/blob/master/specs-go/config.go

At present, the v1 cgroups config is applied via containerd/cgroups routines that directly accept LinuxResources (s below):

singularity/internal/pkg/cgroups/cgroups_linux.go

Line 79 in 1fbbe63

m.cgroup, err = cgroups.New(cgroups.V1, path, s)

For cgroups v2 a v2.LoadManager followed by a v2.NewManager call will setup resource limits. The resources type is now v2.Resources (https://github.com/imilchev/cgroups/blob/dd572fc860afa53ef84ec0a8cd040b3e86f78bb4/v2/manager.go#L59) - we cannot simply pass the specs-go LinuxResources type, even though it has had a 'Unified' attribute added to support representing v2.

We need to investigate a way of moving forward... how the existing toml structure maps into the v2.Resources etc.

@dtrudg dtrudg added this to the SingularityCE 3.9.0 milestone Jun 2, 2021
@dtrudg dtrudg added the roadmap Features / changes that are scheduled to be implemented label Jul 2, 2021
@dtrudg dtrudg mentioned this issue Aug 31, 2021
Merged
dtrudg referenced this issue in dtrudg/singularity Aug 31, 2021
@dtrudg
cgroups v2 support (apply from v1/OCI spec)
7ff8e91 
Support execution on a cgroups v2 unified hierarchy system, by using
the `containerd/cgroups/v2` translation of the
`opencontainers/runtime-spec` `LinuxResources` structure (which we use
for cgroups config toml files) into v2 Resources. Device limits are
implemented with the eBPF translation.

This will mean that existing v1 / OCI style cgroups configuration
files can be used on a system configured for the unified hierarchy
without an ugly failure. Also fixes failures with the `oci` commands
on these systems.

Note that we don't yet support the Unified attribute from
LinuxResources, which is a free-form map[string][string] for
expressing cgroups v2 configuration natively.

Fixes #60
This was referenced Aug 31, 2021
Closed
Closed
dtrudg referenced this issue in dtrudg/singularity Aug 31, 2021
@dtrudg
cgroups v2 support (apply from v1/OCI spec)
e2d2207 
Support execution on a cgroups v2 unified hierarchy system, by using
the `containerd/cgroups/v2` translation of the
`opencontainers/runtime-spec` `LinuxResources` structure (which we use
for cgroups config toml files) into v2 Resources. Device limits are
implemented with the eBPF translation.

This will mean that existing v1 / OCI style cgroups configuration
files can be used on a system configured for the unified hierarchy
without an ugly failure. Also fixes failures with the `oci` commands
on these systems.

Note that we don't yet support the Unified attribute from
LinuxResources, which is a free-form map[string][string] for
expressing cgroups v2 configuration natively.

Fixes #60
dtrudg referenced this issue in dtrudg/singularity Aug 31, 2021
@dtrudg
cgroups v2 support (apply from v1/OCI spec)
5503bc5 
Support execution on a cgroups v2 unified hierarchy system, by using
the `containerd/cgroups/v2` translation of the
`opencontainers/runtime-spec` `LinuxResources` structure (which we use
for cgroups config toml files) into v2 Resources. Device limits are
implemented with the eBPF translation.

This will mean that existing v1 / OCI style cgroups configuration
files can be used on a system configured for the unified hierarchy
without an ugly failure. Also fixes failures with the `oci` commands
on these systems.

Note that we don't yet support the Unified attribute from
LinuxResources, which is a free-form map[string][string] for
expressing cgroups v2 configuration natively.

Fixes #60
This was referenced Oct 27, 2021
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dtrudg dtrudg

@tri-adam tri-adam

Labels
enhancement New feature or request roadmap Features / changes that are scheduled to be implemented techdebt
Projects
None yet
Milestone
SingularityCE 3.9.0
Development

Successfully merging a pull request may close this issue.

cgroups v2 support (apply from v1 / OCI spec) dtrudg/singularity
2 participants
@dtrudg @tri-adam