NVIDIA Docker failed to start container due to cgroup v2

[Abdillah]

Describe the bug
NVIDIA Docker (virtualisation.docker.enableNvidia) cannot be used on default NixOS option due to cgroup v2 not supported by libnvidia-container (the error, root cause). The container refuse to spawn because this runtime error.

$ nvidia-docker run -it -p 3000:3000 mycroft/mimic2:gpu

docker: Error response from daemon: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: Running hook #1:: error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: container error: cgroup subsystem devices not found: unknown.
ERRO[0003] error waiting for container: context canceled 

There are two potential solutions as NVIDIA/libnvidia-container#111 (comment),

1. Change systemd to hybrid mode when NVIDIA enabled (systemd.enableUnifiedCgroupHierarchy = false;)
2. Switching off cgroup v2 support in nvidia-container-runtime per Non-default nvidia-container-runtime-hook config file NVIDIA/nvidia-container-runtime#47 (comment).

   I encountered this issue exactly because I'm running rootless docker with nvidia runtime using your usernetes. Everything works, except have to set no-cgroups = true in /etc/nvidia-container-runtime/config.toml

To Reproduce
Steps to reproduce the behavior:

1. Clone mycroft/mimic2 repository and enter the directory. This might be any repository or docker image with gpu requirements.
2. Execute the build command docker build -t mycroft/mimic2:gpu -f gpu.Dockerfile .
3. Execute the run command nvidia-docker run -it -p 3000:3000 mycroft/mimic2:gpu.

Expected behavior
Run happily ever after.

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.10.40, NixOS, 21.11pre293089.1c2986bbb80 (Porcupine)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.4pre20210503_6d2553a`
 - channels(root): `"nixos-21.11pre293089.1c2986bbb80, nixos-hardware, nixos-unstable-21.05pre283367.0a5f5bab0e0"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
- systemd.enableUnifiedCgroupHierarchy
- virtualisation.docker.enableNvidia

# a list of nixos modules affected by the problem
module:
- systemd
- nvidia-docker

[Abdillah]

Cgroupv2 is now supported by https://github.com/NVIDIA/libnvidia-container per v1.8.0 release.