Can not run kata-containers with nerdctl in rootless mode

[lisongmin]

Hi,

When i try to run kata-containers with nerdctl in rootless mode, there is a fatal error

$ nerdctl --debug run --runtime io.containerd.kata.v2 -it alpine sh
DEBU[0000] rootless parent main: executing "/usr/bin/nsenter" with [-r/ -w/home/lsm/work/github/nerdctl --preserve-credentials -m -n -U -t 424166 -F nerdctl --debug run --runtime io.containerd.kata.v2 -it alpine sh] 
WARN[0000] To isolate bridge networks, CNI plugin "isolation" needs to be installed in CNI_PATH ("/opt/cni/bin"), see https://github.com/AkihiroSuda/cni-isolation 
WARN[0000] cannot set cgroup manager to "systemd" for runtime "io.containerd.kata.v2" 
FATA[0000] operation not permitted: unknown  

logs in container shows:

$ nerdctl logs bcc7cdec6843
FATA[0000] failed to open "/home/lsm/.local/share/nerdctl/1935db59/containers/default/bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10/bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10-json.log", container is not created with `nerdctl run -d`?: stat /home/lsm/.local/share/nerdctl/1935db59/containers/default/bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10/bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10-json.log: no such file or directory 

the specified directory only has resolv.conf file:

$ ls -l /home/lsm/.local/share/nerdctl/1935db59/containers/default/bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10                                                                                                       
Alias tip: ll /home/lsm/.local/share/nerdctl/1935db59/containers/default/bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10
total 4
-rw-r--r-- 1 lsm lsm 57 Mar 24 20:20 resolv.conf

the containerd logs:

Mar 24 20:20:59 stable containerd-rootless.sh[424191]: time="2021-03-24T20:20:59.973694862+08:00" level=debug msg="stat snapshot" key="sha256:cb381a32b2296e4eb5af3f84092a2e6685e88adbc54ee0768a1a1010ce6376c7"
Mar 24 20:20:59 stable containerd-rootless.sh[424191]: time="2021-03-24T20:20:59.986019983+08:00" level=debug msg="prepare snapshot" key=bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10 parent="sha256:cb381a32b2296e4eb5af3f84092a2e6685e88adbc54ee0768a1a1010ce6376c7"
Mar 24 20:21:00 stable containerd-rootless.sh[424191]: time="2021-03-24T20:21:00.005997956+08:00" level=debug msg="event published" ns=default topic=/snapshot/prepare type=containerd.events.SnapshotPrepare
Mar 24 20:21:00 stable containerd-rootless.sh[424191]: time="2021-03-24T20:21:00.009349200+08:00" level=debug msg="get snapshot mounts" key=bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10
Mar 24 20:21:00 stable containerd-rootless.sh[424191]: time="2021-03-24T20:21:00.052399224+08:00" level=debug msg="event published" ns=default topic=/containers/create type=containerd.events.ContainerCreate
Mar 24 20:21:00 stable containerd-rootless.sh[424191]: time="2021-03-24T20:21:00.061838750+08:00" level=debug msg="get snapshot mounts" key=bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10
Mar 24 20:21:00 stable containerd-rootless.sh[424191]: time="2021-03-24T20:21:00.141498645+08:00" level=debug msg="registering ttrpc server"
Mar 24 20:21:00 stable containerd-rootless.sh[424191]: time="2021-03-24T20:21:00.141667806+08:00" level=debug msg="serving api on abstract socket" socket="[inherited from parent]"
Mar 24 20:21:00 stable containerd-rootless.sh[424191]: time="2021-03-24T20:21:00.141700703+08:00" level=info msg="starting signal loop" namespace=default path=/run/containerd/io.containerd.runtime.v2.task/default/bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10 pid=425356
Mar 24 20:21:00 stable containerd-rootless.sh[424191]: time="2021-03-24T20:21:00.142074716+08:00" level=debug msg="converting /run/containerd/io.containerd.runtime.v2.task/default/bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10/config.json" pid=425356 sandbox=bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10 source=virtcontainers subsystem=compatoci
Mar 24 20:21:00 stable kata[425356]: time="2021-03-24T20:21:00.146501275+08:00" level=info msg="loaded configuration" file=/etc/.ro921613803/kata-containers/configuration.toml format=TOML pid=425356 sandbox=bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10 source=katautils
Mar 24 20:21:00 stable kata[425356]: time="2021-03-24T20:21:00.146660798+08:00" level=info msg="IOMMUPlatform is disabled by default." pid=425356 sandbox=bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10 source=katautils
Mar 24 20:21:00 stable containerd-rootless.sh[424191]: time="2021-03-24T20:21:00.146501275+08:00" level=info msg="loaded configuration" file=/etc/.ro921613803/kata-containers/configuration.toml format=TOML pid=425356 sandbox=bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10 source=katautils
Mar 24 20:21:00 stable containerd-rootless.sh[424191]: time="2021-03-24T20:21:00.146660798+08:00" level=info msg="IOMMUPlatform is disabled by default." pid=425356 sandbox=bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10 source=katautils
Mar 24 20:21:00 stable containerd-rootless.sh[424191]: time="2021-03-24T20:21:00.147275144+08:00" level=info msg="shm-size detected: 67108864" source=virtcontainers subsystem=oci
Mar 24 20:21:00 stable kata[425356]: time="2021-03-24T20:21:00.147275144+08:00" level=info msg="shm-size detected: 67108864" source=virtcontainers subsystem=oci
Mar 24 20:21:00 stable kata[425356]: time="2021-03-24T20:21:00.148721538+08:00" level=info msg="create netns" netns=/run/user/1000/netns/net-b37cc011-a5c1-e7df-e334-25ec56812f79 pid=425356 sandbox=bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10 source=katautils
Mar 24 20:21:00 stable containerd-rootless.sh[424191]: time="2021-03-24T20:21:00.148721538+08:00" level=info msg="create netns" netns=/run/user/1000/netns/net-b37cc011-a5c1-e7df-e334-25ec56812f79 pid=425356 sandbox=bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10 source=katautils
Mar 24 20:21:00 stable containerd-rootless.sh[424191]: time="2021-03-24T20:21:00.202585125+08:00" level=debug msg="garbage collected" d=6.465577ms
Mar 24 20:21:00 stable containerd-rootless.sh[424191]: time="2021-03-24T20:21:00.215223216+08:00" level=debug msg="failed to delete task" error="rpc error: code = NotFound desc = container does not exist bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10: not found" id=bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10
Mar 24 20:21:00 stable containerd-rootless.sh[424191]: time="2021-03-24T20:21:00.217176886+08:00" level=info msg="shim disconnected" id=bcc7cdec68437f9d9c91110d5dac7529838146cf06dfd67dbfb35128ec78fb10
Mar 24 20:21:00 stable containerd-rootless.sh[424191]: time="2021-03-24T20:21:00.217307144+08:00" level=error msg="copy shim log" error="read /proc/self/fd/16: file already closed"

[AkihiroSuda]

Does kata even support rootless?

[AkihiroSuda]

I don't think Kata supports rootless currently, so closing this one

[zibinpan]

kata 3.0 is now supported rootless. I followed the instruction to set the rootless for kata, and followed the doc to set the rootless for nerdctl. However, when I tried to run the following command to create a container in rootless:
nerdctl run -it --runtime io.containerd.kata.v2 ubuntu:20.04 sh
It raised error as:
FATA[0000] failed to create shim task: could not create VMM user: exit status 1: unknown

So how to fix this problem? Thank you.

@lisongmin @AkihiroSuda