Fix CI create multi node cluster failures by nerdctl

[yankay]

The CI uses the container installed by apt-get install docker-ce. The version may not be right.
So, it has been changed to the "nerdctl full package" to fix CI nerdctl test failures.

Fixes #3533

[yankay]

The error logs change to

Command Output: time="2024-02-28T06:55:49Z" level=fatal msg="failed to verify networking settings: failed to create default network: subnet 10.4.0.0/24 overlaps with other one on this address space"

It may relate to: containerd/nerdctl#1371

[aojea]

I prefer to have pinned dependencies in ci and not a moving target

[yankay]

Thanks @aojea

It has been changed by using the containerd.service included by nerd-full pacakge.

[aojea]

why are these no longer required?

[yankay]

HI @aojea

Because the nerdctl-full contains the CNI dependency, it also includes:

# nerdctl (full distribution)
- nerdctl: v1.7.4
- containerd: v1.7.13
- runc: v1.1.12
- CNI plugins: v1.4.0
- BuildKit: v0.12.5
- Stargz Snapshotter: v0.15.1
- imgcrypt: v1.1.9
- RootlessKit: v2.0.1
- slirp4netns: v1.2.2
- bypass4netns: v0.4.0
- fuse-overlayfs: v1.13
- containerd-fuse-overlayfs: v1.0.8
- Kubo (IPFS): v0.26.0
- Tini: v0.19.0
- buildg: v0.4.1

ref to: https://github.com/containerd/nerdctl/releases/tag/v1.7.4

Using the nerdctl-full instead of Minimal Package can make the CI easy to maintaince, because the nerdctl manage the dependency version mapping.

[aojea]

/lgtm
/approve

Thanks for keeping the CI healthy

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aojea, yankay

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [aojea]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[aojea]

I´ve seen this failure https://github.com/kubernetes-sigs/kind/actions/runs/8067092392/job/22036693761?pr=3530

ERROR: failed to create cluster: command "nerdctl run --name kind-worker2 --hostname kind-worker2 --label io.x-k8s.kind.role=worker --privileged --security-opt seccomp=unconfined --security-opt apparmor=unconfined --tmpfs /tmp --tmpfs /run --volume /var --volume /lib/modules:/lib/modules:ro -e KIND_EXPERIMENTAL_CONTAINERD_SNAPSHOTTER --detach --tty --label io.x-k8s.kind.cluster=kind --net kind --restart=on-failure:1 --init=false kindest/node:v1.29.2@sha256:51a1434a5397193442f0be2a297b488b6c919ce8a3931be0ce822606ea5ca245" failed with error: exit status 1

Command Output: time="2024-02-27T15:19:47Z" level=fatal msg="failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: time="2024-02-27T15:19:42Z" level=fatal msg="failed to call cni.Setup: plugin type=\"firewall\" failed (add): running [/usr/sbin/iptables -t filter -N CNI-FORWARD --wait]: exit status 4: iptables v1.8.7 (nf_tables): CHAIN_USER_ADD failed (File exists): chain CNI-FORWARD\n"\nFailed to write to log, write /var/lib/nerdctl/1935db59/containers/default/6d22f642f1a5ec76f7a3b0ad7d1852ba8002b5572215f8713e4b95f3bc62ed7a/oci-hook.createRuntime.log: file already closed: unknown"

[aojea]

This is still failing

https://github.com/kubernetes-sigs/kind/actions/runs/8505888448/job/23295140055?pr=3563

RROR: failed to create cluster: command "nerdctl run --name kind-worker2 --hostname kind-worker2 --label io.x-k8s.kind.role=worker --privileged --security-opt seccomp=unconfined --security-opt apparmor=unconfined --tmpfs /tmp --tmpfs /run --volume /var --volume /lib/modules:/lib/modules:ro -e KIND_EXPERIMENTAL_CONTAINERD_SNAPSHOTTER --detach --tty --label io.x-k8s.kind.cluster=kind --net kind --restart=on-failure:1 --init=false kindest/node:v1.29.2@sha256:51a1434a5397193442f0be2a297b488b6c919ce8a3931be0ce822606ea5ca245" failed with error: exit status 1

Command Output: time="2024-04-01T08:34:37Z" level=fatal msg="failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: time=\"2024-04-01T08:34:34Z\" level=fatal msg=\"failed to call cni.Setup: plugin type=\\\"firewall\\\" failed (add): running [/usr/sbin/iptables -t filter -N CNI-ISOLATION-STAGE-2 --wait]: exit status 4: iptables v1.8.7 (nf_tables):  CHAIN_USER_ADD failed (File exists): chain CNI-ISOLATION-STAGE-2\\n\"\nFailed to write to log, write /var/lib/nerdctl/1935db59/containers/default/18e88fcb538d49417539810b[25](https://github.com/kubernetes-sigs/kind/actions/runs/8505888448/job/23295140055?pr=3563#step:8:26)67922886e120771be00f165b5d64cf41a381f5/oci-hook.createRuntime.log: file already closed: unknown"

Stack Trace: 
sigs.k8s.io/kind/pkg/errors.WithStack
	sigs.k8s.io/kind/pkg/errors/errors.go:59
sigs.k8s.io/kind/pkg/exec.(*LocalCmd).Run
	sigs.k8s.io/kind/pkg/exec/local.go:124
sigs.k8s.io/kind/pkg/cluster/internal/providers/nerdctl.createContainerWithWaitUntilSystemdReachesMultiUserSystem
	sigs.k8s.io/kind/pkg/cluster/internal/providers/nerdctl/provision.go:383
sigs.k8s.io/kind/pkg/cluster/internal/providers/nerdctl.planCreation.func3
	sigs.k8s.io/kind/pkg/cluster/internal/providers/nerdctl/provision.go:123
sigs.k8s.io/kind/pkg/errors.UntilErrorConcurrent.func1
	sigs.k8s.io/kind/pkg/errors/concurrent.go:30
runtime.goexit
	runtime/asm_amd64.s:1598
Error: Process completed with exit code 1.