Problems arising from Podman and k8s cohabiting /etc/cni/net.d

[sysrich]

Is this a BUG REPORT or FEATURE REQUEST?

/kind feature

Description

This is bug/feature/discussion point/plea for help

The problem at its heart is arguably not a problem with Podman, but a container ecosystem problem which makes Podmans life hell

CNI can be a mess, and the needs of podman and the needs of a configured cri-o/k8s stack can quite easily cause chaos in the real world

In openSUSE Kubic we see the problem through the following example

Following the guidance in https://github.com/containers/libpod/blob/master/cni/README.md we put our example config in /etc/cni/net.d

And at this point, networking in podman works

Then we setup Kubernetes, and use a manifest like https://raw.githubusercontent.com/coreos/flannel/bc79dd1505b0c8681ece4de4c0d86c5cd2643275/Documentation/kube-flannel.yml to configure our CNI provider

At which point, networking in podman is broken because flannel CNI's config is unversioned and podman (rightly) cant make any sense of the combined configurations that now exist in /etc/cni/net.d.

But we can get podman working by configuring cni_default_network in libpod.conf

But at this point, networking in k8s/cri-o is still utterly broken because that still cant understand the combined configurations living in /etc/cni/net.d

So as much as like to say "everything is fine" because podman is working, in reality the situation is really quite unpalatable because I actually need k8s to run stuff also ;)

In openSUSE Kubic we're moving to avoid the issue by not installing podman and cri-o on the same systems.

We're expecting k8s systems to use cri-o with a k8s focused CNI solution and stand-alone container hosts to use podman with a podman-centric CNI solution.

But this undermines the intended role of podman as a debugging tool for cri-o, so a better solution is needed long term.

So, as I see it, we have a bunch of questions we need to find an answer to;

• How important is the same network config when podman/cri-o are cohabiting the same system? If it's critical, then maybe podman needs to be as laissez-faire with accepting imperfect CNI configurations as k8s is.

• Can cri-o/k8s do something akin to podmans cni_default_network magic? If so, that might be a solution, but I cant find any sign of such a solution in any documentation anywhere.

• Would it make more sense for podman to use a separate folder for it's CNI configuration?

• Or is there a better solution that I'm missing here?

What do we all think?

Thanks in advance for your thoughts :)

[mheon]

Hm.

@mrunalp Any objections to adding a cni_default_network option to crio.conf to specify which CNI network to use by default?

[rhatdan]

@mrunalp @baude We need to clean this up? Seems a shame that these can not easily share the same CNI Infrastructure.

[rhatdan]

@mheon @baude Do we still have this issue?

[mheon]

Yes.

This should be moved to CRI-O though. Podman is working fine in this case. CRI-O needs the cni_default_network change

[rhatdan]

@sysrich Would you mind opening this same issue on CRI-O?

[sysrich]

@rhatdan done :) cri-o/cri-o#2121