Removing kube-proxy and kube-flannel after cluster created

[jlowe64]

Hello,

Is there any way to remove kube-proxy and flannel after the cluster has been created in order to use fully the Cilium CNI? I've tried to do this multiple times but any time the system reboots or I perform a k8s upgrade, even if the machine config is changed, it just resorts back to kube-proxy and flannel being installed along side Cilium. I also can't figure out if there's any way to remove kube-proxy's iptables after removing them.

So far I have removed daemonsets, configmaps, and edited the machine config to not reference kube-proxy, as well as I have also patched it with cni set to none, so that cilium can be managed externally.

I'm also having a lot of unpredictable behavior having two CNI and the features duplicated between Cilium and kube-proxy/flannel.

Any help or ideas appreciated!

Thanks!

[smira]

Talos controlplane nodes (if you have more than one) try to sync bootstrap manifests at least on reboot, might do it on config change, etc.

So you want to disable Flannel and kube-proxy with the machine config.

You can verify that they are disabled with talosctl get manifests -n <node> for each controlplane node. They should all not have Flannel/kube-proxy manifests, after that things should be fine.

[gecube]

Hi! We need a proper instructions with the precise steps how to remove flannel and kube-proxy on working cluster... Right now it is not clear how to properly remove them.

[smira]

Changing CNI on the fly is not something that I would recommend to do for anyone who doesn't feel to be a Kubernetes expert.

But the answer is above, there's nothing magic, it's all Kubernetes manifests that should be removed (from Kubernetes) after Talos part is disabled not to to try apply them back.

[gecube]

@smira Agree, but sometimes it is simpler than just to recreate cluster from the scratch. For me the most complex part is that the manifests for kube-proxy and flannel are hidden in configs and I need to guess how they were installed.

[smira]

They are not hidden, when enabled, talosctl get manifests shows them all in a pretty explicit way.

[gecube]

@smira but not after the changing the configuration... After changing machine description, the talosctl get manifests won't show them as it does not already manage them. Right?