K3s build windows (no agents)

[KnicKnic]

I got k3s building on windows.
I essentially disabled the following components

1. rootless
2. netpool
3. syssetup

I also disabled a few chmod of files.

This can then run a kubernetes control plane. It does not run the worker nodes. You need to --disable-agents and --disable-network-policy

.\k3s.exe server --disable-agent -d c:\tmp\k3s --disable-network-policy

This is related to issue #114 as compiling on windows will help that goal.

[erikwilson]

Thanks for the PR @KnicKnic! Do you have any thoughts on what needs to be done in order for the agent to work?

[KnicKnic]

Thanks for the PR @KnicKnic! Do you have any thoughts on what needs to be done in order for the agent to work?

Well a few days ago I had the same thought, so I did a minimal effort to what would it take to start the agents. Check #114 (comment) essentially I disabled a few checks (link to my branch is in the comment). The kubelet starts up and registers, however networking/CNI was not configured. Honestly I don't know too much about networking configuration and don't wanna learn seeing how I didn't find a doc that explains it in a straightforward doc, they just wrote powershell that automate everything for you. Today I pinged a few more people that have contributed to rancher that have touched windows, maybe they can help with setting up flannel to get past that. Maybe you could help with traction here @erikwilson ?

Now you have a storageless master node you could schedule pods against?

After that comes storage.

• I see that documentation exists here https://github.com/rancher/local-path-provisioner/blob/master/README.md#usage .
  • Have not read it
  • Possibly creating something similar using FSRM or the likes for enforced quotas of hostpath sizes.
• Or use NFS/SMB server for multinode (or singlenode) access.
  • Potentially just leveraging some CSI plugin.
    • One small work item is would you probably need to bind the CSI proxy into k3s to leverage this.

Also comes the k3s tunnel-proxy .
I have no clue how it works, or if it comes into play when starting a master. However the master did join. Maybe it just worked? Or maybe only the workers would need to use it. Have zero knowledge here of what the component is here or why isn't directly a connection to the api server endpoint used?

Also I don't know much about how cert rotation happens in k3s, I assume it would be the same on windows?

I know you start some deployments, coredns, traefik, metric-server, (helm?) ... I have run helm, coredns & traefik on windows before, someone would need to publish images with windows for multiarch, also verify that nothing like chmod happens that breaks their cross platform operation.
I didn't see any daemonsets, but if you have any, those would need to get fixed if they require privilege. Most likely the simplest fix is elevating them into k3s, as then they would have privilege in windows, however you lose the ability to as easily configure due to not mounting secrets / config maps.

[erikwilson]

Thanks for the info! As far as I can tell our bundled version of flannel should have windows support, we default to vxlan (https://github.com/rancher/flannel/tree/v0.11.0-k3s.1/backend/vxlan) but it looks like host-gw should also work. From my understanding containerd should work in windows also.

From looking at https://github.com/coreos/flannel/blob/master/Documentation/backends.md and https://docs.microsoft.com/en-us/virtualization/windowscontainers/kubernetes/network-topologies it may just be a configuration issue where we need to add something like VNI to https://github.com/rancher/k3s/blob/master/pkg/agent/flannel/setup.go

For the k3s tunnel-proxy, I think it should be okay because it is pure go and just creates websocket connections. It is actually used so the api-server can connect to kubelet or containerd on the nodes. The important part is that the server node ips are accessible to agents. Using a single node server with agent enabled is probably the easiest way to get this working.

I think cert rotation should also be okay or a small effort to fix.

Merging this and hopefully we can get the other stuff worked out.