Kubernetes HA - investigate HA Proxy

[zgorizzo69]

Investigate HA proxy to see if it fits for our HA deployment of kubernetes

• Describe setup
• Document inner working
• POC

linked to https://github.com/threefoldtech/home/issues/547

[zgorizzo69]

on branch k3s-ha folder /docs/kubernetes/ressources/ha/docker
There is a makefile to try the HA setup with HAProxy and keepalive for the FIX registration IP around masters.

The distributed database is ensure by DQlite. There is a bug if you try to kill master1 dqlite does'nt seems to be able to elect a new leader check this. other than that it works fine.

HAProxy configuration can be found here /docs/kubernetes/ressources/ha/docker/scripts/haproxy.sh

Keepalived configuration can be found here /docs/kubernetes/ressources/ha/docker/scripts/keepalived.sh

@LeeSmet Can you base on these adapt the tfclient to be able to deploy master nodes with a HA setup ? Like an option to deploy by default 3 master nodes with HA Proxy and keepalived configured and start at launch and with k3s launched with the correct setting (see dcoker-compose file) so that I can test and adapt my kubernetes/startup script and doc ?

[zgorizzo69]

we have to first start master 1 with HAProxy listening on 8443 and keep alive assigning it a virtual IP.
Only then we should start the master 2 and 3 with the parameter --server https://{VIRTUAL_IP}:8443
Otherwise there will be no LB if we bind then all to master1 IP

[DylanVerstraete]

Closing because we don't have any current plans for this in grid 3.0