Support DNS SRV for external etcd endpoints discovery

[YanzheL]

Is this a request for help?

no

What keywords did you search in kubeadm issues before filing this one?

external etcd, DNS SRV

Is this a BUG REPORT or FEATURE REQUEST?

FEATURE REQUEST

Versions

kubeadm version (use kubeadm version): 1.16.3

Environment:

• Kubernetes version (use kubectl version): 1.16.3

What happened?

Currently, we can configure external etcd endpoints in kubeadm-config.yaml like this

apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
...
etcd:
    external:
        endpoints:
        - https://ETCD_0_IP:2379
        - https://ETCD_1_IP:2379
        - https://ETCD_2_IP:2379
....

It needs me to provide all endpoints of external etcd cluster nodes, which lacks a mechanism of "auto discovery".

What you expected to happen?

Inspired by etcd cluster DNS discovery, etcd cluster nodes support DNS SRV lookup via etcd argument --discovery-srv etcd.example.com to discover other nodes, I suggest that kubeadm can use a similar way to discover external etcd endpoints.

For example:

...
etcd:
    external:
        endpoints:
        - srv://etcd.example.com
....

Anything else we need to know?

Their implementation for this feature: etcd-io/etcd#8281

[neolit123]

hi, this request is related to:
#1598
but for external etcd.

It needs me to provide all endpoints of external etcd cluster nodes, which lacks a mechanism of "auto discovery".

you can also put your external etcd nodes behind a load balancer and only list the LB endpoint instead of all the members.

AFAIK, this will not work:

etcd:
    external:
        endpoints:
        - srv://etcd.example.com

because in the case of external etcd the list of endpoints (or a single LB) are simply passed on the kube-apiserver with (TLS ensured). for external etcd kubeadm does not manage etcd members and the DNS discovery.

this feels to me like a feature request for the kube-apiserver.

i'm going to keep this open for a discussion, but i don't think there is much we can do on the kubeadm side.

@kubernetes/sig-cluster-lifecycle

[YanzheL]

Thanks.
I understand I can use load balancer, but it brings more complexity to setup a LB.

In a simple case without load balancer, it'd be better if we can just use DNS discovery, so we don't need to change the kubeadm config every time when a new etcd member joins the cluster.
The member change in etcd cluster shouldn't affect the kubeadm config.

[neolit123]

The dns srv must be passed to the kube-apiserver and i dont think it supports it currently.

[neolit123]

The dns srv must be passed to the kube-apiserver and i dont think it
supports it currently.

to confirm that you can go to the k8s slack channel #sig-api-machinery and ask there if the kube-apiserver supports this. i don't think it does, which means that we can't do much on the kubeadm side as the api server still needs a list of etcd members URLs.

[neolit123]

/close
please re-open ionce the above question is answered.

[k8s-ci-robot]

@neolit123: Closing this issue.

In response to this:

/close
please re-open ionce the above question is answered.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.