Support providing just a CA and key for transport certs

[dbason]

Proposal

The operator should support the option to just provide a CA cert and key and then generate the self signed transport certs from that.
Use case. Why is this important?
This is primarily to support cross cluster connections. It would greatly simplify work of connecting up multiple clusters if we could provide a CA that would then be trusted by all of the clusters. AFAIK the elasticsearch keygen tool supports this so hopefully it is just a matter of wiring this up?

[dbason]

I'm having a poke around the code now and I think this looks doable. Full disclosure I'm not a developer in any way shape or form, but I'll see if I can put together something to start this.

At this point I'm looking to update the transportSpec in elasticsearch and provide a SecretRef there to allow the transport certs to be signed with a predefined CA. This would allow a user administering multiple clusters to generate a single CA that would then be trusted for all their clusters; simplifying the work of setting up remote search.

Something like this? dbason@1d53be2

[sebgl]

Another way of handling this (not sure if better) is to use the established naming convention.
I think if you pre-create the <es-name>-es-transport-ca-internal secret, the operator will detect it's already there and, if valid, will use it to sign certificates.

More generally I don't know if this feature should be ECK's responsibility, vs. using something such as CertManager. Happy to hear other people thoughts about it.

[dbason]

@sebgl the comment about CertManager seems fair, but there would still need to be a change to allow custom transport certs. At the moment there is only the option to provide custom http certs.

I'll change the title and initial post to reflect that this is explicitly for transport certs

[MHenn1g]

This feature would be a great enhancement, if not to say necessity for one of our use-cases, in which we would need to link an external (kind of static) application to elasticsearch via the transport interface.
Not only would it enhance the automation, but at the same time the stability of operations.
At the moment, we are able to link the software, however, if the operator handles the ca management, this connection may break if the operator decides to rotate the ca unexpectedly.
Therefore I would like to ask, if there is any update regarding this request?

[sebgl]

@dbason @MHenn1g We discussed this with the team and decided we should do it. No release date we can give yet, however.

[deveshk0]

is there any plan to support the same with http layer?

[pebrc]

is there any plan to support the same with http layer?

Please see https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-tls-certificates.html#k8s-setting-up-your-own-certificate