possible optimization: repetitive private key reads

[howardjohn]

Title: possible optimization: repetitive private key reads

Description:
When investigating #19774, I found another area that may be improved (possibly). Currently, for each cluster/listener, we do a full boringssl processing of the same key/cert pair (I think). This ends up being fairly expensive at scale. With a large number of clusters, startup time is decreased from 5.5s to 4.5s in my tests when using 2048 bit RSA keys vs ECDSA keys (since they are cheaper to process).

RSA:

ECDSA:

I know very little about boringssl or the lifecycles here, but my naive thought is that it could be read once and shared among each cluster

cc @lambdai

[daixiang0]

Sounds possible if a cluster use only one Envoy, then all repetitive keys in the Envoy config could be read once.

BTW, is RSA still widely used even its poor performance?

[lambdai]

While a SSL_CTX is relative hard to share across clusters, my hunch is that private key and tls certificate are supposed to immutable and shared.

There are definitely lots of issues need to be resolved if we'd like to share cert chain among various TlsContext
'd like to hear some high level feedback from @PiotrSikora @lizan if cert_chain (aka X509) is shareable.

[PiotrSikora]

This can be definitely done. Most of the work is migration from X509 to BoringSSL's CRYPTO_BUFFER (already done in #10621, but that PR never landed) and then enabling CRYPTO_BUFFER_POOL.

[lambdai]

Thank you @PiotrSikora

Though the goal is slightly different, #10621 is a great source to understand the underlying load/verification.

By reading it, I can confirm lots of TODOs if I want to go with the shared X509, such as the same underlying cert chain bytes can be valid in ssl_ctx_A but invalid in ssl_ctx_B.

I don't understand if CRYPTO_BUFFER can save CPU cycles or memory footprint. If there are, what are they?

[PiotrSikora]

Though the goal is slightly different, #10621 is a great source to understand the underlying load/verification.

The stated goal is different, but 90%+ of the code is the same.

I don't understand if CRYPTO_BUFFER can save CPU cycles or memory footprint. If there are, what are they?

I suspect BoringSSL doesn't process new certificates if they already exist in the CRYPTO_BUFFER_POOL, so most likely both CPU and memory, but you'd have to verify that yourself.