Connect: allow configuration of upstream connection pool limits

[banks]

Scope:

• No RFC needed
• Some minor UX work to decide best config to support that will be maximally future compatible with other proxy integrations. The rationale can be given in PR description and signed off as part of review.
• O(days) task to add the config, plumb though to Envoy and write tests (assuming familiarity)

---

Allow configuration of Envoy's client side connection/request limiting.

Note this is NOT the global rate limit filter that calls an external service which is most of what you find in the envoy docs for rate limiting.

Envoy somewhat confusingly calls this "Circuit Breakers". (The thing most other systems call a "Circuit breaker" Envoy calls "Outlier Detection"). See https://www.envoyproxy.io/docs/envoy/latest/api-v2/api/v2/cluster/circuit_breaker.proto.

I propose allowing both max_connections and max_pending_requests/max_requests to be controlled in appropriate services (depends on the protocol being L4 vs L7).

Bear in mind that the UX we build to expose these needs to be general enough to map to other proxies reasonably well too. The semantics of max_[pending_]requests are not exactly a "rate limit" in terms of req/s but I think HAProxy and Nginx can both support similar semantics to these: e.g. HAProxy maxconn behaves the same as max_requests I think (please check) and maxqueue like max_pending_requests. nginx has max_conns and queue which seem equivalent but are in Commercial version only. Exact identical behaviour is not so important more just that we are not adding things to our first-class config UX that can't work in anything but Envoy or works wildly differently everywhere else etc.

[ghost]

Hey there,

This issue has been automatically locked because it is closed and there hasn't been any activity for at least 30 days.

If you are still experiencing problems, or still have questions, feel free to open a new one 👍.