proxy: Prevent unavailable services from leaking resources

[olix0r]

Typically, idle services are evicted from the proxy (after ~60s without new requests). However, if the service is not available (i.e. because there are no endpoints in the service), then buffers for this service can stop processing requests and fill up. Even though requests are canceled, buffer space is never created and the buffer's worker can never detect that the send-side of the buffer has been dropped. This results in a resource leak including metrics and discovery resolutions.

In order to address this situation, I believe that we should modify the buffering strategy to support cancelation such that the buffer's worker can properly detect the situation when its caller has been dropped and all of its requests have been canceled; and I think this means that we need to replace the tower::buffer layer with an implementation that supports cancelation natively.

[olix0r]

linkerd/linkerd2-proxy#401 ensures service discovery resolutions are dropped when the resolution buffer is full for 60s, indicating the balancer hasn't been polled.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[olix0r]

This has been addressed by linkerd/linkerd2-proxy#456