OkHttpClient connection leak in OperationSupport / HttpResponse

[belks]

Hi,

while checking our logs we observed a warning from the OkHttpClient that a connection was leaked. As you can see in the stack trace at the very bottom of this ticket our application tried to call BaseOperation.list(), but ran into an unauthorized error. Shortly after that the OkHttpClient reports a connection leak.

Looking at your code we can see that in case of an unsuccessful response OperationSupport.assertResponseCode() calls OperationSupport.createStatus() which in turn gets the HTTP body string by calling HttpResponse.bodyString(). The default implementation of bodyString() of the HttpResponse interface then gets one of the SupportedResponses. In case of an input stream or reader this then calls IOHelpers.readFully. However the readFully does not close the stream that it is reading from. Our assumption is that in case of an error the SupportedResponses.INPUT_STREAM or SupportedResponses.READER is used. When looking at the OkHttpResponseImpl we see that the response body is closed in one case explicitly and for the string() and bytes() calls it is closed implicitly (according to the Kotlin sources of OkHttp). However, for the charStream() and byteStream() the response body should probably be closed be the consumer of the stream which is not happening in the SupportedResponses. And therefore the connection is leaked.

2022-12-31 22:08:19.788 WARN  [okhttp3.OkHttpClient] A connection to https://172.20.0.1/ was leaked. Did you forget to close a response body? To see where this was allocated, set the OkHttpClient logger level to FINE: Logger.getLogger(OkHttpClient.class.getName()).setLevel(Level.FINE);

2022-12-31 22:03:18.952 [...] WARN io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://172.20.0.1:443/api/v1/nodes?labelSelector=test%2Ftier%3Dapp. Message: Unauthorized! Configured service account doesn't have access. Service account may have been revoked. Unauthorized.
	at io.fabric8.kubernetes.client.KubernetesClientException.copyAsCause(KubernetesClientException.java:238) ~[kubernetes-client-api-6.3.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.waitForResult(OperationSupport.java:536) ~[kubernetes-client-6.3.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.list(BaseOperation.java:418) ~[kubernetes-client-6.3.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.list(BaseOperation.java:383) ~[kubernetes-client-6.3.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.list(BaseOperation.java:93) ~[kubernetes-client-6.3.1.jar:?]
[...]
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://172.20.0.1:443/api/v1/nodes?labelSelector=test%2Ftier%3Dapp. Message: Unauthorized! Configured service account doesn't have access. Service account may have been revoked. Unauthorized.
at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.requestFailure(OperationSupport.java:742) ~[kubernetes-client-6.3.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.requestFailure(OperationSupport.java:722) ~[kubernetes-client-6.3.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.assertResponseCode(OperationSupport.java:671) ~[kubernetes-client-6.3.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.lambda$handleResponse$0(OperationSupport.java:601) ~[kubernetes-client-6.3.1.jar:?]
[...]

Fabric8 Kubernetes Client version

6.3.1

Steps to reproduce

1. Call BaseOperation.list()
2. Let the server return an unauthorized response for your request with the corresponding status code triggering the client to call OperationSupport.createStatus()
3. Logs show a leaked OkHttp connection shortly afterwards

Expected behavior

No connection leak warning from OkHttp

Runtime

other (please specify in additional context)

Kubernetes API Server version

other (please specify in additional context)

Environment

Amazon

Fabric8 Kubernetes Client Logs

2022-12-31 22:08:19.788 WARN  [okhttp3.OkHttpClient] A connection to https://172.20.0.1/ was leaked. Did you forget to close a response body? To see where this was allocated, set the OkHttpClient logger level to FINE: Logger.getLogger(OkHttpClient.class.getName()).setLevel(Level.FINE);

2022-12-31 22:03:18.952 [...] WARN io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://172.20.0.1:443/api/v1/nodes?labelSelector=test%2Ftier%3Dapp. Message: Unauthorized! Configured service account doesn't have access. Service account may have been revoked. Unauthorized.
	at io.fabric8.kubernetes.client.KubernetesClientException.copyAsCause(KubernetesClientException.java:238) ~[kubernetes-client-api-6.3.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.waitForResult(OperationSupport.java:536) ~[kubernetes-client-6.3.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.list(BaseOperation.java:418) ~[kubernetes-client-6.3.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.list(BaseOperation.java:383) ~[kubernetes-client-6.3.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.BaseOperation.list(BaseOperation.java:93) ~[kubernetes-client-6.3.1.jar:?]
[...]
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://172.20.0.1:443/api/v1/nodes?labelSelector=test%2Ftier%3Dapp. Message: Unauthorized! Configured service account doesn't have access. Service account may have been revoked. Unauthorized.
at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.requestFailure(OperationSupport.java:742) ~[kubernetes-client-6.3.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.requestFailure(OperationSupport.java:722) ~[kubernetes-client-6.3.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.assertResponseCode(OperationSupport.java:671) ~[kubernetes-client-6.3.1.jar:?]
	at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.lambda$handleResponse$0(OperationSupport.java:601) ~[kubernetes-client-6.3.1.jar:?]
[...]

Additional context

Unfortunately I don't know the kubernetes flavor or API server version, but it is anyways not relevant for this issue.