[storage-blob][APPENDBLOCK] Memory Leak

[sdnetwork]

• Package Name: @azure/storage-blob
• Package Version: 12.0.0-preview.2
• Operating system: ALL
• nodejs
  • version: 10.15.3

Describe the bug

when you append a block to appendblob, the memory increase at each append to finally reach the limit of process

To Reproduce
Steps to reproduce the behavior:

1. just append a block to an appendblob

[himadrinath]

also seeing this with uploadFileToBlockBlob with blockBlobURL memory increase when uploading and still holding their after upload finished

[jeremymeng]

@sdnetwork @himadrinath Thanks for the feedback! We will look into it.

[jeremymeng]

@himadrinath Are you seeing the similar behavior with the released v10 version of the library?

[himadrinath]

@jeremymeng after update to 10.5.0 looks like solved the problem

[jeremymeng]

Thanks @himadrinath for verifying it. So the issue only happens when using preview version. We are looking into it.

[jeremymeng]

It looks like we always create new agents in nodeFetchClient.prepareRequest() when there's no proxy settings.

azure-sdk-for-js/sdk/core/core-http/lib/nodeFetchHttpClient.ts

Line 59 in 8767c40

const agent = httpRequest.url.startsWith("https") ? new https.Agent(options) : new http.Agent(options);

This returned requestInit is used to combine with the original request:

https://github.com/azure/azure-sdk-for-js/blob/master/sdk/core/core-http/lib/fetchHttpClient.ts#L112

Maybe we should check httpRequest.agent too before creating new agents.

[jeremymeng]

Maybe we should check httpRequest.agent too before creating new agents.

Never mind, it could be undefined then a global agent might be used.

[jeremymeng]

The root cause of this issue is #5398, which is exposed by a fix in WebResource.clone() to pass keepAlive setting to clones.

As we use clones of the original WebResource to make requests, before the fix KeepAlivePolicy would never wor since the setting is not passed. After the fix, KeepAlivePolicy would not work as expected either since we are creating a new agent for each request.

While we are working on a proper fix for #5398, I made a change in PR #5396 to not passing keepAlive setting to clones so we don't have memory leaks.

[ghost]

Thanks for working with Microsoft on GitHub! Tell us how you feel about your experience using the reactions on this comment.