Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance of uploading file to block blob: use fs.stat() instead of fs.statSync() #5056

Closed
5 tasks
rabinovichor opened this issue Sep 8, 2019 · 4 comments
Assignees
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. enhancement good first issue This issue tracks work that may be a good starting point for a first-time contributor Storage Storage Service (Queues, Blobs, Files)

Comments

@rabinovichor
Copy link

  • Package Name:
    @azure/storage-blob
  • Package Version:
    10.4.1
  • Operating system:
    macOs
  • nodejs
    • version:
      v12.5.0
  • browser
    • name/version:
      Chrome 76 on macOS (Mojave)
  • typescript
    • version:
      3.2.2
  • Is the bug related to documentation in

Describe the bug
File Name: highlevel.node.ts
** Function :** uploadFileToBlockBlob : 39

Before uploading the file, you are using 'fs.statSync' in order to get the size of the file, probably to use it has Content-Length.
The use of a synchronous function is severe, certainly with the use of Promises.

Once you adopt Promises pattern for writing asynchronous code but still using dependency with synchronous functions, you can cause the main thread to be block. (for example using Promises.all).

To Reproduce
Steps to reproduce the behavior:
1.

Expected behavior
Use 'fs.stat' instead of 'fs.statSync' and use await within the function.
When you await a promise, the function is paused in a non-blocking way until the promise settles.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

@bterlson
Copy link
Member

bterlson commented Sep 8, 2019

Thanks for the report @orrabinovich88 (and welcome to GitHub!! 😄). Preferring async over sync for these built-ins makes sense.

Curious, did you notice some slowness or blocking of main thread in your application? If so can you quantify how much/how bad it was?

@rabinovichor
Copy link
Author

Hi @bterlson, and thanks for the quick reply!

Yes, once you work with a huge amount of files that are processed together, you will feel the difference with no doubt, and that's exactly what i'm doing at my job :).

Moreover Node actually do not recommend using this function anymore.
Thanks.

@kaerm kaerm added Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. Storage Storage Service (Queues, Blobs, Files) labels Sep 9, 2019
@triage-new-issues triage-new-issues bot removed the triage label Sep 9, 2019
@XiaoningLiu XiaoningLiu added enhancement good first issue This issue tracks work that may be a good starting point for a first-time contributor labels Sep 10, 2019
@XiaoningLiu
Copy link
Member

Definitly we should use async operations, thanks for heading up!
At the same time, welcome contribution!

@jeremymeng jeremymeng changed the title Improve performance of uploading file to block blob Improve performance of uploading file to block blob: use fs.stat() instead of fs.statSync() Sep 16, 2019
@ghost
Copy link

ghost commented Sep 20, 2019

Thanks for working with Microsoft on GitHub! Tell us how you feel about your experience using the reactions on this comment.

@github-actions github-actions bot locked and limited conversation to collaborators Apr 12, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. enhancement good first issue This issue tracks work that may be a good starting point for a first-time contributor Storage Storage Service (Queues, Blobs, Files)
Projects
None yet
Development

No branches or pull requests

6 participants