Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perf: use io copy with buffer #6389

Merged
merged 2 commits into from
Apr 25, 2024
Merged

Perf: use io copy with buffer #6389

merged 2 commits into from
Apr 25, 2024

Conversation

Mmx233
Copy link
Contributor

@Mmx233 Mmx233 commented Apr 24, 2024

Using io.CopyBuffer can reduce the resource consumption caused by multiple memory allocations and appropriately increase the buffer size to optimize the performance of Copy.

This may mitigate the issue described in #6325

@Mmx233 Mmx233 marked this pull request as draft April 24, 2024 14:53
@pull-request-size pull-request-size bot added size/M and removed size/S labels Apr 24, 2024
@Mmx233
Copy link
Contributor Author

Mmx233 commented Apr 24, 2024

Test image available at mmx233/alist:v3.33.0-rc2, I will test it at tomorrow morning for my test environment is down now.

@Mmx233 Mmx233 marked this pull request as ready for review April 24, 2024 17:56
@Mmx233
Copy link
Contributor Author

Mmx233 commented Apr 25, 2024

My test environment is keeping unavailable. I tested it on my local computer. Using a 20GB single file, Windows amd64 SSD, with signature enabled, HTTP service, Chrome single-threaded download, and Free Download Manager 15-threaded download.

Only the occupation of alist is being measured. Iowait is an important performance metric but cannot be calculated in the current environment. I only ran it once and manually retrieved the data from the task manager, and there may be many other unstable factors in between. So the resulting outcome should be considered for reference only.

original io copy

Threads CPU Usage Memorary Usage Speed
1 10% 19.2Mb 400Mb/s
15 12% 20.9Mb 500Mb/s

io copy buffer

Threads CPU Usage Memorary Usage Speed
1 4.5% 16.5Mb 400Mb/s
15 8% 19Mb 485Mb/s

@xhofe xhofe merged commit b95df1d into AlistGo:main Apr 25, 2024
3 checks passed
@Mmx233 Mmx233 deleted the improve/io-perf branch May 7, 2024 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants