Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Torrents slow to a crawl after only a few minutes of fast downloading #13314

Closed
marcmy opened this issue Sep 2, 2020 · 21 comments
Closed

Torrents slow to a crawl after only a few minutes of fast downloading #13314

marcmy opened this issue Sep 2, 2020 · 21 comments
Labels
Not an issue User error, problem unrelated to qBittorrent, feature already implemented, etc Performance

Comments

@marcmy
Copy link

marcmy commented Sep 2, 2020

Please provide the following information

qBittorrent version and Operating System

4.2.5 / Windows 10

What is the problem

After only a few minutes of downloading (normally) on my 1Gbps Fiber connection, all torrents basically come to a stop, downloading at a mere KB/s or B/s, with some spikes here and there. I have tried using 3 different drives, experimented with different cache settings, along with the other advanced options. Nothing seems to work here. I've tried everything at this point. I replaced one of the drives due to poor benchmark speed, and the new drive is having the same issue. I'm not even downloading that fast, maybe 50-60MB/s, I don't know what could cause the drive to not be able to keep up.

What is the expected behavior

I expect qBittorrent to continue downloading normally and slow down if it absolutely needs to. However, it ends up slowing/stopping the downloads for HOURS, until I restart the program. This can't be normal.

Extra info(if any)

I have 32gb ram and even experimented with using ALL of it as cache, which let me download for longer, but not by much. Not sure where to go from here.

@marcmy marcmy closed this as completed Sep 2, 2020
@marcmy marcmy reopened this Sep 2, 2020
@FranciscoPombal
Copy link
Member

FranciscoPombal commented Sep 2, 2020

@marcmy Can't reproduce. For best performance, you should set Async I/O threads to 8 if you're downloading to HDDs, or 4 times the number of CPU cores if using an SSD. Disk cache can be set to auto. Everything else as default settings is fine.

@FranciscoPombal FranciscoPombal added Can't reproduce Issues that cannot be reproduced Performance labels Sep 2, 2020
@marcmy
Copy link
Author

marcmy commented Sep 2, 2020 via email

@ghost
Copy link

ghost commented Sep 3, 2020

What the disk activity % in task manager when you face this problem?
also in qbit go view>statistics and tell us what the cache overload % while the performance problem is happening.

@marcmy
Copy link
Author

marcmy commented Sep 3, 2020

What the disk activity % in task manager when you face this problem?
also in qbit go view>statistics and tell us what the cache overload % while the performance problem is happening.

OK will do, I'm moving a bunch of data atm so it might be a while before I can get any accurate readings.

@FranciscoPombal
Copy link
Member

@marcmy

Like I said before, set disk cache to auto. Though that typically overshoots with such high amounts of RAM, so if you want to save a little RAM, 256 MiB or even 128 MiB will be enough to not cause a performance bottleneck, while wasting little to no RAM as useless excessive cache.

Seagate 8tb 5400 rpm 256 mb cache

Oof, is that one of those Seagate drives marketed as "Archive drives"? What is the specific model number (Seagate model numbers are typically something like STXXXXXXXXXX)?

If it is one such archive drive, that is a problem, because it uses a technology called SMR (Shingled Magnetic Recording), which has really bad sustained write performance, especially if writes have a non-sequential pattern (as is typically the case with torrents) and also if the drive happens to write to regions near other exsiting files. The pattern you're observing where speed tanks to next to nothing after a while seems to be consistent with patterns observed in SMR drives in similar workflows. Also 5400rpm is slow, and you'll only ever be able to sustain respectable speeds on hard drives (10's/100's of MiB/s) as long as you're only downloading 1 or 2 torrents at the same time and with "Download pieces in sequential order".

@marcmy
Copy link
Author

marcmy commented Sep 4, 2020 via email

@c0re100
Copy link

c0re100 commented Sep 4, 2020

It's an ST8000DM004-2CX188. seagate barracuda. I don't think it's an archive drive, and surprisingly the drive performs better in benchmarks than my other two 7200 rpm drives. Maybe that's due to their age and wear and tear. I have not specifically checked for sustained writes or anything. I'll look into that after I'm done copying, as well as the stats.

On Thu, Sep 3, 2020, 12:52 PM Francisco Pombal @.***> wrote: @marcmy https://github.com/marcmy Like I said before, set disk cache to auto. Though that typically overshoots with such high amounts of RAM, so if you want to save a little RAM, 256 MiB or even 128 MiB will be enough to not cause a performance bottleneck, while wasting little to no RAM as useless excessive cache. Seagate 8tb 5400 rpm 256 mb cache Oof, is that one of those Seagate drives marketed as "Archive drives"? What is the specific model number (Seagate model numbers are typically something like STXXXXXXXXXX)? If it is one such archive drive, that is a problem, because it uses a technology called SMR (Shingled Magnetic Recording), which has really bad sustained write performance, especially if writes have a non-sequential pattern (as is typically the case with torrents) and also if the drive happens to write to regions near other exsiting files. The pattern you're observing where speed tanks to next to nothing after a while seems to be consistent with patterns observed in SMR drives in similar workflows. Also 5400rpm is slow, and you'll only ever be able to sustain respectable speeds on hard drives (10's/100's of MiB/s) as long as you're only downloading 1 or 2 torrents at the same time and with "Download pieces in sequential order". — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13314 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFAG73TADLHHMYQ55Y5XJWLSD7CTFANCNFSM4QTBLQYA .

{9B8441A9-20BA-40C1-B462-61EF836421AA}
It's SMR harddisk lol

@marcmy
Copy link
Author

marcmy commented Sep 4, 2020 via email

@Seeker2
Copy link

Seeker2 commented Sep 4, 2020

Something you can try is making a LARGE ramdrive (like 10+ GB size) and downloading first to there and having qBitTorrent move to your hard drive/s on completion of the torrents.
(I use IMDISK for this, which is free software.)

Obviously will only work if your downloads fit in the ramdrive, so some really large downloads will need to be to a non-SMR HDD.
On the bright side, downloading to a ramdrive means qBitTorrent can be set to use a much smaller cache size, like 32 MB.

@FranciscoPombal
Copy link
Member

@marcmy

Torrenting with SMR drives is generally a bad idea. It is one of the use cases in which they can perform really bad. SMR drives are designed for a "write sequentially infrequently, read many times" type of workflow - not the case for torrenting, NAS RAID arrays, etc. HDD manufacturers have been caught hiding the underlying technology used in their drives, ultimately resulting in lawsuits filed against them. Unfortunately, if you want a non-SMR drive, you'll have to pay for it. There are no cheap 8 TB non-SMR drives. Fortunately, due to the controversy, the big 3 manufacturers (Seagate, WD, Toshiba) have now published information detailing exactly which model numbers are SMR. Be sure to know what you're in for when buying hard drives.

Further reading:
https://www.servethehome.com/surreptitiously-swapping-smr-into-hard-drives-must-end/
https://arstechnica.com/information-technology/2020/06/lawsuit-vs-western-digital-wants-to-end-any-use-of-smr-in-nas-drives/
https://www.tomshardware.com/news/wd-lists-all-drives-slower-smr-techNOLOGY

Your experience will definitely improve with a non-SMR drive. That being said, as a disclaimer, at this point we can't be sure it will fix all your problems. Good luck!

@marcmy
Copy link
Author

marcmy commented Sep 4, 2020 via email

@Seeker2
Copy link

Seeker2 commented Sep 4, 2020

Seeding torrents off a SMR HDD should be fast enough though, so long as they're not significantly fragmented (files-wise).

@FranciscoPombal
Copy link
Member

@Seeker2 If you are only seeding it's fine (still probably not so good performance, especially with many simultaneously active torrents because it's a hard drive, after all), but OP specifically states they observe the performance problem when downloading.

@Seeker2
Copy link

Seeker2 commented Sep 5, 2020

Yes, I agree that downloading performance is awful on SMR HDDs.

Even getting files onto the SMR HDD for seeding in any manner can be problematic.
A bulk copy of 10 GB of files (1-file-at-a-time!) to a SMR HDD will probably choke something awful once the SMR HDD runs out of its limited non-SMR space.

@marcmy
Copy link
Author

marcmy commented Sep 12, 2020 via email

@FranciscoPombal
Copy link
Member

@marcmy Very strange. Please post logs and settings in plain text. You're using the 64-bit version, correct? You could also try one of these recent experimental builds (master commit 1c87073): https://github.com/FranciscoPombal/qBittorrent/actions/runs/241080799

@marcmy
Copy link
Author

marcmy commented Sep 15, 2020 via email

@FranciscoPombal
Copy link
Member

@marcmy Your configuration includes some potentially problematic choices, like Downloads\SaveResumeDataInterval=5. This is typically too frequent, and it may be impacting negatively your disk performance for all other tasks. I would suggest trying to reproduce this issue with all default settings (backup C:\Users\<username>\AppData\Roaming\qBittorrent and delete it from the original location to easily reset all settings to defaults) except for async I/O set to 8, disk cache to auto, and file pool size to 4000. If you still get the same results, try with anti-virus disabled.

@marcmy
Copy link
Author

marcmy commented Sep 16, 2020 via email

@marcmy marcmy closed this as completed Sep 16, 2020
@FranciscoPombal
Copy link
Member

@marcmy Thanks for the troubleshooting information. Hopefully this information can also help others having issues with external drives.

@FranciscoPombal FranciscoPombal added Not an issue User error, problem unrelated to qBittorrent, feature already implemented, etc and removed Can't reproduce Issues that cannot be reproduced labels Sep 16, 2020
@qbittorrent qbittorrent locked as resolved and limited conversation to collaborators Sep 16, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Not an issue User error, problem unrelated to qBittorrent, feature already implemented, etc Performance
Projects
None yet
Development

No branches or pull requests

4 participants