Some youtube songs download speed varies often -- takes too long #30065

gitreset-hard · 2021-10-06T22:05:02Z

Checklist

I'm reporting a broken site support issue
I've verified that I'm running youtube-dl version 2021.06.06
I've checked that all provided URLs are alive and playable in a browser
I've checked that all URLs and arguments with special characters are properly quoted or escaped
I've searched the bugtracker for similar bug reports including closed ones
I've read bugs section in FAQ

Verbose log

10.11.101.20 - - [06/Oct/2021 21:43:01] "GET /files/load_by?weblink=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DrGN3nTC9Stg&folder=live_analysis%2Fmusimatch%2Fweblink%2F531d02e5-554f-44c7-8863-378993cdb659&bucket=msm-s3-redacted-eu-west-1-prd&start=0&stop=1500 HTTP/1.1" 200 -
[youtube] rGN3nTC9Stg: Downloading webpage
Download options: {'format': '140', 'outtmpl': '1633556601845.m4a', 'noplaylist': True, 'noprogress': True, 'cachedir': False}
[youtube] Downloading just video rGN3nTC9Stg because of --no-playlist
[youtube] rGN3nTC9Stg: Downloading player d33d444d
[download] Destination: 1633556601845.m4a
[download] Download completed
[ffmpeg] Correcting container in "1633556601845.m4a"
ffmpeg version 4.4 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10.3.1 (Alpine 10.3.1_git20210424) 20210424
  configuration: --prefix=/usr --enable-avresample --enable-avfilter --enable-gnutls --enable-gpl --enable-libass --enable-libmp3lame --enable-libvorbis --enable-libvpx --enable-libxvid --enable-libx264 --enable-libx265 --enable-libtheora --enable-libv4l2 --enable-libdav1d --enable-postproc --enable-pic --enable-pthreads --enable-shared --enable-libxcb --enable-libsrt --enable-libssh --enable-libvidstab --disable-stripping --disable-static --disable-librtmp --enable-vaapi --enable-vdpau --enable-libopus --enable-vulkan --enable-libsoxr --enable-libwebp --enable-libaom --disable-debug
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '1633556601845.m4a':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf58.76.100
  Duration: 00:04:33.49, start: 0.000000, bitrate: 127 kb/s
  Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
Output #0, ipod, to 'cut_1633556601845.m4a':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf58.76.100
  Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
size=       0kB time=00:00:00.00 bitrate=N/A speed=N/A    
size=    4253kB time=00:04:33.46 bitrate= 127.4kbits/s speed=1.06e+04x    
video:0kB audio:4193kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.418841% ```



---------------

## Description


We are lately running into issues with downloading youtube songs that have previously worked fine

Tracks are taking too long to download at random. Sometimes it succeeds, but can take a minute randomly.
This is the log of where the hangup is: (full output below snippet)

size=    4253kB time=00:04:33.46 bitrate= 127.4kbits/s **speed=1.06e+04x**    

does something look off in the logs?

FWIW we've been doing it like this for a long time and it worked fine. Updated the version as well today and this is happening a lot at random.

The text was updated successfully, but these errors were encountered:

dirkf · 2021-10-06T22:26:10Z

Duplicate of #29326.

gitreset-hard · 2021-10-07T03:32:57Z

Duplicate of #29326.

@dirkf ah thanks for that. A possible solution to manipulate a var is mentioned but nothing concrete to update the library.

gitreset-hard · 2021-10-07T03:34:40Z

I have an alternative tool, can download that file in less than a second:
PS C:\> Measure-Command { youtube -v -1 -a 140 rGN3nTC9Stg }
TotalSeconds      : 0.8858769
https://github.com/89z/mech

how does your downloader get past the issue of random throttling?

gitreset-hard · 2021-10-07T03:44:53Z

how does your downloader get past the issue of random throttling?

Download in chunks:

https://github.com/89z/mech/blob/v1.33.6/youtube/format.go#L48-L49

Not sure why so many people keep having this issue, Ive had the same code for 4 month, no trouble with slow speed at all.

I believe we've been using it for years without issue but suddenly it's been breaking a lot. So to clarify, you are chunking the video/timestamp into 10million (couldn't find what the underscores mean in 10_000_000) and then downloading them ? I imagine it still downloads as 1 file?

gitreset-hard · 2021-10-07T03:46:45Z

how does your downloader get past the issue of random throttling?

Download in chunks:

https://github.com/89z/mech/blob/v1.33.6/youtube/format.go#L48-L49

Not sure why so many people keep having this issue, Ive had the same code for 4 month, no trouble with slow speed at all.

here someone said chunking didn't mitigate the issue in this case. Maybe it is an issue for people who frequently download vs occasional

#29326 (comment)

gitreset-hard · 2021-10-07T03:54:34Z

@89z Thanks, I will bring it up. It seems like we are getting throttled

here's a screenshot from the docker container showing the speed https://i.imgur.com/UylAO01.png

as for chunking, it did nothing:

/app/Download # youtube-dl -f 140 http://www.youtube.com/watch?v=4-BtZWGaPNY --http-chunk-size 10M
[youtube] 4-BtZWGaPNY: Downloading webpage
[download] Resuming download at byte 169985
[download] Destination: NAIMA - Let Me See You [NCS Release]-4-BtZWGaPNY.m4a
[download]  36.1% of 2.47MiB at 48.44KiB/s ETA 00:33

gitreset-hard · 2021-10-07T04:02:06Z

thanks for that. Good to know. I may try to put your code into our network and run it to see if that works

…

On Wed, Oct 6, 2021 at 9:59 PM Steven Penny ***@***.***> wrote: Again, works fine here. Same video, same format: PS C:\> Measure-Command { youtube -v -1 -a 140 4-BtZWGaPNY } TotalSeconds : 0.7471733 So unless you can provide a reproducible test case, I cannot help you. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#30065 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ARCL2E32LQPZMWK26EX5DWDUFULJ5ANCNFSM5FP2U6XA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

crumhirnd · 2021-10-08T00:26:12Z

https://www.youtube.com/watch?v=ry1Vdz6KO-o

84436 · 2021-10-08T03:41:54Z

I'm on 2021.06.06 and I also experience throttling. My network is capable of 4K@60 playback perfectly.
Here's the 4 videos I've tested:
https://youtu.be/o-AeEM1Nk0c
https://youtu.be/oT3mCybbhf0
https://youtu.be/kJQP7kiw5Fk
https://youtu.be/LXb3EKWsInQ

FYI yt-video is an alias for

youtube-dl -o "D:/Downloads/YouTube/%%(uploader)s/%%(title)s.%%(ext)s" -f bestvideo+bestaudio --write-sub --sub-lang en-US,en-GB,en --embed-subs

gitreset-hard · 2021-10-08T03:56:17Z

@84436 @crumhirnd yeah this has been a common issue it seems. I’m seeing threads a few years old as well

Today I tried chunking, running locally (outside the AWS EC2 instance it runs normally) and throttled all the time.

I’m trying to get @89z script as a temporary workaround but getting stuck in it

fastily · 2021-10-08T08:17:26Z

youtube-dl isn't being actively maintained. This is a fixed issue in the spiritual successor to youtube-dl, yt-dlp

84436 · 2021-10-08T08:29:47Z

@fastily thanks for the suggestion; it works perfectly as a drop-in replacement.

Speaking of being inactive, I've just noticed right now there are 800+ PRs pending on this repo...

rautamiekka · 2021-10-11T09:26:39Z

works perfectly as a drop-in replacement.

Near drop-in replacement but once you've adjusted your commands/config it's indeed a drop-in replacement.

dirkf · 2022-01-31T00:37:37Z

Closed in af9e725.

dirkf added the duplicate label Jan 29, 2022

dirkf linked a pull request Jan 29, 2022 that will close this issue

[YouTube] Unthrottle downloads by responding to the "n" parameter challenge #30184

Closed

11 tasks

dirkf closed this as completed Jan 31, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Some youtube songs download speed varies often -- takes too long #30065

Some youtube songs download speed varies often -- takes too long #30065

gitreset-hard commented Oct 6, 2021 •

edited

Loading

dirkf commented Oct 6, 2021

gitreset-hard commented Oct 7, 2021

gitreset-hard commented Oct 7, 2021

gitreset-hard commented Oct 7, 2021

gitreset-hard commented Oct 7, 2021

gitreset-hard commented Oct 7, 2021 •

edited

Loading

gitreset-hard commented Oct 7, 2021 via email

crumhirnd commented Oct 8, 2021

84436 commented Oct 8, 2021

gitreset-hard commented Oct 8, 2021 •

edited

Loading

fastily commented Oct 8, 2021

84436 commented Oct 8, 2021

rautamiekka commented Oct 11, 2021

dirkf commented Jan 31, 2022

Some youtube songs download speed varies often -- takes too long #30065

Some youtube songs download speed varies often -- takes too long #30065

Comments

gitreset-hard commented Oct 6, 2021 • edited Loading

Checklist

Verbose log

dirkf commented Oct 6, 2021

gitreset-hard commented Oct 7, 2021

gitreset-hard commented Oct 7, 2021

gitreset-hard commented Oct 7, 2021

gitreset-hard commented Oct 7, 2021

gitreset-hard commented Oct 7, 2021 • edited Loading

gitreset-hard commented Oct 7, 2021 via email

crumhirnd commented Oct 8, 2021

84436 commented Oct 8, 2021

gitreset-hard commented Oct 8, 2021 • edited Loading

fastily commented Oct 8, 2021

84436 commented Oct 8, 2021

rautamiekka commented Oct 11, 2021

dirkf commented Jan 31, 2022

gitreset-hard commented Oct 6, 2021 •

edited

Loading

gitreset-hard commented Oct 7, 2021 •

edited

Loading

gitreset-hard commented Oct 8, 2021 •

edited

Loading