Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[youporn] fix url metadata detection (width and bits) to allow best v… #20425

Closed
wants to merge 2 commits into from

Conversation

Kasperdilday
Copy link

@Kasperdilday Kasperdilday commented Mar 20, 2019

The slash in the end seems to have gone, metadata format has changed. This fixes best file selection.

[x] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

[x] Bug fix

Please see more details below.

@dstftw
Copy link
Collaborator

dstftw commented Mar 20, 2019

Provide concrete examples of what it's supposed to fix.

@Kasperdilday
Copy link
Author

Oh, yes, sorry. I see the problem. Last second I switched to the example video from the file to prepare the URL example and it noticed just now that it is different, as in working with the "old" code.

There are a few examples where the videoUrl has a different format and adapted the regex so that it works for both versions.

/videos\/201703\/11\/109285532\/720P_4000K_109285532.mp4?rate=248k&burst=1400k&validfrom=1553107800&validto=1553122200&hash=NzBS4CUWB2RpgA9thDRS0Ouw5PM%3D

Interesting is the .mp4 where there is a / in other URLs.

I added the example to the list in the code.

Speaking of output difference:

without fix:
youtube-dl --list-formats https://www.youporn.com/watch/13922959/femdom-principal/  
[YouPorn] femdom-principal: Downloading webpage
[info] Available formats for 13922959:
format code  extension  resolution note
0            mp4        unknown    
1            mp4        unknown    
2            mp4        unknown    (best)

with fix:
youtube-dl --list-formats https://www.youporn.com/watch/13922959/femdom-principal/
[YouPorn] femdom-principal: Downloading webpage
[info] Available formats for 13922959:
format code  extension  resolution note
240p-400k    mp4        240p        400k 
480p-2000k   mp4        480p       2000k 
720p-4000k   mp4        720p       4000k  (best)

regression test:

youtube-dl --list-formats https://www.youporn.com/watch/505835/sex-ed-is-it-safe-to-masturbate-daily/
[YouPorn] sex-ed-is-it-safe-to-masturbate-daily: Downloading webpage
[info] Available formats for 505835:
format code   extension  resolution note
180p-150k     3gp        180p        150k 
240p-240k-0   mp4        240p        240k 
240p-240k-1   mp4        240p        240k 
480p-750k-0   mp4        480p        750k 
480p-750k-1   mp4        480p        750k 
720p-1500k-0  mp4        720p       1500k 
720p-1500k-1  mp4        720p       1500k  (best)

@dstftw
Copy link
Collaborator

dstftw commented Mar 20, 2019

Add a test.

youtube_dl/extractor/youporn.py Outdated Show resolved Hide resolved
@Kasperdilday
Copy link
Author

Kasperdilday commented Apr 4, 2019

Added a test and fixed the existing ones - anything more I can do to make this acceptable?

Sorry I deleted the PR template but I thought it was for reporting issues - I think I added the important parts manually.

@@ -145,9 +160,8 @@ def _real_extract(self, url):
r'(?s)<div[^>]+class=["\']submitByLink["\'][^>]*>(.+?)</div>',
webpage, 'uploader', fatal=False)
upload_date = unified_strdate(self._html_search_regex(
[r'Date\s+[Aa]dded:\s*<span>([^<]+)',
r'(?s)<div[^>]+class=["\']videoInfo(?:Date|Time)["\'][^>]*>(.+?)</div>'],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not remove old patterns.

comment_count = str_to_int(self._search_regex(
r'>All [Cc]omments? \(([\d,.]+)\)',
webpage, 'comment count', fatal=False))
comment_count = len(re.findall(r'<div[^>]+class=([\"\']).*?videoComment\b.*?\1', webpage))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants