Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DPlay] unable to download free or premium episodes from dplay.no, es.dplay.com and it.dplay.com with user account #24554

Closed
5 tasks done
havarh opened this issue Mar 31, 2020 · 21 comments

Comments

@havarh
Copy link

havarh commented Mar 31, 2020

Checklist

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dl version 2020.03.24
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--username', 'PRIVATE', '--password', 'PRIVATE', 'https://www.dplay.no/programmer/praktisk-info-med-jon-almaas/sesong-5-episode-9', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.03.24
[debug] Python version 3.6.8 (CPython) - Linux-4.15.0-88-generic-x86_64-with-Ubuntu-18.04-bionic
[debug] exe versions: ffmpeg 3.4.6, ffprobe 3.4.6, phantomjs ., rtmpdump 2.4
[debug] Proxy map: {}
[debug] Using fake IP 84.213.190.196 (NO) as X-Forwarded-For.
[DPlay] praktisk-info-med-jon-almaas/sesong-5-episode-9: Downloading token
[DPlay] praktisk-info-med-jon-almaas/sesong-5-episode-9: Downloading JSON metadata
[DPlay] praktisk-info-med-jon-almaas/sesong-5-episode-9: Downloading JSON metadata
ERROR: This video is only available for registered users. Use --username and --password or --netrc to provide account credentials.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 627, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2238, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/dplay.py", line 163, in _get_disco_api_info
    display_id, headers=headers)['data']['attributes']['streaming']
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 892, in _download_json
    expected_status=expected_status)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 870, in _download_json_handle
    expected_status=expected_status)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 660, in _download_webpage_handle
    urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 645, in _request_webpage
    raise ExtractorError(errmsg, sys.exc_info()[2], cause=err)
youtube_dl.utils.ExtractorError: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 627, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2238, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/dplay.py", line 163, in _get_disco_api_info
    display_id, headers=headers)['data']['attributes']['streaming']
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 892, in _download_json
    expected_status=expected_status)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 870, in _download_json_handle
    expected_status=expected_status)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 660, in _download_webpage_handle
    urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 645, in _request_webpage
    raise ExtractorError(errmsg, sys.exc_info()[2], cause=err)
youtube_dl.utils.ExtractorError: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 797, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/dplay.py", line 247, in _real_extract
    url, display_id, host, 'dplay' + country, country)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/dplay.py", line 172, in _get_disco_api_info
    self.raise_login_required()
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 936, in raise_login_required
    expected=True)
youtube_dl.utils.ExtractorError: This video is only available for registered users. Use --username and --password or --netrc to provide account credentials.

Description

I'm unable to download free or premium content from dplay.no with a user account. I've been able to download free content earlier, but now a (free) user account is required. I've made a test account with a disposable e-mail address, and used this commandline to test the download:

youtube-dl --verbose --username bupekatich@wemel.top --password Barrowman42 https://www.dplay.no/programmer/praktisk-info-med-jon-almaas/sesong-5-episode-9

But I get this errormessage:

ERROR: This video is only available for registered users. Use --username and --password or --netrc to provide account credentials.

(I've verified that the account works in a browser)

@havarh
Copy link
Author

havarh commented Mar 31, 2020

I've also tried with this URL format:
https://www.dplay.no/videoer/praktisk-info-med-jon-almaas/sesong-5-episode-9
And also with the test url from dplay.py:
https://www.dplay.no/videoer/i-kongens-klr/sesong-1-episode-7
But I get the same error message

@havarh
Copy link
Author

havarh commented Mar 31, 2020

The problem seems to be with reCAPTCHA. dplay.no has implemented it on the login page. If I use the --cookie attribute with cookies catched with this Chrome extension it works
https://chrome.google.com/webstore/detail/cookiestxt/njabckikapfpffapmjgojcnbfjonfjfg/related
or this Firefox extension
https://addons.mozilla.org/nb-NO/firefox/addon/cookies-txt-one-click/

@felisucoibi
Copy link

Same problem with es.dplay.com and it.dplay.com, maybe you can edit title to add this urls too so can be fixed all of them at the same time.

@havarh havarh changed the title [DPlay] unable to download free or premium episodes from dplay.no with user account [DPlay] unable to download free or premium episodes from dplay.no, es.dplay.com and it.dplay.com with user account Mar 31, 2020
@havarh
Copy link
Author

havarh commented Mar 31, 2020

Same problem with es.dplay.com and it.dplay.com, maybe you can edit title to add this urls too so can be fixed all of them at the same time.

OK, done. But I doubt this could be fixed that easily. reCAPTCHA is hard to script around. Better just use my --cookie approach

@felisucoibi
Copy link

felisucoibi commented Mar 31, 2020

I'm trying the cookie approach but i cant login , which cookie is the one needed? there are a few of them.
First problem i had is the cookie.txt file needs this header:

# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This is a generated file!  Do not edit.

Bests.

@havarh
Copy link
Author

havarh commented Mar 31, 2020

I just used every cookie that those extensions made. If you use the Chrome extension click on:

To download cookies for this tab click here

@felisucoibi
Copy link

what is your exact command you also use --username --password appart from --cookie ?

@havarh
Copy link
Author

havarh commented Mar 31, 2020

No, I didn't use --username or --password. This is my exact command line:
youtube-dl --cookie dplay.no-cookies.txt https://www.dplay.no/videoer/mythbusters/dropping-a-bomb

@felisucoibi
Copy link

Maybe is different dplay.no from es.dplay.com and it.dplay.com it keeps asking me the username and password.

@dstftw dstftw closed this as completed Apr 3, 2020
@felisucoibi
Copy link

I think you marked all the duplicates as duplicate and nwo there is no original bug one reporting the bug, isn't it? I dont see any opened bug with this bug but 5 of them closed as duplicate.

@Arnvidr
Copy link

Arnvidr commented Apr 7, 2020

I think the same problem is tracked in open issue #22972

@felisucoibi
Copy link

I think the same problem is tracked in open issue #22972

that bug is from 2019, differenet output error, the patform changed 3 times since 2019.

@Arnvidr
Copy link

Arnvidr commented Apr 7, 2020

What difference do you see? The errors seem to me to be the same.

@felisucoibi
Copy link

What difference do you see? The errors seem to me to be the same.
the output is different, if 2019 bug would be fixed it would not fix the 2020-04 bug, even the output is different. Now you need to be registered to access any video, in 2019, there was a different api to do it. Anyway, just fix it, and if somebody marks a bug as duplicate, please refer to it.

@Arnvidr
Copy link

Arnvidr commented Apr 8, 2020

Now you need to be registered to access any video, in 2019, there was a different api to do it. Anyway, just fix it, and if somebody marks a bug as duplicate, please refer to it.

Definitely agree with this, and definitely the reason the older bug appeared must have been different, but I still don't see the difference in the error output, outside of probably being ran on different systems (Windows vs something else). The stack traces seem to touch on the same code paths, giving the same error messages?

Definitely trying to fix the older issue would make the developer hit this issue either way, so I think fixing the other bug would also fix this issue. Hopefully a developer pitches in with any info.

@havarh
Copy link
Author

havarh commented Apr 8, 2020

This particular bug is probably very difficult to fix without adding a GUI element that could be used to solving the captcha on the login page. That's also how this differs from #22972 in that Dplay didn't use captcha back then. Also the problem I had was easily bypassed with a browser cookie capture, while a cookie file didn't fix the problems in #22972.

@Arnvidr
Copy link

Arnvidr commented Apr 9, 2020

I guess I'll try to get a cookie from Chrome then, because using a cookie from the equivalent Firefox extension changed nothing for me, giving me the same error.

I would also still like someone to point out if they actually see any difference in the errors between the issues, as everyone just keeps mentioning what the pages themselves have changed since the first issue was opened. They still seem the same to me.

@felisucoibi
Copy link

I guess I'll try to get a cookie from Chrome then, because using a cookie from the equivalent Firefox extension changed nothing for me, giving me the same error.

I would also still like someone to point out if they actually see any difference in the errors between the issues, as everyone just keeps mentioning what the pages themselves have changed since the first issue was opened. They still seem the same to me.

me too wit firefox cookie is not working but i did not try with chrome cookie, did it work?

@RobertusIT
Copy link

I guess I'll try to get a cookie from Chrome then, because using a cookie from the equivalent Firefox extension changed nothing for me, giving me the same error.
I would also still like someone to point out if they actually see any difference in the errors between the issues, as everyone just keeps mentioning what the pages themselves have changed since the first issue was opened. They still seem the same to me.

me too wit firefox cookie is not working but i did not try with chrome cookie, did it work?

Hi

have you tried ? with chrome cookie, no luck

@Arnvidr
Copy link

Arnvidr commented Apr 11, 2020

Finally had time to try it, and the chrome cookie worked for me.

@RobertusIT
Copy link

Finally had time to try it, and the chrome cookie worked for me.

Hi, please i report my issue here

but i use dplay.it, ahve you used the same website?

If yes, can you help me ?

I have tried a lot of extension to export cookie from chrome, but anyway doesn't works, seems a youtube-dl trouble.

@ytdl-org ytdl-org locked and limited conversation to collaborators Apr 11, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

5 participants