Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[YouTube] Check whether player responses are valid for all InnerTube clients used #1070

Merged
merged 4 commits into from
Jun 18, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -868,6 +868,10 @@ public void onFetchPage(@Nonnull final Downloader downloader)
playerMicroFormatRenderer = youtubePlayerResponse.getObject("microformat")
.getObject("playerMicroformatRenderer");

if (isPlayerResponseNotValid(playerResponse, videoId)) {
throw new ExtractionException("Initial player response is not valid");
}

final byte[] body = JsonWriter.string(
prepareDesktopJsonBuilder(localization, contentCountry)
.value(VIDEO_ID, videoId)
Expand Down Expand Up @@ -1058,6 +1062,11 @@ private void fetchTvHtml5EmbedJsonPlayer(@Nonnull final ContentCountry contentCo
final JsonObject tvHtml5EmbedPlayerResponse = getJsonPostResponse(PLAYER,
createDesktopPlayerBody(localization, contentCountry, videoId, sts, true,
html5Cpn), localization);

if (isPlayerResponseNotValid(tvHtml5EmbedPlayerResponse, videoId)) {
return;
}

final JsonObject streamingData = tvHtml5EmbedPlayerResponse.getObject(
STREAMING_DATA);
if (!isNullOrEmpty(streamingData)) {
Expand All @@ -1067,35 +1076,40 @@ private void fetchTvHtml5EmbedJsonPlayer(@Nonnull final ContentCountry contentCo
}

/**
* Checks whether an additional player response is not valid.
* Checks whether a player response is invalid.
*
* <p>
* If YouTube detect that requests come from a third party client, they may replace the real
* player response by another one of a video saying that this content is not available on this
* app and to watch it on the latest version of YouTube.
* app and to watch it on the latest version of YouTube. This behavior has been observed on the
* {@code ANDROID} client, see
* <a href="https://github.com/TeamNewPipe/NewPipe/issues/8713">
* https://github.com/TeamNewPipe/NewPipe/issues/8713</a>.
* </p>
*
* <p>
* We can detect this by checking whether the video ID of the player response returned is the
* same as the one requested by the extractor.
* YouTube may also sometimes for currently unknown reasons rate-limit an IP, and replace the
* real one by a player response with a video that says that the requested video is
* unavailable. This behaviour has been observed in Piped on the InnerTube clients used by the
* extractor ({@code ANDROID} and {@code WEB} clients) which should apply for all clients, see
* <a href="https://github.com/TeamPiped/Piped/issues/2487">
* https://github.com/TeamPiped/Piped/issues/2487</a>.
* </p>
*
* <p>
* This behavior has been already observed on the {@code ANDROID} client, see
* <a href="https://github.com/TeamNewPipe/NewPipe/issues/8713">
* https://github.com/TeamNewPipe/NewPipe/issues/8713</a>.
* We can detect this by checking whether the video ID of the player response returned is the
* same as the one requested by the extractor.
* </p>
*
* @param additionalPlayerResponse an additional response to the one of the {@code HTML5}
* client used
* @param videoId the video ID of the content requested
* @param playerResponse a player response from any client
* @param videoId the video ID of the content requested
* @return whether the video ID of the player response is not equal to the one requested
*/
private static boolean isPlayerResponseNotValid(
@Nonnull final JsonObject additionalPlayerResponse,
@Nonnull final JsonObject playerResponse,
@Nonnull final String videoId) {
return !videoId.equals(additionalPlayerResponse.getObject("videoDetails")
.getString("videoId", ""));
return !videoId.equals(playerResponse.getObject("videoDetails")
.getString("videoId"));
}

private static void storePlayerJs() throws ParsingException {
Expand Down