-
Notifications
You must be signed in to change notification settings - Fork 417
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support delivery methods other than progressive HTTP #810
Commits on May 28, 2022
-
Add support of other delivery methods than progressive HTTP in Stream…
… classes Stream constructors are now private and streams can be constructed with new Builder classes per stream class. This change has been made to prevent creating and using several constructors in stream classes. Some default cases have been also added in these Builder classes, so not everything has to be set, depending of the service and the content.
Configuration menu - View commit details
-
Copy full SHA for 2f061b8 - Browse repository at this point
Copy the full SHA 2f061b8View commit details -
Remove fetching of the DASH manifest extracted when getting informati…
…on of a content with StreamInfo DashMpdParser is only working with YouTube streams, as it uses the ItagItem class. Also improve code and comments of StreamInfo (especially final use where possible).
Configuration menu - View commit details
-
Copy full SHA for ad993b9 - Browse repository at this point
Copy the full SHA ad993b9View commit details -
Move DashMpdParser to the YouTube package and fix extraction of streams
DashMpdParser is only working with YouTube streams, as it uses the ItagItem class. Also update creation of AudioStreams and VideoStreams objects.
Configuration menu - View commit details
-
Copy full SHA for 7c67d46 - Browse repository at this point
Copy the full SHA 7c67d46View commit details -
[YouTube] Return more values returned inside the ItagItems of the pla…
…yer response and deprecate use of public audio and video fields These fields can be now replaced by a getter and a setter. New fields have been added and will allow the creation of DASH manifests for OTF and ended livestreams. There are: - contentLength; - approxDurationMs; - targetDurationSec; - sampleRate; - audioChannels.
Configuration menu - View commit details
-
Copy full SHA for d5f3637 - Browse repository at this point
Copy the full SHA d5f3637View commit details -
Apply changes in all StreamExtractors except YouTube's one and fix ex…
…traction of PeerTube audio streams as video streams Some code in these classes has been also refactored/improved/optimized. Also fix the extraction of PeerTube audio streams as video streams, which are now returned as audio streams.
Configuration menu - View commit details
-
Copy full SHA for 881969f - Browse repository at this point
Copy the full SHA 881969fView commit details -
Add POST_LIVE_STREAM and POST_LIVE_AUDIO_STREAM stream types
This allows the extractor to determine if a content is an ended audio or video livestream.
Configuration menu - View commit details
-
Copy full SHA for 4330b5f - Browse repository at this point
Copy the full SHA 4330b5fView commit details -
Apply changes in YoutubeStreamExtractor
Extract post live DVR streams as post live streams instead of live streams. A new class has been in order to improve code: ItagInfo, which stores an itag, the content (URL) extracted and if its an URL or not. A functional interface has been added in order to abstract the stream building: StreamBuilderHelper. Also add the cver parameter added by the desktop web client on the corresponding streams (a new method has been added in YoutubeParsingHelper to check this and another for Android streams). Some code in these classes has been also refactored/improved/optimized.
Configuration menu - View commit details
-
Copy full SHA for a857684 - Browse repository at this point
Copy the full SHA a857684View commit details -
[YouTube] Add ability to generate manifests of progressive, OTF and p…
…ost live streams A new class has been added to do so: YoutubeDashManifestCreator. It relies on a new class: ManifestCreatorCache, to cache the content, which relies on a new pair class named Pair. Results are cached and there is a cache per delivery type, on which cache limit, clear factor, clearing and resetting can be applied to each cache and to all caches. Look at code changes for more details.
Configuration menu - View commit details
-
Copy full SHA for 7477ed0 - Browse repository at this point
Copy the full SHA 7477ed0View commit details -
Update DefaultStreamExtractorTest and SoundcloudStreamExtractorTest t…
…o support changes made in Stream classes
Configuration menu - View commit details
-
Copy full SHA for f6ec7f9 - Browse repository at this point
Copy the full SHA f6ec7f9View commit details -
Add tests for YoutubeDashManifestCreator and ManifestCreatorCache
The test added in YoutubeDashManifestCreator uses a video of the Creative Commons channel, licenced under the Creative Commons Attribution licence (reuse allowed). Also remove public keywords of tests in UtilsTest, as suggested by SonarLint, because they are not needed with Junit 5.
Configuration menu - View commit details
-
Copy full SHA for 6985167 - Browse repository at this point
Copy the full SHA 6985167View commit details -
Improve documentation and adress most of the requested changes
Also fix some issues in several places, in the code and the documentation.
Configuration menu - View commit details
-
Copy full SHA for aa4c10e - Browse repository at this point
Copy the full SHA aa4c10eView commit details -
[YouTube] Return a copy of the hardcoded ItagItem instead of returnin…
…g the reference to the hardcoded one in ItagItem.getItag To do so, a copy constructor has been added in the class. This fixes, for instance, an issue in NewPipe, in which the ItagItem values where not the ones corresponsing to a stream but to another, when generating DASH manifests.
Configuration menu - View commit details
-
Copy full SHA for f61e209 - Browse repository at this point
Copy the full SHA f61e209View commit details -
Fix Checkstyle issues, revert resolution string changes for YouTube v…
…ideo streams and don't return the rn parameter in DASH manifests This parameter is still used to get the initialization sequence of OTF and POST-live streams, but is not returned anymore in the manifests. It has been removed in order to avoid fingerprinting based on the number sent (e.g. when starting to play a stream close to the end and using 123 as the request number where it should be 1) and should be added dynamically by clients in their requests. The relevant test has been also updated. Checkstyle issues in YoutubeDashManifestCreator have been fixed, and the changes in the resolution string returned for video streams in YoutubeStreamExtractor have been reverted, as they create issues on NewPipe right now.
Configuration menu - View commit details
-
Copy full SHA for 2fb1a41 - Browse repository at this point
Copy the full SHA 2fb1a41View commit details -
Move ManifestCreatorCache tests to a separate class and remove overri…
…de of equals and hashCode methods in ManifestCreatorCache These methods don't need to be overriden, as they are not excepted to be used in collections. Also improve the toString method of this class, which contains also now clearFactor and maximumSize attributes and for each operations.
Configuration menu - View commit details
-
Copy full SHA for d64d7bb - Browse repository at this point
Copy the full SHA d64d7bbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 436ddde - Browse repository at this point
Copy the full SHA 436dddeView commit details -
[YouTube] Support the iOS client in YoutubeDashManifestCreator and de…
…crypt again the n parameter, if present, for all clients This commits reverts a new behavior introduced in this branch, which only applied the decryption if needed on streams from the WEB client. Also fix rebase issues and documentations style in YoutubeDashManifestCreator.
Configuration menu - View commit details
-
Copy full SHA for 07b045f - Browse repository at this point
Copy the full SHA 07b045fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 50272db - Browse repository at this point
Copy the full SHA 50272dbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 159d05c - Browse repository at this point
Copy the full SHA 159d05cView commit details -
[YouTube] Secure DashManifestCreator against XEE attack
Also remove duplicate lines from javadoc comments, otherwise checkstyle complaints. XEE prevention is inspired from https://github.com/ChuckerTeam/chucker/pull/201/files For an explanation of XEE see https://rules.sonarsource.com/java/RSPEC-2755, though the solution reported there causes crashes on Android
Configuration menu - View commit details
-
Copy full SHA for ba68b8c - Browse repository at this point
Copy the full SHA ba68b8cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8226fd0 - Browse repository at this point
Copy the full SHA 8226fd0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5c83409 - Browse repository at this point
Copy the full SHA 5c83409View commit details -
[YouTube] Refactor YoutubeDashManifestCreator
- Remove all of the methods used to access caches and replace them with three caches getters - Rename caches to shorter and more meaningful names - Remove redundant @throws tags that just say "if this method fails to do what it should do", which is obvious
Configuration menu - View commit details
-
Copy full SHA for 3708ab9 - Browse repository at this point
Copy the full SHA 3708ab9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4da05af - Browse repository at this point
Copy the full SHA 4da05afView commit details -
Configuration menu - View commit details
-
Copy full SHA for 00bbe5e - Browse repository at this point
Copy the full SHA 00bbe5eView commit details -
[YouTube] Reduce exception generation code and move several attribute…
…s of MPD documents into constants Rename YoutubeDashManifestCreationException to CreationException. Also use these constants in YoutubeDashManifestCreatorTest.
Configuration menu - View commit details
-
Copy full SHA for cfc13f4 - Browse repository at this point
Copy the full SHA cfc13f4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2321822 - Browse repository at this point
Copy the full SHA 2321822View commit details -
Configuration menu - View commit details
-
Copy full SHA for 54d323c - Browse repository at this point
Copy the full SHA 54d323cView commit details -
[Bandcamp] Fix regression of Opus radio streams extraction
When moving opus-lo into a constant, opus-lo was renamed to opus_lo and was only used if no MP3 stream was available (which was not the case before the changes in BandcampRadioStreamExtractor related to the addition of the support of all delivery methods), so these changes removed the ability to get Opus streams of Bandcamp radios. This commit reverts this unwanted change.
Configuration menu - View commit details
-
Copy full SHA for 4158fc4 - Browse repository at this point
Copy the full SHA 4158fc4View commit details -
[YouTube] Return approxDurationMs value from YouTube's player respons…
…e in ItagItems generated This change allows to build DASH manifests using YoutubeDashManifestCreator with the real duration of streams and prevent potential cuts of the end of progressive streams, because the duration in YouTube's player response is in seconds and not milliseconds.
Configuration menu - View commit details
-
Copy full SHA for 2f3920c - Browse repository at this point
Copy the full SHA 2f3920cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 301b9fa - Browse repository at this point
Copy the full SHA 301b9faView commit details -
Configuration menu - View commit details
-
Copy full SHA for f17f7b9 - Browse repository at this point
Copy the full SHA f17f7b9View commit details -
[YouTube] Refactor DASH manifests creation
Move DASH manifests creation into a new subpackage of the YouTube package, dashmanifestcreators. This subpackage contains: - CreationException, exception extending Java's RuntimeException, thrown by manifest creators when something goes wrong; - YoutubeDashManifestCreatorsUtils, class which contains all common methods and constants of all or a part of the manifest creators; - a manifest creator has been added per delivery type of YouTube streams: - YoutubeProgressiveDashManifestCreator, for progressive streams; - YoutubeOtfDashManifestCreator, for OTF streams; - YoutubePostLiveStreamDvrDashManifestCreator, for post-live DVR streams (which use the live delivery method). Every DASH manifest creator has a getCache() static method, which returns the ManifestCreatorCache instance used to cache results. DeliveryType has been also extracted from the YouTube DASH manifest creators part of the extractor and moved to the YouTube package. YoutubeDashManifestCreatorTest has been updated and renamed to YoutubeDashManifestCreatorsTest, and YoutubeDashManifestCreator has been removed. Finally, several documentation and exception messages fixes and improvements have been made.
Configuration menu - View commit details
-
Copy full SHA for f7b1515 - Browse repository at this point
Copy the full SHA f7b1515View commit details -
[YouTube] Add missing Nonnull annotations in getCache method of YouTu…
…be DASH manifest creators
Configuration menu - View commit details
-
Copy full SHA for fffbbee - Browse repository at this point
Copy the full SHA fffbbeeView commit details -
Configuration menu - View commit details
-
Copy full SHA for c33d392 - Browse repository at this point
Copy the full SHA c33d392View commit details -
Configuration menu - View commit details
-
Copy full SHA for 044639c - Browse repository at this point
Copy the full SHA 044639cView commit details -
Configuration menu - View commit details
-
Copy full SHA for d652e05 - Browse repository at this point
Copy the full SHA d652e05View commit details -
Configuration menu - View commit details
-
Copy full SHA for b3c620f - Browse repository at this point
Copy the full SHA b3c620fView commit details
Commits on May 29, 2022
-
[SoundCloud] Use the HLS delivery method for all streams and extract …
…only a single stream URL from HLS manifest for MP3 streams SoundCloud broke the workaround used to get a single file from HLS manifests for Opus manifests, but it still works for MP3 ones. The code has been adapted to prevent an unneeded request (the one to the Opus HLS manifest) and the HLS delivery method is now used for SoundCloud MP3 and Opus streams, plus the progressive one (for tracks which have a progressive stream (MP3) and for the ones which doesn't have one, it is still used by trying to get a progressive stream, using the workaround). Streams extraction has been also moved to Java 8 Stream's API and the relevant test has been also updated.
Configuration menu - View commit details
-
Copy full SHA for 287d1df - Browse repository at this point
Copy the full SHA 287d1dfView commit details