Support audio only videos with HLS player

[Chocobozzz]

I did not manage to make our hls.js player work with a mp4 container that does not contain a video stream.

[kontrollanten]

hls.js is filtering out levels without video codec; https://github.com/video-dev/hls.js/blob/b3ed0028e0e7d6e98f5b9d611487540a60e2fc6b/src/controller/level-controller.js#L111

As I understand hls.js it's looking in the master playlist for an explicit audio track, which PeerTube doesn't have. So the solution may be to add "EXT-X-MEDIA:TYPE=AUDIO" to the audio track? https://github.com/video-dev/hls.js/blob/b3ed0028e0e7d6e98f5b9d611487540a60e2fc6b/src/loader/playlist-loader.ts#L298

Current playlist structure:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:BANDWIDTH=126899,RESOLUTION=0x0,CODECS=",mp4a.40.2"
0.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=284418,RESOLUTION=426x240,FRAME-RATE=25,CODECS="avc1.64001f,mp4a.40.2"
240.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=426712,RESOLUTION=640x360,FRAME-RATE=25,CODECS="avc1.64001f,mp4a.40.2"
360.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=572868,RESOLUTION=852x480,FRAME-RATE=25,CODECS="avc1.64001f,mp4a.40.2"
480.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=990164,RESOLUTION=1280x720,FRAME-RATE=25,CODECS="avc1.64001f,mp4a.40.2"
720.m3u8

Update:

It seems that this isn't supported by hls.js. video-dev/hls.js#2654 (comment) How is this handled with the WebTorrent player? Can it downgrade from video to audio because of bandwidth?

[pprw]

Hello.

Does this means we should currently disable "audio only" transcoding in peertube 2.4?

[kontrollanten]

As far as I've experienced the audio only file is created but not provided to the user in the GUI, so it won't do any harm more than eat some disk space. But this only applies to HLS, if you're using WebTorrents it should work.

[roere]

I have a similar issue. But: for some videos, the audio-only option IS available in the player, for some videos not. I confirmed, that in all cases, the pure audio file was created, is in the folder and is available for download.
I couldn't find a pattern yet, in which cases the audio-only option is available.
I am running PT 3.0.1 on Ubuntu 20.04.1, with node 12.13.1 and FFMPEG 4.2.4 (htttps://widemus.de).

[maxmellen]

@Chocobozzz since the 0 resolution file is still being generated for the audio-only output format, would you be open to considering using a different player for the 0 resolution file? We'd be willing to help out integrate this different player as it would be really useful to us 😊
If not can we at least work off the assumption that the 0 resolution file will continue to be generated in the future when using HLS?

Also, is it already possible to access the 0 resolution files directly through the Peertube REST API?

[Chocobozzz]

Hi,

would you be open to considering using a different player for the 0 resolution file?

No sorry it's too complicated.

If not can we at least work off the assumption that the 0 resolution file will continue to be generated in the future when using HLS?

It still can be read/downloaded so even if our player is not able to read it, it could still be useful to generate the 0 resolution.

Also, is it already possible to access the 0 resolution files directly through the Peertube REST API?

Yes, or using the download modal.

[maxmellen]

Got it! Thanks a lot for taking the time to answer 👏

[Chocobozzz]

We plan to implement this feature in 2024. Read the detailed blog post on https://joinpeertube.org/news/peertube-future-2024

[Chocobozzz]

Implemented in #6521

[RokeJulianLockhart]

#3281 (comment)

Does this theoretically mean that a PeerTube instance could now be used as a music distribution site, and use less bandwidth than if it were streaming video?

[Chocobozzz]

Does this theoretically mean that a PeerTube instance could now be used as a music distribution site, and use less bandwidth than if it were streaming video?

Yes for lives (in the next minor/major version). For VOD we still need an additional configuration so admins can choose between generating a video or just serve the audio when the user uploads an audio file only (don't hesitate to create an issue for that use case).

[RokeJulianLockhart]

#3281 (comment)

@Chocobozzz, I don't know enough about PeerTube to create an issue for that use case, but I'll certainly support it if I see an issue for it appear. Thanks for the explanation.