Add rkmpp support #160

tandy-1000 · 2022-06-22T19:15:56Z

TODO:

Add rkmpp for arm64 build
Enable the build flag

Changes

For arm64 platform the rkmpp dependency is added

Issues
Fixes #34

docker-build.sh

nyanmisaka · 2022-06-23T08:46:45Z

#161 has been merged, you can rebase to avoid the CI error.

Shadowghost · 2022-06-23T13:42:31Z

Can we rebase and squash before merging please :)

nyanmisaka

Apply these suggestions and squash this PR to one commit.

docker-build.sh

tandy-1000 · 2022-06-23T15:51:41Z

Squashed!

nyanmisaka · 2022-06-23T15:59:27Z

Once the CI is done, you need to verify that artifact works on a rockchip SBC.

tandy-1000 · 2022-06-23T17:32:56Z

I've installed the artifact, how do I test it?

nyanmisaka · 2022-06-23T17:42:03Z

https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/rkmppdec.c

/usr/lib/jellyfin-ffmpeg/ffmpeg -decoders | grep rkmpp

hwdec:
/usr/lib/jellyfin-ffmpeg/ffmpeg -c:v h264_rkmpp -i h264.mp4 -an -sn -f null -
/usr/lib/jellyfin-ffmpeg/ffmpeg -c:v hevc_rkmpp -i hevc.mp4 -an -sn -f null -

swdec:
/usr/lib/jellyfin-ffmpeg/ffmpeg -c:v h264 -i h264.mp4 -an -sn -f null -
/usr/lib/jellyfin-ffmpeg/ffmpeg -c:v hevc -i hevc.mp4 -an -sn -f null -

tandy-1000 · 2022-06-23T18:07:04Z

(WIP)

Decoders

$ /usr/lib/jellyfin-ffmpeg/ffmpeg -decoders | grep rkmpp
ffmpeg version 5.0.1-Jellyfin Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-libs=-lfftw3f --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-shared --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-static --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --toolchain=hardened --enable-cross-compile --enable-rkmpp --arch=arm64 --cross-prefix=/usr/bin/aarch64-linux-gnu-
  libavutil      57. 17.100 / 57. 17.100
  libavcodec     59. 18.100 / 59. 18.100
  libavformat    59. 16.100 / 59. 16.100
  libavdevice    59.  4.100 / 59.  4.100
  libavfilter     8. 24.100 /  8. 24.100
  libswscale      6.  4.100 /  6.  4.100
  libswresample   4.  3.100 /  4.  3.100
  libpostproc    56.  3.100 / 56.  3.100
 V..... h264_rkmpp           h264 (rkmpp) (codec h264)
 V..... hevc_rkmpp           hevc (rkmpp) (codec hevc)
 V..... vp8_rkmpp            vp8 (rkmpp) (codec vp8)
 V..... vp9_rkmpp            vp9 (rkmpp) (codec vp9)

H264 hw decode

$  /usr/lib/jellyfin-ffmpeg/ffmpeg -c:v h264_rkmpp -i Big_Buck_Bunny_1080_10s_1MB.mp4 -an -sn -f null -
ffmpeg version 5.0.1-Jellyfin Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-libs=-lfftw3f --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-shared --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-static --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --toolchain=hardened --enable-cross-compile --enable-rkmpp --arch=arm64 --cross-prefix=/usr/bin/aarch64-linux-gnu-
  libavutil      57. 17.100 / 57. 17.100
  libavcodec     59. 18.100 / 59. 18.100
  libavformat    59. 16.100 / 59. 16.100
  libavdevice    59.  4.100 / 59.  4.100
  libavfilter     8. 24.100 /  8. 24.100
  libswscale      6.  4.100 /  6.  4.100
  libswresample   4.  3.100 /  4.  3.100
  libpostproc    56.  3.100 / 56.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Big_Buck_Bunny_1080_10s_1MB.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
  Duration: 00:00:10.00, start: 0.000000, bitrate: 837 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 832 kb/s, 60 fps, 60 tbr, 15360 tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
[h264_rkmpp @ 0xaaab04df3320] Failed to initialize MPP context (code = -1).
[h264_rkmpp @ 0xaaab04df3320] Failed to initialize RKMPP decoder.
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_rkmpp) -> wrapped_avframe (native))
Error while opening decoder for input stream #0:0 : Unknown error occurred

nyanmisaka · 2022-06-23T18:22:18Z

[h264_rkmpp @ 0xaaab04df3320] Failed to initialize MPP context (code = -1).
[h264_rkmpp @ 0xaaab04df3320] Failed to initialize RKMPP decoder.

Add -v debug to that cli. Also you have to make sure the rockchip model is supported by rkmpp.

https://github.com/rockchip-linux/mpp/blob/786b79f9767d24bed618be60046546b508f9190e/inc/mpp_err.h#L27
https://github.com/rockchip-linux/mpp/blob/17a47899be092eb019729dcf28b3030485d35c75/mpp/mpi.cpp#L432

Actually I'm not familiar with rkmpp. You have to debug on your own.

Quedale · 2022-08-15T02:06:24Z

I doubt mpp support can be added straight out of jellyfin's setup.
It requires a specific rockchip linux kernel 4.4.x.

Fortunately armbian maintains a legacy build for that kernel which makes the initial setup easy, but without this very specific kernel, the mpp library will fail to initialize. (Mainline linux doesn't have /dev/vpu_service available)

Assuming that full rkmpp integration is just a holygrail, would there be a way to force jellyfin to "try" to use ffmpeg's mpp codec?

So far, I got a custom ffmpeg build with mpp enabled.
I also changed jellyfin's default ffmpeg binary, but I can't get hw accel to work with jellyfin.
Is there anyway I can point jellyfin to use h264_rkmpp or hevc_rkmpp with ffmpeg?

tandy-1000 · 2022-08-15T10:17:31Z

Assuming that full rkmpp integration is just a holygrail, would there be a way to force jellyfin to "try" to use ffmpeg's mpp codec?

So far, I got a custom ffmpeg build with mpp enabled. I also changed jellyfin's default ffmpeg binary, but I can't get hw accel to work with jellyfin. Is there anyway I can point jellyfin to use h264_rkmpp or hevc_rkmpp with ffmpeg?

I'm pretty sure that's what this PR is trying to do, I guess I got stalled because I'm not using that specific kernel.

nyanmisaka · 2022-08-15T11:40:09Z

Is there anyway I can point jellyfin to use h264_rkmpp or hevc_rkmpp with ffmpeg?

This PR has just enabled rkmpp encoder in our custom ffmpeg.

To utilize rkmpp you still need to modify the source code of Jellyfin to add support for it. Or use a ffmpeg wrapper to replace the software encoder libx264 with rkmpp encoder.

Quedale · 2022-08-18T01:03:23Z

Is there anyway I can point jellyfin to use h264_rkmpp or hevc_rkmpp with ffmpeg?

This PR has just enabled rkmpp encoder in our custom ffmpeg.

To utilize rkmpp you still need to modify the source code of Jellyfin to add support for it. Or use a ffmpeg wrapper to replace the software encoder libx264 with rkmpp encoder.

Thanks, I wasn't sure if I was missing anything obvious.

I suppose the solution doesn't necessarily need to be specific for rkmpp. Adding the ability to define ffmpeg's encoder/decoder wouldn't be a terrible idea.

I'm thinking that if we can define decoder/encoder separately, it could leverage 2 separate hardware.

The reason I'm thinking like this is because on RK3399, hevc and h264 shares the same VPU. Which limits its transcoding capability to 1080p@30fps if i'm not mistaken. (I read that somewhere, sorry I don't have the source)
The RockPro64 for instance has a PCI-E port, which could be used to split that load.

I might be thinking too far ahead. I just thought I'd share the thought.

Jumps down the rabbit hole

dmarkey · 2023-03-22T21:08:48Z

Hey, is this going anywhere, or am I able to test it out somehow?

IngwiePhoenix · 2023-04-25T22:37:30Z

I have a FriendlyElec NanoPi R6s.

Is there any way I can help with testing? Hardware accellerated transcoding would be really neat! :)

nyanmisaka · 2023-04-26T01:03:13Z

I have a FriendlyElec NanoPi R6s.

Is there any way I can help with testing? Hardware accellerated transcoding would be really neat! :)

#34 (comment)

nyanmisaka · 2024-01-29T15:31:38Z

Superseded by #318

nyanmisaka requested changes Jun 22, 2022

View reviewed changes

docker-build.sh Show resolved Hide resolved

nyanmisaka marked this pull request as ready for review June 22, 2022 20:08

nyanmisaka requested changes Jun 23, 2022

View reviewed changes

docker-build.sh Outdated Show resolved Hide resolved

Add rkmpp support

63943f2

tandy-1000 force-pushed the rkmpp-support branch from 4de4a3f to 63943f2 Compare June 23, 2022 15:51

Merge branch 'jellyfin' into rkmpp-support

518c199

RadxaYuntian mentioned this pull request Jun 27, 2023

No hardware accelerated video decoding and encoding radxa-build/rock-3a#7

Closed

RadxaYuntian mentioned this pull request Dec 18, 2023

Feature request: enable/integrate rockchip hwaccel support #34

Closed

nyanmisaka closed this Jan 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add rkmpp support #160

Add rkmpp support #160

tandy-1000 commented Jun 22, 2022 •

edited

Loading

nyanmisaka commented Jun 23, 2022

Shadowghost commented Jun 23, 2022

nyanmisaka left a comment •

edited

Loading

tandy-1000 commented Jun 23, 2022

nyanmisaka commented Jun 23, 2022

tandy-1000 commented Jun 23, 2022

nyanmisaka commented Jun 23, 2022 •

edited

Loading

tandy-1000 commented Jun 23, 2022

nyanmisaka commented Jun 23, 2022 •

edited

Loading

Quedale commented Aug 15, 2022 •

edited

Loading

tandy-1000 commented Aug 15, 2022 •

edited

Loading

nyanmisaka commented Aug 15, 2022 •

edited

Loading

Quedale commented Aug 18, 2022 •

edited

Loading

dmarkey commented Mar 22, 2023

IngwiePhoenix commented Apr 25, 2023

nyanmisaka commented Apr 26, 2023

nyanmisaka commented Jan 29, 2024

Add rkmpp support #160

Add rkmpp support #160

Conversation

tandy-1000 commented Jun 22, 2022 • edited Loading

nyanmisaka commented Jun 23, 2022

Shadowghost commented Jun 23, 2022

nyanmisaka left a comment • edited Loading

Choose a reason for hiding this comment

tandy-1000 commented Jun 23, 2022

nyanmisaka commented Jun 23, 2022

tandy-1000 commented Jun 23, 2022

nyanmisaka commented Jun 23, 2022 • edited Loading

tandy-1000 commented Jun 23, 2022

nyanmisaka commented Jun 23, 2022 • edited Loading

Quedale commented Aug 15, 2022 • edited Loading

tandy-1000 commented Aug 15, 2022 • edited Loading

nyanmisaka commented Aug 15, 2022 • edited Loading

Quedale commented Aug 18, 2022 • edited Loading

dmarkey commented Mar 22, 2023

IngwiePhoenix commented Apr 25, 2023

nyanmisaka commented Apr 26, 2023

nyanmisaka commented Jan 29, 2024

tandy-1000 commented Jun 22, 2022 •

edited

Loading

nyanmisaka left a comment •

edited

Loading

nyanmisaka commented Jun 23, 2022 •

edited

Loading

nyanmisaka commented Jun 23, 2022 •

edited

Loading

Quedale commented Aug 15, 2022 •

edited

Loading

tandy-1000 commented Aug 15, 2022 •

edited

Loading

nyanmisaka commented Aug 15, 2022 •

edited

Loading

Quedale commented Aug 18, 2022 •

edited

Loading