Libheif plugin support

[russkel]

Describe why it is important and where it will be useful

libheif has released a new version that has a number of new codec plugins, notably ffmpeg which would allow hardware accelerated encoding.

Describe your proposed solution

Using the dynamically compiled option for the plugins and having an option in the python library to specify the codec plugin to use.

Describe alternatives you've considered, if relevant

No response

Additional context

No response

[bigcat88]

The next release will increase the minimum required libheif to 1.14.1 (adding support for writing image orientation to a file and encoding images in YCbCr without converting to RGB), and after that in the next release after the next - your proposal is an excellent candidate for implementation.

[bigcat88]

I'm ready to increase the minimum required libheif version to 1.17.
If someone can tell me how I can build a libheif for macOS or Ubuntu/ArchLinux/AlmaLinux so that it sees ffmpeg, I'll try to add support.

[bigcat88]

do not sure that the result will be success, but looks promising.

[bigcat88]

Alpine 3.18

I do not worry about Windows, cause I am very familiar with how Windows loads libraries and there will be ok.

Now the main question:

What next after -DENABLE_PLUGIN_LOADING=ON ?
How it will be used? Is there need for heif_load_plugin or heif_load_plugins or even both?
Or specifying plugin directory only as ENV var is enough?

@russkel

[russkel]

Hi @bigcat88

This looks great! I will be honest with you, I have not ever used libheif, only your handy library ;).

Or specifying plugin directory only as ENV var is enough?

Are you intending on packaging the plugins with this library? If so then perhaps the plugin directory could be provided in someway in that it will find them without having to provide the path. The Cmake var PLUGIN_DIRECTORY is used for this.

Apart from that it looks like that's all that is required. You then specify the decoder/encoder name as part of normal libheif usage.

[russkel]

Also I wonder why there is no ffmpeg encoder, only decoder. This was what I was interested in 😅

[bigcat88]

Also I wonder why there is no ffmpeg encoder, only decoder. This was what I was interested in 😅

Because no one has yet made such a plugin and functionality as an ffmpeg encoder for libheif. Only decoder was implemented(and I already tested it locally, it has a limited abilities and can not decode some HDR HEICs currently)

Are you intending on packaging the plugins with this library?

The main reason I started work on this is to move x265 HEIF encoder to a separate python plugin to remove GPL2.0 license from binary wheels as it is hard to me to maintain binary wheels with so many libraries that has different licenses.

Second reason is the upcoming CUDA encoder/decoder for Libheif, as I hope after that also a Intel/AMD plugins will arise a bit later.

I already know before started working on this issue that ffmpeg decoder plugin has currently limited capabilities, that's why I was not rushing it

[russkel]

I was interested in ffmpeg because it has the CUDA/Intel/etc encoders, but if they're going to be directly supported at some point that's great.