Discussion regarding the runtime and approach

[abique]

Hi,

Thank you very much for your work on getting plugins into flatpak.
I'm a Bitwig Studio developer and I recently published Bitwig to Flathub.

I am also the maintainer of the linux port of u-he plugins, and I consider publishing those to flathub, but I'd like to clarify a few things first.

If I understand correctly, using Bitwig from flatpak will give me access to the plugins installed:

• on the system
• on the user home directory
• on flatpak if they share the same runtime

Is that correct?

What happens to plugins using an older runtime?
They may still work with a more recent runtime? It is forward compatible most of the time I suppose?

If the runtime needs to be the same for the plugin and the DAW then the transition to a newer runtime is tricky isn't it? It push additional problems for both the user and the developers making it a bad experience for the end user because he'd need to understand the runtime issue.

To me if we say that a plugin can target an old runtime and be used by a daw using a recent runtime, then it is a no brainer, the plugins needs to target the oldest possible runtime and the daw the most recent one. But I doubt that this approach will guarantee that the plugin will work.

Regarding u-he plugins, they are linked statically and only exports the vst2 and vst3 entry points. So they are safe from symbol clash during after dlopen(). And they don't depend on the flatpak runtime. So if there is a way to say "any runtime" or "no runtime"?

Can a plugin installed by flatpak be used by an app started on the system?

As a side note I wrote this a while ago: https://gist.github.com/abique/4c1b9b40f3413f0df1591d2a7c760db4 which is a different approach, but it would take years, had a mixed reception (so the idea is maybe wrong?) and might also never happen so I'm very happy if we have a good solution today.

Regards,
Alex

[abique]

Why isn't the version of org.freedesktop.LinuxAudio.Plugins implied by the package runtime version?

[tank-trax]

I have a few questions... as I have hit a brick wall

how does one point a DAW that is in one container to a plugin that is in another container? where is the link? because it's not possible to navigate to that location... it's as if Host and Plugin are on different VM or Emulations...

What would be the mechanism to bridge from Host to Plugin with Flatpak?

currently if I open Bitwig it can only read plugins that our outside the Flatpak ecosystem and only if they are in the /home/user folder

[hfiguiere]

If I understand correctly, using Bitwig from flatpak will give me access to the plugins installed:

* on the system

No. Flatpak app don't have access to anything installed as part of the OS (think /usr /opt or whatever else).

* on the user home directory

Yes. Apps can have access to the $HOME directory. It is the default for most app that support audio plugins. See --filesystem=home in the finish-args section which Bitwig has.
There are still possibilities it doesn't work, but this is more for plugins that would be dynamically linked.

* on flatpak if they share the same runtime

Any Linux audio plugins packaged for the same runtime version. Right now all are on 20.08. I made sure to transition quickly from 19.08 when the transition happened.

What happens to plugins using an older runtime?
They may still work with a more recent runtime? It is forward compatible most of the time I suppose?

They are ignored, ie Flatpak will not load them with the app. The extension point is versioned.

If the runtime needs to be the same for the plugin and the DAW then the transition to a newer runtime is tricky isn't it? It push additional problems for both the user and the developers making it a bad experience for the end user because he'd need to understand the runtime issue.

There are issues that needs work.
Like this one flatpak/flatpak#4208

To me if we say that a plugin can target an old runtime and be used by a daw using a recent runtime, then it is a no brainer, the plugins needs to target the oldest possible runtime and the daw the most recent one. But I doubt that this approach will guarantee that the plugin will work.

Regarding u-he plugins, they are linked statically and only exports the vst2 and vst3 entry points. So they are safe from symbol clash during after dlopen(). And they don't depend on the flatpak runtime. So if there is a way to say "any runtime" or "no runtime"?

The runtime is the runtime from the app anyway. I can see if maybe we can have a unversionned extension point in parallel.
And just to clarify: Freedesktop 20.08 is the base for KDE 5.15, GNOME 3.38 and GNOME 40.

Can a plugin installed by flatpak be used by an app started on the system?

No. This is solely for flatpak.

[hfiguiere]

I have a few questions... as I have hit a brick wall

how does one point a DAW that is in one container to a plugin that is in another container? where is the link? because it's not possible to navigate to that location... it's as if Host and Plugin are on different VM or Emulations...

Each app is in its own sandbox.

The extension installed are automatically "mounted" following the extension point. For bitwig it is defined as:

add-extensions:
  org.freedesktop.LinuxAudio.Plugins:
    directory: extensions/Plugins
    version: '20.08'
    add-ld-path: lib
    merge-dirs: lxvst;vst3
    subdirectories: true
    no-autodownload: true

A plugin is installed in the sandbox in /app/extensions/Plugins/Name
And from that directory, the content of vst3 is linked into /app/extensions/Plugins/vst3 and the content of lxvst linked into /app/extensions/Plugins/lxvst.
(same applies for lv2 and ladspa, but since it's not supported in Bitwig, there are ignored)

What would be the mechanism to bridge from Host to Plugin with Flatpak?

None. Except for what is $HOME.

currently if I open Bitwig it can only read plugins that our outside the Flatpak ecosystem and only if they are in the /home/user folder

There is an issue with Bitwig is that the search path need to be set (I tested that, it still works)
/app/extensions/Plugins/lxvst and /app/extensions/Plugins/vst3 for VST2 and VST3 respectively (packaged as Flatpak). Ideally if there is a way to make this by default, as LXVST_PATH and VST3_PATH env while set, are ignored by Bitwig.

As for the argument that VST3_PATH isn't part of the VST3 spec I filed an issue and there is no standard way otherwise.
steinbergmedia/vst3sdk#67
Ardour and Carla support it though.

For example flatpak search vst3 should list plugins that have VST3.

[hfiguiere]

BTW I have a manifest for Podolski if you want to give it a try :-)

[tank-trax]

it's still not clear to me as the Bitwig container is here

/var/lib/flatpak/app/com.bitwig.BitwigStudio/x86_64/stable/active/files/extensions/Plugins/

and Surge VST3 is here

/var/lib/flatpak/runtime/org.freedesktop.LinuxAudio.Plugins.Surge/x86_64/20.08/active/files/vst3/Surge.vst3/

how does Bitwig locate and subsequently load the Surge VST3?

[hfiguiere]

You instruct Bitwig to look for ST3 in /app/extensions/Plugins/vst3

flatpak does the rest.

(think of mounting the extension filesystem in the right place in the bitwig Flatpak sandbox)

[tank-trax]

I got it by adding...

[tank-trax]

yes /app/extensions/Plugins/vst3 is better

thanks!!!

[hfiguiere]

This is what I tested with:

[tank-trax]

@hfiguiere

BTW I have a manifest for Podolski if you want to give it a try :-)

Sure thing :-)

[abique]

Thank you very much! :)
Just add /app/extensions/Plugins/ bitwig should deal with the rest.
I'll try to push some fixes for 3.3.8.
Regarding u-he plugins, if you can get them working I think it is OK to push them using extra-data. Let's discuss it again because I want to change the installer to do system install or user install but using standard paths.

[tank-trax]

it's better to use /app/extensions/Plugins/vst3 and /app/extensions/Plugins/lxvst as it isolates the plugins only rather than the whole sandbox when using /app/extensions/Plugins/

you will notice duplicates otherwise

[abique]

I have a strong feeling that plugins using an older runtime could be mounted by an application using a newer runtime. That would work most of the time.

[hfiguiere]

Thank you very much! :)
Just add /app/extensions/Plugins/ bitwig should deal with the rest.

You don't want to do that..
Specifically reference /app/extensions/Plugins/lxvst and /app/extensions/Plugins/vst3

Regarding u-he plugins, if you can get them working I think it is OK to push them using extra-data. Let's discuss it again because I want to change the installer to do system install or user install but using standard paths.

I can't get them to work since they insist on "installing" stuff in $HOME. I actually only tried Podolski and it can't find the UI (and I suspect the rest).

[abique]

I also wonder about two things:

1. what is the workflow for a plugin developer?
2. what is the workflow for a DAW developer?

The plugin developer need a way to quickly iterate:

• build (dev problem)
• "deploy", how does it look if the DAW is installed via flatpak and does not have access to the filesytem?
• test & debug

The daw developer needs to be able to debug his plugin host, but how does he load a plugin installed via flatpak into his daw being developed without a sandbox? Does he need to "deploy" to flatpak first?

[abique]

Maybe we need to start this discussion on the flatpak mailing list?

[Bleuzen]

Hi, I'm still very much interested in this discussion because the current situation sucks for both, maintainers and users.
The thing that bothers me the most is that we branch plugins.

First, this makes it harder for maintainers because we have to maintain 2-3 branches of the exact same plugin(s). Feels like a waste of time and can lead to lacking motivation to backport stuff everytime. Would be great to have only one package/branch per plugin which works everywhere.

It gets even worse if you consider the users perspective.
Currently it is really hard for them because one can't simply "install a plugin".

It should be:

• flatpak install flathub org.freedesktop.LinuxAudio.Plugins.Helm OR installing via graphical software manager with one click on a install button
• it is now available in all hosts (or at least those with same or higher runtime version). DONE.

While it currently is:

• You have to get your head around the branching going on. Not everyone (non technical people) is able to understand this and even if, it is very frustrating.
• Even if you managed to install a plugin, it will be available in some hosts, but not in others, because apps use different runtime version. Again, can be really frustrating.
• Even if you understand that you manually have to install multiple versions of the plugins, you will first have to go to GitHub and check out the manifests of the apps you want to use the plugins with to see which branches you need to install.
• Even if you successfully installed multiple version of the exact same plugin, it is really sad. Thinking about my notebook, constantly running out of space and also update sizes since the same plugins have to be downloaded multiple times.
• Every year when there is a new runtime version the workflow will break. If users currently use Ardour (currently 21.08 branch) and have all the plugins from the 21.08 branch installed, next year when Ardour switches to 22.08, users who just apply causual updates will every year find everything broken afterwards, because plugins are missing.

Basically we expect everyone to be a technical person / power user.
Also this branching makes it hard to display plugins in graphical software stores. Should it just display 3 (or next year 4) times the same plugin? This is very confusing to users and may to some just look like a bug. How should users (who don't track manifests on github) know which one to install and that they most likely have to install multiple ones. No. Tbf. this is unacceptable.
There should be just one install button and the plugin should work everywhere, in every host, no matter the runtime version.

---

Sorry for the rant. This can be easly frustrating, also to me who knows how to deal with it.
This should really be improved.

I see that there is something going on (or not? nothing happened since april)... at least there is an issue for flatpak:
flatpak/flatpak#4208

This is about flatpak automatically installing plugins from new branches.
While this seems like a step forward, it doesn't fix the problem, only feels like putting make-up on to make it look a little more friendly.

This doesn't fix:

• Users having to understand branching to initially install plugins
• Wasting space and update time because most likely multiple plugin branches will need to keep installed for different hosts
• Still breaking workflows every year, for example if Ardour is already updated to 22.08 runtime while some plugins don't have a 22.08 branch yet.
• Extra work for maintainers, having to update multiple plugin branches
• Being able to easly display the plugins one entry per plugin in graphical stores and having only one install button which installs it for every host

So we cleary need to get rid if this branching if possible. But at this point I'm not sure yet. From my understanding it should be possible. Having only one branch per plugin, building all the plugins against an older SDK while using newer SDK for hosts/apps. Usually this works.

For example have a look at the helm plugin in the archlinux repository.
It was last (re-)built on 2020-04-05. So it was built against libraries from april 2020 and this exact package still works great nowadays on a fully updated system, in any newer host.
There are more examples, for example downloaded binaries from CI builds against older Ubuntu LTS versions usually work great on up to date rolling distros and more...

Also this quote from @abique makes me even more confident about it:

I have a strong feeling that plugins using an older runtime could be mounted by an application using a newer runtime. That would work most of the time.

This makes me pretty sure that plugin binaries built against the 20.08 SDK should work in 21.08 runtime.
If this is true, why don't we simply build all plugins agains 20.08 SDK (if possible)?
(Of course this won't work for every plugin, for example really new ones, needing more recent libraries, but for most it could work I guess.)

Then we could, in host apps like Ardour, mount plugins from multiple branches, which could look like this in the manifest:

...
  "add-extensions": {
    "org.freedesktop.LinuxAudio.Plugins": {
      "directory": "extensions/Plugins",
      "version": "21.08;20.08",
      "add-ld-path": "lib",
      "merge-dirs": "ladspa;lv2;lxvst;vst3",
      "subdirectories": true,
      "no-autodownload": true
    }
  },
...

or maybe, if we can't repurpose the version field (or completely ommit it?):

...
  "add-extensions": {
    "org.freedesktop.LinuxAudio.Plugins": {
      "directory": "extensions/Plugins",
      "version": "21.08",
      "add-ld-path": "lib",
      "merge-dirs": "ladspa-21.08;lv2-21.08;lxvst-21.08;vst3-21.08",
      "subdirectories": true,
      "no-autodownload": true
    },
    "org.freedesktop.LinuxAudio.Plugins": {
      "directory": "extensions/Plugins",
      "version": "20.08",
      "add-ld-path": "lib",
      "merge-dirs": "ladspa-20.08;lv2-20.08;lxvst-20.08;vst3-20.08",
      "subdirectories": true,
      "no-autodownload": true
    }
  },
...

or something else but like this, which would make Ardour load all plugins built against 20.08 and 21.08 SDK.

For now to me that looks like it could work. But at the same time I'm not sure. I guess there is a reason why @hfiguiere didn't choose to do it like this from the beginning which I currently fail to see/understand. I'm interested in an explanation in this case.