Potentially significant drawbacks to forcing plugin versions to remain in lock-step with Kibana version

[epixa]

The current plan for 5.0 is to enforce a requirement at installation time for plugin versions to be in lock-step with the current Kibana version. I'm not sure how a plugin author can realistically use our plugin installer when it forces them to keep their own plugin version locked with Kibana's version at even the patch level. There are two key consequences that I foresee with this model:

Every time we release new versions of Kibana, regardless of the extent of changes in Kibana itself, users and customers will need to make a hard choice:

• delay upgrading Kibana until plugin versions are updated
• run Kibana in an unrecommended state (plugins with previous version numbers)
• uninstall all plugins that do not match the Kibana version and hope they do get their versions synchronized sooner rather than later
• don't use plugins with Kibana so you don't have this problem

All four of those scenarios are less than ideal, and the first one is particularly egregious. Users shouldn't need to make decisions about delaying critical security updates to Kibana that have no realistic concerns of BC breaks just because the author of some crucial plugin they use is unavailable.

Worse than users delaying upgrading while plugin versions get in sync: plugin authors have no ability to release critical bug/security fixes to their plugins without waiting for Kibana to release a new version. That means that if a plugin introduced a security hole, even one that warranted a CVE, Kibana users couldn't take advantage of the fix until the Kibana version catches up.

I suppose the plugin author could force push over their existing releases/tags, but that would mean that no system could reliably assume that any two installs of the same plugin version were identical, which makes the very practice of versioning software pretty dubious.

Are these problems that have been anticipated when designing this new plugin model, and do we have workarounds or solutions in mind? /cc @rashidkpc @kimchy

[epixa]

The only solution I can think of that addresses all of the concerns is probably too ambitious for the short term, which would be to create an official plugin repository/service that plugin authors would publish their plugins to and the installer would download plugins from. That would allow us to track meta information about each and every plugin about version compatibility and such and make decisions about which version to download based on the current kibana version. But, as I mentioned, that would be a pretty huge undertaking to even get a simple production grade system built and deployed.

I do have a proposal that I believe is possible in the short term (even for 5.0) that at least addresses the second consequence that I described above - that plugin authors cannot release important security fixes for users without a Kibana release:

Rather than depend at all on individual plugin versions, can we require that all packs must have an ELASTIC_VERSION file in the root of the archive that includes the exact elastic version that the plugin is known to support. This file would need to be updated with every release, so the first consequence would likely still stand, but it would allow people to release new versions of their plugin while still pointing to the current stable version of Kibana. The urls we use for our packs would be based on the version in that file, so you'd still grab https://download.elastic.co/packs/5.0.0/xpack.zip. We'd encourage third party plugin authors to publish their plugins to a url that includes the ELASTIC_VERSION value as well to make sure their users are always getting the latest plugin for that version of the Elastic stack.

[w33ble]

I like the idea of the plugin specifying the version it supports. And in the absence of that param, we could just fall back to the plugin's version too. But as you say, that way plugins can push as many updates as they need to, and be at whatever version they'd like, but we still know if the plugin is supposed to work with a given version of Kibana, and can still fail at install time.

{
  "name": "my-plugin",
  "version": "5.0.14",
  "kibana-version": "5.0.0"
}

Where kibana-version must be a specific version, and not a semver pattern or anything else.

[simianhacker]

The other thought would be to leave it up to the plugin to enforce which version of Kibana it depends on; plugin authors should know what's best for their plugin.

[bevacqua]

@simianhacker In that sense why not just follow a community-established pattern like how Grunt plugins work, or how a package.json specifies what versions of Node.js that package supports?

[epixa]

I don't want to get into the weeds to much, but I think it'd be important for the version to be tracked as a single value at the root rather than interspersed throughout every plugin in the pack. Otherwise there would be multiple different implementations of product version enforcement or configuration throughout more complex packs. Basically, this is a universal requirement to our entire stack, not just Kibana.

[simianhacker]

@bevacqua That would work.

The more I think about It just feels weird that Kibana would be in charge of saying "Plugin XYZ" is not compatible.

[rashidkpc]

The problem with letting the plugin declare what version it works with is that the plugin author has no way of knowing, at the time they publish the archive, that we're going to make a breaking change in a 2 weeks. Heck, we probably don't even know.

[w33ble]

^^ what @rashidkpc said

I think letting the author specify what version they built/tested against makes sense, and forcing that to be a version and not semver, also makes sense.

I still want to hear how the community deals with this on the ES side, which has had this hard version matching for a while now...

[epixa]

We can still have the requirement that any given plugin version must be locked to only a single version (at the patch level) of the stack while also allowing plugin authors to release new versions of their plugins without waiting for a release from us.

[rashidkpc]

@epixa they can already do that if we use semver version comparison, eg, you can publish 5.0.0-YOURVERSION.

For example 5.0.0-0.1.1 and 5.0.0-0.1.2 would both be installable on Kibana 5.0.0

[epixa]

That's a great point that I hadn't considered. I assume it works with multiple dashes (e.g. 5.0.0-snapshot-mypluginv2)?

[rashidkpc]

Semver ignores everything after -, so yes

[w33ble]

That's how most people understand the -whatever in semver, but that's not actually how it works. Ignoring that bit is kind of a hack, but I'm on board with it.

Also #6402 needs to be updated then: https://github.com/elastic/kibana/pull/6402/files#diff-c4b573ceeecebe784de5e7197365d306R52

[kimchy]

the extra version after the patch release of Kibana (or the stack moving forward) is what plugin authors do today with ES itself in the 2.x release train. I think it is a good enough solution that is very infrequently needed if we stick to timely releases.

By definition, if you align plugins with Kibana versions, then the author has defined what they tested against (and they only have to deal with one version). This is the benefit we saw in ES itself. It is more work for the plugin authors, but ends up resulting in a much better quality experience for the users.

I suspect that as we mature, we might move to minor compatibility compared to patch level. To me, the balance is the end user experience of the plugins, compared to asking the plugin authors to work a bit more. For example, what would happen if we need to upgrade node version to a newer version for security reasons in a patch level, and we cause a lot of plugins to fail..., who will go and fix them, or even know they will fail to begin with...

[epixa]

@kimchy It's comforting to at least know that the semver suffix is how elasticsearch manages that at the moment. Can users upgrade elasticsearch without also upgrading their corresponding plugins?

[kimchy]

@epixa no, all the way to patch level (at least today). That was the point, that the maturity of plugin ecosystem was offloaded on the users pre 2.x (much bigger number) compared to plugin authors (much smaller number).

It also helped, to be honest, with re-qualifying certain plugins that were not maintained at all, and just "happened" to work.

[epixa]

@kimchy Have you seen problems with people delaying upgrading elasticsearch due to their plugins not updating? I'm concerned that Kibana users may have to delay updating Kibana if their installs require certain plugins that haven't been updated yet.

[kimchy]

@epixa not yet, though conceptually, Kibana will end up having more plugins down the road compared to ES. If you think about it though, this create a good amount of good pressure to get plugins that are used to get upgraded and verified to work with new versions, including contributions to those plugins. I view this "problem" as a healthy one.

[epixa]

It looks like the semver suffix satisfies my biggest concern, and while I'm still concerned about people delaying upgrading Kibana due to plugins, I suppose that's something we won't really know until we ship the change and see it in action.

Unless someone else wants to jump in, I'm going to close this issue later today.

[w33ble]

#6402 will get updated to comply with the semver check discussed here. I think we're good here.

[jccq]

Guys hi, @epixa mentions "running kibana in an unrecommended state" (letting it try to use plugins that havent yet received an update). Is there going to be such a switch ?

my2c - Given that a lot of plugins for Kibana are going to be pretty visual and not as likely to subtly break as in Elasticsearch (and also because Kibana is pretty much "read only") i would find one such switch .. sensible to have.

[elvarb]

Sorry to barge in after such a long time.

Isn't the semver solution here similar to the one Logstash uses? That plugins define minimum and maximum version level they work on.

In the past, the early days of Logstash I saw people hanging into previous versions of Logstash and Elasticsearch simply because essential plugins were not supported in the latest releases. Most of the time the only thing that needed to do was modify the defined supported versions in the plugin code.

Would it make sense to allow Kibana to run plugins that do not define the latest version as compatible but also in the interface show a clear warning that Plugin X has not been tested for this version of Kibana, use at your own risk ?

I have managed Atlassian products for example for 5-10 years, they have a huge plugin community and with each release of Confluence or JIRA you run the risk of leaving out some plugins. My way of managing the use of the products has always been to warn users that do not get attached to your third party plugins, use the standard ones if you can. (btw, they have the same structure like discussed here, min and max versions)

[epixa]

@elvarb A warning isn't serious enough of a reaction to certain plugins not being compatible. Let's imagine that someone was using a custom github auth plugin to block access to their data in Kibana, and that plugin was relying on an authentication hook that was removed or altered in such a way that the plugin would no longer have any effect. That security plugin no longer operates and Kibana is wide open when it's installed, and people don't know that until after they upgrade Kibana and their data gets exposed to the world.

Elastic is constantly testing our own plugin for X-Pack security against every release of Kibana, so we've removed that risk for Kibana/X-Pack users. The version requirement is enforcing the same expectation on plugin authors as well until Kibana's plugin API is stable enough that this sort of nightmare scenario is no longer realistic.

[elvarb]

@epixa that is a good point. I was stuck in thinking that plugins would mostly be visualization and interface based. Maybe then class what type of plugin there are. Time will tell how the ecosystem Developes