Discuss: Move to zip instead of tar.gz for the format of the package.

[ph]

Currently, the package registry uses the tar.gz as the compressed package format for shipping the packages down to Kibana. Sadly this format is not supported directly by Windows. It would be nice if we would move to a zip file format this would allow anyone to download the package on windows unzip it and inspect the content.

@ruflin I vaguely remember you mentioning that there were some issues in Kibana that prevent us from moving to ZIP.

[ruflin]

We had zip support in the past but it was removed here: #59 @jfsiii might chime in what works better and why on the Kibana side.

With the work @mtojek is doing on #472 it will be pretty simple to support both. If we support both, we need to discuss if we should adjust the /search API output:

Example:

  {
    "description": "aws Integration",
    "download": "/epr/aws/aws-0.0.3.tar.gz",
    "icons": [
      {
        "src": "/package/aws/0.0.3/img/logo_aws.svg",
        "title": "logo aws",
        "size": "32x32",
        "type": "image/svg+xml"
      }
    ],
    "name": "aws",
    "path": "/package/aws/0.0.3",
    "title": "aws",
    "type": "integration",
    "version": "0.0.3"
  },

[jfsiii]

I'd like to support both/multiple. tar.gz makes things easier on the Manager if it's on darwin or linux. .zip would be great for those on Windows.

I'm not sure about the specific API. Some options that come to mind are:

• add a query param like archive_format=zip with a default and the download key would remain a string.
• move to an array of objects like icons

[ruflin]

@jfsiii If you are thinking of an array, are you thinking of one with objects inside? If we go this path, it would allow us to add "metadata" to each download link potentially if needed so Kibana for example does not have to take the suffix to decide which download link to use but could do it on a specific field that tells it, it is a tar.gz.

[ruflin]

I opened #512 as a draft to get some initial feedback.

[roncohen]

we should only support one format. We agreed that unless there's big complications, it should be ZIP. It's natively support in MacOS and Windows and Linux users will be able to manage.

[ph]

Adding this to b2, we need to do this before we go out of beta.

[ruflin]

I opened a PR to remove the list again as we didn't use it yet: #584 If we move to zip, will need to figure out a non breaking way in b2, but should be doable :-)

[jfsiii]

@ruflin pinging since this is targeting 7.10. I don't think it'll be a big deal for Kibana, but LMK when there is a PR/Docker image I can use to test

[ruflin]

@jfsiii Can you forsee any platform / streaming specific challenges around decompression zip vs tar.gz in Kibana? Will ping you as soon as this moves forward. We need to first sort out a separate registry for 7.10 as this will be breaking.

[jfsiii]

@ruflin writing down what I said to you on Monday, I think we'll be ok. We used to support both. We currently only deal with it in one place, where we decompress it into a Buffer. I think we can replace untar* with unzip* in one place and be good

https://github.com/elastic/kibana/blob/b4b6428c1ca095f8e1388bcec29d8d508ec08a22/x-pack/plugins/ingest_manager/server/services/epm/registry/index.ts#L178-L187

[ph]

@skh for awareness with your work for installing a package without the registry.

[skh]

We used to support both.

On the Kibana/EPM side I'd like to suggest that we support both again, at least for a transitional period until all registries are safely switched over.

[ruflin]

I opened #628 with the change. @jfsiii Could you take a stab at the change? If it is "simple" I'm ok to support both in Kibana for a few days so we can roll out everything properly and things keep working.

[jfsiii]

@ruflin @skh I put my first pass at it in as a draft PR elastic/kibana#76197

[ruflin]

@jfsiii Great, did you manage to test this with the changed registry?