Delete package index.json

[mtojek]

Delete package index.json from archive and do not serve it on endpoint (not used by Kibana).

[elasticmachine]

💚 Build Succeeded

Expand to view the summary

Build stats

• Build Cause: [Started by upstream project "Beats/package-storage/master" build number 4]

• Start Time: 2020-06-02T12:37:14.783+0000

• Duration: 4 min 1 sec

[ruflin]

Please hold back on this. I don't think we should remove this, will follow up with more details on why.

[mtojek]

I'm confused now. The index.json is still accessible on / and Kibana doesn't use the one in the archive. Could you please provide detailed overview on why this should stay as is?

[ruflin]

I misread the code and I thought it also removes the one from the service and / is not possible anymore. My bad.

About removing it from the package: Did you try building a config and save it in Kibana? I'm curious if it still works.

[jfsiii]

I ran the registry locally with

docker pull docker.elastic.co/package-registry/package-registry:PR-488
docker run -p 8080:8080  $(docker image ls docker.elastic.co/package-registry/package-registry:PR-488 -q)

and ES with (needs gold+ to set the registry)

yarn es snapshot --license=trial  -E xpack.security.authc.api_key.enabled=true

and Kibana with

 yarn start --no-base-path --xpack.ingestManager.enabled=true --xpack.ingestManager.epm.registryUrl=http://localhost:8080

then added a Data Source for nginx and everything looked good

[mtojek]

I confirm that it works too, but in the first try I've selected the aws integration and accidentally found a side issue in Kibana: elastic/kibana#67938

[nchaulet]

I am curious here if deleting index.json is not the wrong solution, should we not use it in the packageIndexHandler instead?

[mtojek]

We're adjusting the solution to not require mage build at all, hence the index.json is created dynamically. Keep in mind that in the future we'll have 100000.. of revisions, so we might face scaling problems storing all of these.

[ruflin]

@nchaulet Mid and long term users will be able to build packages in Kibana and upload them manually to Kibana. For these packages, the index.json is missing. This change makes sure, they are not used "accidentially" by the Ingest Manager when not accessing the registry directly.

[ruflin]

LGTM. Lets get it in but keep an eye out if someone is going to report issues.