Kibana logs messages about missing docker.container.name,docker.container.id and docker.container.image fields when user tries to import metricbeat related saved objects from 5.6.0

[bhavyarm]

Kibana version: 6.0.0 beta2

Elasticsearch version: 6.0.0 beta2

Server OS version: darwin_x86_64

Browser version: chrome latest

Browser OS version: OS X

Original install method (e.g. download page, yum, from source, etc.): from staging

Description of the problem including expected versus actual behavior: I imported beats related saved objects(mostly metricbeat) from 5.6.0 to 6.0.0(beta2) Things go mostly according to expectations except for one thing. Kibana logs messages about some of the imports failing because it couldn't locate docker.container.name, docker.container.id and docker.container.image. These fields do not exist in metricbeat 6.0.

Logging this here after discussing it with @jbudz.

Steps to reproduce:

1. Set up ES/Kibana/Metricbeat 5.6.0
2. reindex .kibana
3. Export saved objects
4. Set up ES/Kibana/metricbeat 6.0.0(beta2 BC2)
5. Import exported json from 5.6.0 - you get the error about not being able to locate metricbeat indexpattern (didn't create it earlier because of the other bug with saved objects)
6. Create indexpattern metricbeat-*
7. Import again
8. Kibana logs messages about missing docker.container.name, docker.container.id and docker.container.image

Please ignore the new visualization error from screenshot.

[bhavyarm]

cc @exekias

[exekias]

I'm testing this, it may be possible that these fields are not correctly populated by Metricbeat in the index template

[bhavyarm]

@exekias I am glad you are taking a look at this. Buzz me if you need any help. Cheers!

[jbudz]

I think the beats side is fine. My guess is when upgrading kibana the reindex script concatenates a documents type with its id, to create a new id (and avoid collisions when going to a single type). Linked saved searches are saved in a different field and they are probably not updating. This works in 5.6 because we left a layer in to search for both, but in 6.0 it's removed.

/cc @tylersmalley

[exekias]

I've researched this, it seems Metricbeat is not adding docker.container.name, docker.container.id and docker.container.image to ES index template, I created an issue for it: elastic/beats#5075.

In the other hand, it's still adding those fields to Kibana index patterns. Could that be causing this issue?

[exekias]

The issue on beats has been fixed here: elastic/beats#5077 and backported to 6.0 branch for next rc1: elastic/beats#5079

[tylersmalley]

Great, thanks @exekias.

[jbudz]

Closing, looks resolved by ^