Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move Metricbeat module fields to ECS #10218

Closed
wants to merge 37 commits into from
Closed

Conversation

ruflin
Copy link
Contributor

@ruflin ruflin commented Jan 21, 2019

As part of the migration to ECS and some cleanup I went through the metricbeat modules to see if there are also fields which should be migrated to ECS. As there are just too many fields to check I focused on the following 3 types: keyword, ip, contains network or host in the name.

So far I only added some notes to the fields.yml for discussion. When we agree what should happen we can apply to to all the modules/metricsets.

The idea is to apply the changes in other PR's and have this only for discussions.

If there are more fields that need migration, please comment with them here.

@ruflin ruflin added discuss Issue needs further discussion. Metricbeat Metricbeat ecs labels Jan 21, 2019
@ruflin ruflin self-assigned this Jan 21, 2019
@ruflin ruflin requested review from a team as code owners January 21, 2019 14:56
@ruflin
Copy link
Contributor Author

ruflin commented Jan 21, 2019

@webmat @andrewkroh Feedback would be appreciated.

@ruflin ruflin added the Team:Integrations Label for the Integrations team label Jan 21, 2019
Copy link
Contributor

@webmat webmat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I'm only partway done. It's a big one LOL

I will pick up at Kibana next time around

metricbeat/module/aerospike/namespace/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/apache/status/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/docker/container/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/docker/network/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/elasticsearch/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/golang/expvar/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/haproxy/stat/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/haproxy/stat/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/http/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/memcached/stats/_meta/fields.yml Outdated Show resolved Hide resolved
@ruflin ruflin mentioned this pull request Jan 22, 2019
Copy link
Contributor

@webmat webmat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, tried to cover everything. Some of the messages are short and to the point LOL.

metricbeat/module/kibana/stats/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/kubernetes/event/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/kubernetes/state_pod/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/kubernetes/state_pod/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/logstash/node/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/vsphere/virtualmachine/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/zookeeper/mntr/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/zookeeper/mntr/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/zookeeper/mntr/_meta/fields.yml Outdated Show resolved Hide resolved
@ruflin
Copy link
Contributor Author

ruflin commented Jan 23, 2019

@ycombinator Could you check the discussions in this PR which are for the Elastic Stack modules and if you agree, open PR(s) for the changes?

ruflin added 15 commits January 24, 2019 14:57
As part of the migration to ECS and some cleanup I went through the metricbeat modules to see if there are also fields which should be migrated to ECS. As there are just too many fields to check I focused on the following 3 types: keyword, ip, contains network or host in the name.

So far I only added some notes to the fields.yml for discussion. When we agree what should happen we can apply to to all the modules/metricsets.
@ruflin
Copy link
Contributor Author

ruflin commented Jan 24, 2019

@webmat For now I suggest we skip all changes related to host.hostname as we don't have a very good solution in combination with the host metadata processor.

ycombinator added a commit that referenced this pull request Jan 31, 2019
This PR updates a few fields in the Elastic stack modules in Metricbeat, viz. `elasticsearch`, `logstash`, and `kibana` to their ECS names and types.

Based on discussions happening in #10218, but only scoped to Elastic stack modules.
metricbeat/module/apache/status/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/nginx/stubstatus/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/couchbase/node/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/rabbitmq/exchange/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/rabbitmq/queue/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/rabbitmq/queue/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/uwsgi/status/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/vsphere/host/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/vsphere/host/_meta/fields.yml Outdated Show resolved Hide resolved
metricbeat/module/vsphere/virtualmachine/_meta/fields.yml Outdated Show resolved Hide resolved
Copy link
Contributor Author

@ruflin ruflin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Closing this PR as everything we wanted to address has follow up PR's.

@ruflin ruflin closed this Feb 5, 2019
@ruflin ruflin deleted the metricbeat-ecs branch February 5, 2019 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discuss Issue needs further discussion. ecs Metricbeat Metricbeat Team:Integrations Label for the Integrations team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants