Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change field type of http header from nested to object #5258

Merged
merged 1 commit into from
Sep 29, 2017

Conversation

ruflin
Copy link
Contributor

@ruflin ruflin commented Sep 27, 2017

The field type of http headers was set to nested instead of object. In metricbeat we normally do not used nested fields. Also nested fields are not compatible with the sorting on index time feature coming in 6.0.

The problem with indexing the headers is that it could lead to field explosion if there are many different headers. An alternative would be to not index the headers. For now my recommendation is if someone has too many headers, filters should be used to remove most of the entries before it is sent to Elasticsearch.

@ruflin ruflin added the review label Sep 27, 2017
Copy link
Contributor

@exekias exekias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a changelog? what do you think about adding a warning about fields explosion in the docs too?

@ruflin
Copy link
Contributor Author

ruflin commented Sep 28, 2017

@exekias I'm struggling where to add this in the Changelog. I personally would probably treat this as a bug as I think nested was not intentional.

Will add it to the docs.

@ruflin ruflin force-pushed the nested-to-object-in-headers branch from 221b8bf to 38eb372 Compare September 28, 2017 06:57
@ruflin ruflin added the bug label Sep 28, 2017
@ruflin
Copy link
Contributor Author

ruflin commented Sep 28, 2017

@exekias Pushed the docs change and added it to bugs for now.

The field type of http headers was set to nested instead of object. In metricbeat we normally do not used nested fields. Also nested fields are not compatible with the sorting on index time feature coming in 6.0.

The problem with indexing the headers is that it could lead to field explosion if there are many different headers. An alternative would be to not index the headers. For now my recommendation is if someone has too many headers, filters should be used to remove most of the entries before it is sent to Elasticsearch.
@ruflin ruflin force-pushed the nested-to-object-in-headers branch from 38eb372 to 8c0857c Compare September 28, 2017 06:58
@exekias
Copy link
Contributor

exekias commented Sep 28, 2017

but LGTM too 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants