Remove timezone conditionals from pipeline template #12253
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some filebeat modules can optionally convert timezones, this option works by: * Adding a field with the timezone to the generated events * Conditionally adding the timezone to the date processor in the ingest pipeline when installed This results in two possible pipelines, that are going to depend on the configuration used on setup. This is problematic because doesn't allow multiple beats to have different settings, and is also problematic when users want to change the setting, as the pipeline needs to be installed again. This change removes the condition from the template, and adds an additional date processor that uses the timezone only if this value is available. This way the installed pipeline is always the same, can be used with filebeats with different configurations and doesn't need to be updated if the setting changes in filebeat.
jsoriano
added
review
[zube]: In Review
Team:Integrations
ruflin
reviewed
May 23, 2019
There was a problem hiding this comment.
++ on this approach. We should apply this to all conditions we have in the pipelines if possible.
I wonder if this deserves a changelog entry as in case someone had it disabled but has the field, it would do processing now. I would argue this is expected and would even fix some deployments which were broken before.
jsoriano
force-pushed
the
convert-timezone-conditional-template
branch
from
5ee73a6 to
7a5c920
Compare
exekias
approved these changes
May 24, 2019
jsoriano
added a commit
to jsoriano/beats
that referenced
this pull request
May 28, 2019
jsoriano
added a commit
that referenced
this pull request
May 29, 2019
3 tasks
9 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Some filebeat modules can optionally convert timezones, this option
works by:
pipeline when installed
This results in two possible pipelines, that are going to depend on the
configuration used on setup. This is problematic because doesn't allow
multiple beats to have different settings, and is also problematic when
users want to change the setting, as the pipeline needs to be installed
again.
This change removes the condition from the template, and adds an
additional date processor that uses the timezone only if this value is
available. This way the installed pipeline is always the same, can be
used with filebeats with different configurations and doesn't need to be
updated if the setting changes in filebeat.
To be applied in other modules too if approved.