Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove convert_timezone option from filebeat system module #12410

Merged

Conversation

jsoriano
Copy link
Member

@jsoriano jsoriano commented Jun 4, 2019

Timezone is now obtained from the system where filebeat is running, and
timezone conversion when parsing the timestamp is always done. This
timezone information is included in the event.timezone field, it can
be overriden with the add_fields processor, or removed with the
remove_fields processor.

Closes #11858

@jsoriano jsoriano added module discuss Issue needs further discussion. Filebeat Filebeat Team:Integrations Label for the Integrations team labels Jun 4, 2019
@jsoriano jsoriano self-assigned this Jun 4, 2019
Timezone is now obtained from the system where filebeat is running, and
timezone conversion when parsing the timestamp is always done. This
timezone information is included in the `event.timezone` field, it can
be overriden with the `add_fields` processor, or removed with the
`remove_fields` processor.
@jsoriano jsoriano force-pushed the filebeat-system-remove-convert-timezone branch from 1059f14 to ae43ca3 Compare June 4, 2019 08:50
@exekias
Copy link
Contributor

exekias commented Jun 4, 2019

👍 to this change, I would update docs to explain how timezone works and how to override it. It could be a block attached to all these modules, like we do with Metricbeat HTTP & SSL settings:

settings: ["ssl", "http"]

This will also need a changelog (breaking change)

@jsoriano jsoriano added the v7.3.0 label Jun 4, 2019
@jsoriano jsoriano force-pushed the filebeat-system-remove-convert-timezone branch from fce63bf to c0e0053 Compare June 4, 2019 10:27
@jsoriano
Copy link
Member Author

jsoriano commented Jun 4, 2019

@exekias I have added the documentation as a new file that can be imported, so it can be added to the documentation of any module, after the settings section, like this:
Captura de pantalla de 2019-06-04 12-25-45
Changelog entry is already in the breaking changes section.

Copy link
Member

@ruflin ruflin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. It's kind of a breaking change but at the same time a bug fix :-)

It's great that in case we find later a case where it is actually a breaking change (I hope not) there is a workaround by with removing the processor or removing the field.

To disable this conversion, the `event.timezone` field can be removed with
the `drop_fields` processor.

If logs are originated from systems or applications with a different timezone to
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume that is for the syslog case etc?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example. But this can really happen in any case where the application generating the logs is configured with a different timezone than metricbeat.

@exekias
Copy link
Contributor

exekias commented Jun 5, 2019

I think this can be moved out of draft :)

@jsoriano jsoriano marked this pull request as ready for review June 5, 2019 12:28
@jsoriano jsoriano requested review from a team as code owners June 5, 2019 12:28
@jsoriano jsoriano merged commit aae123e into elastic:master Jun 6, 2019
@jsoriano jsoriano deleted the filebeat-system-remove-convert-timezone branch June 6, 2019 16:19
jsoriano added a commit to jsoriano/beats that referenced this pull request Jun 6, 2019
Apply elastic#12410 to the rest of modules.
jsoriano added a commit that referenced this pull request Jun 11, 2019
Timezone is now obtained from the system where filebeat is running, and
timezone conversion when parsing the timestamp is always done. This
timezone information is included in the `event.timezone` field, it can
be overriden with the `add_fields` processor, or removed with the
`remove_fields` processor.

Apply #12410 to the rest of modules.
andrewvc pushed a commit to andrewvc/beats that referenced this pull request Jun 12, 2019
…#12410)

Timezone is now obtained from the system where filebeat is running, and
timezone conversion when parsing the timestamp is always done. This
timezone information is included in the `event.timezone` field, it can
be overriden with the `add_fields` processor, or removed with the
`remove_fields` processor.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discuss Issue needs further discussion. enhancement Filebeat Filebeat module Team:Integrations Label for the Integrations team v7.3.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Filebeat system module: Default to var.convert_timezone: true
4 participants