Add timezone to the exported event

[monicasarbu]

There are a few requests to export timezone in order to be able to find the local timestamp. For that, we think the best option would be to export also the timezone (optionally) in the event, and make sure the @timestamp is always sent in UTC.

It still needs to be discussed what it's the best option to enhance the event with the timezone field, using a processor or a configuration option (maybe under the Elasticsearch output?).

Related to this, the following PRs were created by the community: #3622, #3743

[martinscholz83]

Hi @monicasarbu. If we just want to export the timezone then maybe a config option is sufficient, because we don't change any other field. I would use a processor if we want to create any target field. For example

zone: "America/New_York"
target: "localtime"

[monicasarbu]

If we add a config option to add the timezone information, then we need to add it to the output as it's a characteristic of the output, so all the supported outputs would have a timezone configuration.

Considering that only few users would need timezone, I think it's much easier to add this functionality in a processor than with a configuration option. If we create a processor, then we can additionally add localtime for example.

We could have an add_location processor that adds the timezone. It would be nice to be able to format the timezone (like America/Curacao or -04:00). By default, the timezone should not be exported.

[martinscholz83]

Hi @monicasarbu. Sounds like this is something what i have started in #3743. So if i understand it correctly you want to give the user an option to set a timezone and put that time into an field localtime.
For example

processors:
 - add_timezone:
     zone: "America/Curacao /required     
     target: "localtime" /optional
     export_timezone: true /optional

...
...

output
{
  "@timestamp": "2016-05-23T08:05:34.853Z", 
  ...,
  ...,
  
  "localtime": "2016-05-23T08:05:34.853-04:00",
  "timezone": "America/Curacao"
}

[monicasarbu]

@maddin2016 Yes, we discussed between us and we decided it's best to create a processor that gives the user the option to add the timezone. We are not all convinced if it's a good idea to export localtime as well, so I would say to export only timezone for now. I would call the processor add_location or something more general in order to leave the opportunity to add additional fields in the future (for example if the users are requesting to add also localtime).

[ruflin]

I would not go with add_location as a name as I would expect geoip info under this processor. I quite like add_timezone as I currently don't have a better idea.

Perhaps the field should be added under beat.timezone as this is an information about the beat itself on where it runs.

[monicasarbu]

My mistake, I meant add_locale.

[monicasarbu]

I am re-opening this issue, as it will be nice to be able to configure the format of the timezone. Currently, the timezone has the format something like this CEST.

[ruflin]

@monicasarbu Should we open a new issue for that? Because "Add timezone to the exported event" is done.

[monicasarbu]

@ruflin I didn't open a new issue as we discussed in this issue about the possible formats for timezone and it's part of the add_locale processor. If you feel it's better, we can also open another issue.

[monicasarbu]

Closing this issue in favor of #3974.