Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Filebeat] Add timezone information in apache error fileset #13304

Merged
merged 5 commits into from
Aug 26, 2019
Merged

[Filebeat] Add timezone information in apache error fileset #13304

merged 5 commits into from
Aug 26, 2019

Conversation

kaiyan-sheng
Copy link
Contributor

Apache error log timestamp looks like [Thu May 12 08:28:57.652118 2011] which does not have timezone information. This PR is to add add_locale processor to take the local timezone information in the apache error ingest pipeline.

closes #12772

@kaiyan-sheng kaiyan-sheng requested a review from a team as a code owner August 21, 2019 19:13
jsoriano
jsoriano reviewed Aug 22, 2019
View reviewed changes
Copy link
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add include::../include/timezone-support.asciidoc[] to the module docs as in #13313.

@kaiyan-sheng kaiyan-sheng self-assigned this Aug 23, 2019
@kaiyan-sheng
Copy link
Contributor Author

Screen Shot 2019-08-23 at 2 38 16 PM

Tested this with Kibana and Elasticsearch locally. For error fileset, timestamp looks good with event.timezone. For access fileset, timestamp looks good too but it doesn't have the timezone information. @jsoriano Is it necessary to make these two filesets consistent?

@jsoriano
Copy link
Member

@kaiyan-sheng yes, it looks good, I guess that these four events were generated at the shown times in your timezone. Thanks a lot for double checking.
event.timezone is not needed in the access logs because timestamps there include the timezone.

@jsoriano
Copy link
Member

jenkins, test this again please

@kaiyan-sheng
Copy link
Contributor Author

@jsoriano Yes, these are from logs generated right when I started apache and filebeat on Frilday. For example in /var/log/apache2/error_log, these are the two lines matching the events:

[Fri Aug 23 14:34:08.094070 2019] [mpm_prefork:notice] [pid 41551] AH00163: Apache/2.4.33 (Unix) PHP/7.1.23 configured -- resuming normal operations
[Fri Aug 23 14:34:08.094207 2019] [core:notice] [pid 41551] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'

@kaiyan-sheng
Copy link
Contributor Author

jenkins, test this please

@kaiyan-sheng
Copy link
Contributor Author

Merging this, CI failures are not related.

@kaiyan-sheng kaiyan-sheng merged commit f21aaa6 into elastic:master Aug 26, 2019
@kaiyan-sheng kaiyan-sheng deleted the apache_tz branch August 26, 2019 19:58
@ravenbyron ravenbyron mentioned this pull request Jan 8, 2021
Closed
@kaiyan-sheng kaiyan-sheng mentioned this pull request Feb 2, 2021
Merged
kaiyan-sheng added a commit that referenced this pull request Feb 3, 2021
@kaiyan-sheng
Cherry-pick #13304 to 6.8: [Filebeat] Add timezone information in apa…
ad5e991 
…che error fileset (#23818)

* [Filebeat] Add timezone information in apache error fileset (#13304)

* Add timezone information in Apache Error metricset

(cherry picked from commit f21aaa6)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jsoriano jsoriano jsoriano approved these changes

Assignees

@kaiyan-sheng kaiyan-sheng

Labels
Filebeat Filebeat review Team:Integrations Label for the Integrations team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Filebeat apache error fileset doesn't support timezones
3 participants
@kaiyan-sheng @jsoriano @andresrc