Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filebeat Apache module grok pattern should handle absent byte count #3833

Closed
cwurm opened this issue Mar 28, 2017 · 5 comments
Closed

Filebeat Apache module grok pattern should handle absent byte count #3833

cwurm opened this issue Mar 28, 2017 · 5 comments
Labels

Comments

@cwurm
Copy link
Contributor

cwurm commented Mar 28, 2017

In the Apache Combined Log Format, an absent byte count is signified by -. This has to be included in the grok pattern.

Should be something like: "%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \\[%{HTTPDATE:timestamp}\\] \"%{WORD:verb} %{DATA:request} HTTP/%{NUMBER:httpversion}\" %{NUMBER:response:int} (?:-|%{NUMBER:bytes:int}) %{QS:referrer} %{QS:agent}"

Currently, the option - is missing from the default pattern in https://github.com/elastic/beats/blob/master/filebeat/module/apache2/access/ingest/default.json

@cwurm
Copy link
Contributor Author

cwurm commented Mar 29, 2017

Another thing: The grok pattern for access logs has spaces after opening parenthesis and not before at the end: %{NUMBER:apache2.access.body_sent.bytes}( \"%{DATA:apache2.access.referrer}\")?( \"%{DATA:apache2.access.agent}\")?"

Should be %{NUMBER:apache2.access.body_sent.bytes} (\"%{DATA:apache2.access.referrer}\")? (\"%{DATA:apache2.access.agent}\")?"

@cwurm cwurm changed the title Apache module should handle absent byte count Filebeat Apache module grok pattern absent byte count and referrer+agent incorrect whitespace Mar 29, 2017
@cwurm
Copy link
Contributor Author

cwurm commented Mar 29, 2017

Hang on, that the whitespace didn't work was my mistake when I edited the grok pattern to fix the bytes. Apologies.

@cwurm cwurm changed the title Filebeat Apache module grok pattern absent byte count and referrer+agent incorrect whitespace Filebeat Apache module grok pattern should handle absent byte count Mar 29, 2017
@ruflin
Copy link
Contributor

ruflin commented Mar 30, 2017

@cwurm Could you open a PR with this changes? Then we directly see how it affects our tests.

@cwurm
Copy link
Contributor Author

cwurm commented Mar 30, 2017

@ruflin done: #3863

@ruflin
Copy link
Contributor

ruflin commented Mar 31, 2017

@cwurm Thanks a lot for the fix. Closing this one.

@ruflin ruflin closed this as completed Mar 31, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants