Filebeat apache2 module ssl_request_log grok pattern issue

[willemdh]

Using the standard apache2 module for Apache access logs, I get

Provided Grok expressions do not match field value errors for example with:

[10/Aug/2018:09:45:56 +0200] 172.30.0.119 TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 "GET /nagiosxi/ajaxhelper.php?cmd=getxicoreajax&opts=%7B%22func%22%3A%22get_admin_tasks_html%22%2C%22args%22%3A%22%22%7D&nsp=b5c7d5d4b6f7d0cf0c92f9cbdf737f6a5c838218425e6ae21 HTTP/1.1" 1375

Log location: /var/log/httpd/ssl_request_log

Apache version: httpd.x86_64 2.4.6-80.el7.centos.1

Filebeat 6.3.2 config:

- module: apache2
  access:
    enabled: true
    var.paths: ["/var/log/httpd/access_log","/var/log/httpd/ssl_access_log","/var/log/httpd/ssl_request_log"]
  error:
    enabled: true
    var.paths: ["/var/log/httpd/error_log","/var/log/httpd/ssl_error_log"]

error.message in Kibana:

Provided Grok expressions do not match field value: [[10/Aug/2018:09:45:56 +0200] 172.30.0.119 TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 "GET /nagiosxi/ajaxhelper.php?cmd=getxicoreajax&opts=%7B%22func%22%3A%22get_admin_tasks_html%22%2C%22args%22%3A%22%22%7D&nsp=b5c7d5d4b6f7d0cf0c92f9cbdf737f6a5c838218425e6ae21 HTTP/1.1" 1375]

For confirmed bugs, please report:

• Version: 6.3.2
• Operating System: CentOS 6
• Discuss Forum URL: https://discuss.elastic.co/t/apache-access-log-request-grok-failure/143823
• Steps to Reproduce: Parse Apache logs

[faulander]

any news here? Running into the same problem ...

[willemdh]

@ruflin @kaiyan-sheng ,

I seem to be getting some grok parse failures on our apache request logs:

Example log:

[16/Oct/2019:11:53:47 +0200] 11.19.0.217 TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 "GET /appl/ajaxhelper.php?cmd=getxicoreajax&opts=%7B%22func%22%3A%22get_pagetop_alert_content_html%22%2C%22args%22%3A%22%22%7D&nsp=c2700eab9797eda8a9f65a3ab17a6adbceccd60a6cca7708650a5923950d HTTP/1.1" -

The issue seems to be related to the '-' at the end of the logs which should beparsed by %{NUMBER:apache2.access.body_sent.bytes}

This should be migrated to something like (-|%{NUMBER:apache2.access.body_sent.bytes}) ?