Fix grok pattern in filebeat module system/auth without hostname (ela…

…stic#4224)

Some log lines like `Feb  9 21:20:08  sshd[8317]: last message repeated 2 times` do not contain a hostname. This change in the grok pattern makes the hostname optional.

* Make system module tests more verbose on error

CHANGELOG.asciidoc

@@ -71,6 +71,7 @@ https://github.com/elastic/beats/compare/v5.1.1...master[Check the HEAD diff]
 - Properly shut down crawler in case one prospector is misconfigured. {pull}4037[4037]
 - Fix the Mysql slowlog parsing of IP addresses. {pull}4183[4183]
 - Fix issue that new prospector was not reloaded on conflict {pull}4128[4128]
+- Fix grok pattern in filebeat module system/auth without hostname. {pull}[]
 
 *Heartbeat*
 - Add default ports in HTTP monitor. {pull}3924[3924]

filebeat/module/system/auth/ingest/pipeline.json

@@ -15,7 +15,7 @@
           "%{SYSLOGTIMESTAMP:system.auth.timestamp} %{SYSLOGHOST:system.auth.hostname} sudo(?:\\[%{POSINT:system.auth.pid}\\])?: \\s*%{DATA:system.auth.user} :( %{DATA:system.auth.sudo.error} ;)? TTY=%{DATA:system.auth.sudo.tty} ; PWD=%{DATA:system.auth.sudo.pwd} ; USER=%{DATA:system.auth.sudo.user} ; COMMAND=%{GREEDYDATA:system.auth.sudo.command}",
           "%{SYSLOGTIMESTAMP:system.auth.timestamp} %{SYSLOGHOST:system.auth.hostname} groupadd(?:\\[%{POSINT:system.auth.pid}\\])?: new group: name=%{DATA:system.auth.groupadd.name}, GID=%{NUMBER:system.auth.groupadd.gid}",
           "%{SYSLOGTIMESTAMP:system.auth.timestamp} %{SYSLOGHOST:system.auth.hostname} useradd(?:\\[%{POSINT:system.auth.pid}\\])?: new user: name=%{DATA:system.auth.useradd.name}, UID=%{NUMBER:system.auth.useradd.uid}, GID=%{NUMBER:system.auth.useradd.gid}, home=%{DATA:system.auth.useradd.home}, shell=%{DATA:system.auth.useradd.shell}$",
-					"%{SYSLOGTIMESTAMP:system.auth.timestamp} %{SYSLOGHOST:system.auth.hostname} %{DATA:system.auth.program}(?:\\[%{POSINT:system.auth.pid}\\])?: %{GREEDYMULTILINE:system.auth.message}"
+					"%{SYSLOGTIMESTAMP:system.auth.timestamp} %{SYSLOGHOST:system.auth.hostname}? %{DATA:system.auth.program}(?:\\[%{POSINT:system.auth.pid}\\])?: %{GREEDYMULTILINE:system.auth.message}"
         ]
       }
     },

filebeat/tests/system/test_modules.py

@@ -108,7 +108,7 @@ def run_on_file(self, module, fileset, test_file, cfgfile):
             assert obj["fileset"]["module"] == module, "expected fileset.module={} but got {}".format(
                 module, obj["fileset"]["module"])
 
-            assert "error" not in obj
+            assert "error" not in obj, "not error expected but got: {}".format(obj)
 
             if module != "auditd" and fileset != "log":
                 # There are dynamic fields in audit logs that are not documented.