Filebeat module won't start groking unless the default prospectors are turned off

[bohyun-e]

Using 5.3 on Mac OS X

I did not change any of the contents in filebeat.yml and ran the following command
./filebeat -e modules=system -setup
Even though I was able to see the documents come in from Kibana discover tab, the Grok processes did not kick in because the default prospectors need to be turned off by commenting them out as below.

#- input_type: log
# Paths that should be crawled and fetched. Glob based paths.
#paths:
#- /var/log/*.log

@andrewkroh suspects that this is due to the prospector that is in the default config file monitors /var/log/*.log which probably competes with the system module's prospector.

If the users are running the filebeat module setup command, then we should make sure to disable the default filebeat prospector setting.

cc: @tsg - this was one of the reasons why I did not see the built-in syslog dashboard working.

[tsg]

What happens here is that two prospectors are created for the same file. This is not supported by Filebeat, but actually seems to work in my case. Perhaps it didn't work for you or something else was wrong, in any case this is an unsupported situation that's easy to stumble upon.

In 6.0 this will not be an issue, as we plan to change the default configuration to have no prospectors enabled by default.

I think for 5.x we should add a note to the FBM tutorial to disable the default prospector before enabling any modules. @dedemorton What do you think?

[dedemorton]

@tsg I can add a note when I make the other changes that Bohyun suggested.

[bohyun-e]

@tsg

I think for 5.x we should add a note to the FBM tutorial to disable the default prospector before enabling any modules

If this does not and will not work for 5.x, then why do we ship with two prospectors enabled in 5.x? Isn't it a better experience (esp. for Filebeat modules) if we comment them out by default so the users don't have to manually edit the file? As you said, once we change this behavior in 6.x, I would imagine it makes sense to enable for both prospectors. Thoughts?

[tsg]

This is solved in 6.0. For 5.x, we think it's too big of a change to make now, considering that the modules are still in beta, so not really an alternative yet.