-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Filebeat module won't start groking unless the default prospectors are turned off #3912
Comments
What happens here is that two prospectors are created for the same file. This is not supported by Filebeat, but actually seems to work in my case. Perhaps it didn't work for you or something else was wrong, in any case this is an unsupported situation that's easy to stumble upon. In 6.0 this will not be an issue, as we plan to change the default configuration to have no prospectors enabled by default. I think for 5.x we should add a note to the FBM tutorial to disable the default prospector before enabling any modules. @dedemorton What do you think? |
It used to query `*` and I changed it to query `_exists_:system.syslog`, which will cause less confusion on the Syslog module dashboard. Probably it added to the confusion in elastic#3912.
It used to query `*` and I changed it to query `_exists_:system.syslog`, which will cause less confusion on the Syslog module dashboard. Probably it added to the confusion in #3912.
@tsg I can add a note when I make the other changes that Bohyun suggested. |
If this does not and will not work for 5.x, then why do we ship with two prospectors enabled in 5.x? Isn't it a better experience (esp. for Filebeat modules) if we comment them out by default so the users don't have to manually edit the file? As you said, once we change this behavior in 6.x, I would imagine it makes sense to enable for both prospectors. Thoughts? |
It used to query `*` and I changed it to query `_exists_:system.syslog`, which will cause less confusion on the Syslog module dashboard. Probably it added to the confusion in elastic#3912. (cherry picked from commit 6b8cab6)
This is solved in 6.0. For 5.x, we think it's too big of a change to make now, considering that the modules are still in beta, so not really an alternative yet. |
Using 5.3 on Mac OS X
I did not change any of the contents in filebeat.yml and ran the following command
./filebeat -e modules=system -setup
Even though I was able to see the documents come in from Kibana discover tab, the Grok processes did not kick in because the default prospectors need to be turned off by commenting them out as below.
#- input_type: log
# Paths that should be crawled and fetched. Glob based paths.
#paths:
#- /var/log/*.log
@andrewkroh suspects that this is due to the prospector that is in the default config file monitors
/var/log/*.log
which probably competes with the system module's prospector.If the users are running the filebeat module setup command, then we should make sure to disable the default filebeat prospector setting.
cc: @tsg - this was one of the reasons why I did not see the built-in syslog dashboard working.
The text was updated successfully, but these errors were encountered: