-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
default filebeat.prospectors.paths to be commented out #3442
Comments
As you mentioned we try to make the beats work out of the box, mean directly shipping data without having to modify the configuration first which also means finding the configuration when installed as package. The default output is elasticsearch at I agree we should find ways to give the user some "incentive" to think about which logs he wants to harvest and configure it accordingly. Perhaps an other default could help? |
Correct, that is not the issue/point, sorry if I misrepresented. The "out of the box" experience we target with the stack in general is when a person sets up the stack on their laptop or development environment correct? In this situation ingesting the OS logs of your personal laptop probably not expected and can have negative side affects. For example in training classes we see this happen with students all the time. As mentioned, since the default output is Not sure if my above comments make a difference in the viewpoint on the issue? |
I agree that we should improve the behaviour here. The thing I worry about when we just comment it out is that people will start filebeat and say: Hm, nothing happens. The nice part now when doing demo's for example is that lots of data directly shows up in Kibana. Perhaps this problem solves itself with the addition of the filebeat modules. So if you run filebeat and you want to work it out of the box The above would require that we adapt our getting started guide to focus more on modules. |
Well that would only be true if no other changes are made. For example if no prospector is defined (ie no paths listed) then as a user I would expect either a warning, error, or message to be reported by filebeat and a message suggesting what I might do. |
Currently filebeat will stop and print out and error. But a good error is one of the parts I were struggling with pre existence of modules. It's quite tricky to tell users in a short error message what to exactly do in a yaml file and people will get it right (without messing with the indentation) and people would have to touch the config file already for the first time use (which I don't like). But we have that kind of solved now as we can have something like `No prospector defined, use "./filebeat --module=system" to load your system log files or have the system module as defaults. |
Sorry if I'm over trivializing, but isn't this really simply as you put all the details in comments in the yaml file and then in the error from command just point to the appropriate unique test to look at in the yaml file? |
Theoretically yes, in practice unfortunately no because of YAML and indentation ... Also it does not solve the problem that I want users to be able to get started without having to touch the config file. |
I found the current behavior unpleasant as a first time user of filebeat, to the point where I immediately stopped experimenting in fear of other unexpected behavior, until I had time to more carefully read documentation. The potential side effects in this case negate any value this default could provide. Zero required config is a worthy goal, but filebeat is something that, uh, reads files. Users expect to have to tell it what files. As a user of filebeat or most programs I execute on the command line, I expect it to do nothing, maybe throw an error, if I don't tell it which files to process and what to do with them. Exceptions exist, but are very well known and thus expected (ie: "ls" with no arguments) and generally have no side effects. mv with no arguments does nothing but print it's usage. The error can be concise and ideally with some nice FAQ in an obvious place for search engines to find and use to help your users understand what to do. "FileBeat: Nothing to harvest/prospect/whatever, provide a valid harvester/prospector/whatever configuration." I'm sure it could be phrased better. ps |
This does two changes: * Adds `enabled: false` in the default prospector in the short and long configs. (which fixes elastic#3442) * Updates the short config files of the modules to include the path definitions. I think this is better for a "module first" experience, where it gives a bit of context on how they work. This means that the default configuration starts and gives no errors, but also doesn't publish anything. IMO, this behaviour is fine considering configuration reloading.
This does two changes: * Adds `enabled: false` in the default prospector in the short and long configs. (which fixes #3442) * Updates the short config files of the modules to include the path definitions. I think this is better for a "module first" experience, where it gives a bit of context on how they work. This means that the default configuration starts and gives no errors, but also doesn't publish anything. IMO, this behaviour is fine considering configuration reloading. * Print error in case all prospectors are disabled and config reloading is off Also, add the `auth` fileset to the config samples, it was missing.
Currently in filebeat, a fresh out of the box distribution has a default path of
/var/log/*.log
enabled:While this has the benefit/intention of helping making an out-of-the-box good experience for some logs, I would say the any benefits of this are outweighed by the negatives and would instead like to advocate for paths being all commented out by default. The rationale behind this is as follows:
The text was updated successfully, but these errors were encountered: