Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] Add module config reload docs #4930

Merged
merged 1 commit into from
Aug 18, 2017
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 74 additions & 15 deletions filebeat/docs/reload-configuration.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,34 +3,49 @@

beta[]

You can configure Filebeat to dynamically reload prospector configuration files
when there are changes. To do this, you specify a path
(https://golang.org/pkg/path/filepath/#Glob[Glob]) to watch for prospector
configuration changes. When the files found by the Glob change, new prospectors
are started/stopped according to changes in the configuration files.
You can configure Filebeat to dynamically reload configuration files when there
are changes. This feature is available for prospector and module configuration
files only.

To configure this feature, you specify a path
(https://golang.org/pkg/path/filepath/#Glob[Glob]) to watch for configuration
changes. When the files found by the Glob change, new prospectors and/or
modules are started and stopped according to changes in the configuration files.

This feature is especially useful in container environments where one container
is used to tail logs for services running in other containers on the same host.

To enable dynamic config reloading, you specify the `path` and `reload` options
in the `filebeat.config.prospectors` section of the main `filebeat.yml` config
file. For example:
To enable dynamic config reloading, you specify the following options under
`filebeat.config.prospectors` (for prospector configs) or
`filebeat.config.modules` (for module configs):

`path`:: A Glob that defines the files to check for changes.
`reload.enabled`:: When set to `true`, enables dynamic config reload.
`reload.period`:: Specifies how often the files are checked for changes. Do not
set the `period` to less than 1s because the modification time of files is often
stored in seconds. Setting the `period` to less than 1s will result in
unnecessary overhead.

See <<reload-prospector-config>> and <<reload-module-config>> for examples.


[float]
[[reload-prospector-config]]
=== Prospector config

For prospector configurations, you specify reload options in the
`filebeat.config.prospectors` section of the +{beatname_lc}.yml+ file. For
example:

[source,yaml]
------------------------------------------------------------------------------
filebeat.config.prospectors:
enabled: true
path: configs/*.yml
reload.enabled: true
reload.period: 10s
------------------------------------------------------------------------------

`path`:: A Glob that defines the files to check for changes.
`reload.enabled`:: When set to `true`, enables dynamic config reload.
`reload.period`:: Specifies how often the files are checked for changes. Do not
set the `period` to less than 1s because the modification time of files is often
stored in seconds. Setting the `period` to less than 1s will result in
unnecessary overhead.

Each file found by the Glob must contain a list of one or more prospector
definitions. For example:

Expand All @@ -47,6 +62,50 @@ definitions. For example:
scan_frequency: 5s
------------------------------------------------------------------------------


WARNING: It is critical that two running prospectors DO NOT have overlapping
file paths defined. If more than one prospector harvests the same file at the
same time, it can lead to unexpected behaviour.

[float]
[[reload-module-config]]
=== Module config

For module configurations, you specify reload options in the
`filebeat.config.modules` section of the +{beatname_lc}.yml+ file. For example:

[source,yaml]
------------------------------------------------------------------------------
filebeat.config.modules:
enabled: true
path: configs/*.yml
reload.enabled: true
reload.period: 10s
------------------------------------------------------------------------------

TIP: If you are using the <<modules-command,`modules`>> command to enable and
Copy link
Contributor Author

@dedemorton dedemorton Aug 17, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@exekias I'm not sure if I'm going far enough here. I think it might be better to show the modules.d path in the config example here rather than just mentioning it as an aside. WDYT? (still need to write those docs...tomorrow)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think so, we show a common path from doc, while it's clear you can change it if you want

disable configurations, you can enable config reloading for the `modules.d`
directory by specifying `path: ${path.config}/modules.d/*.yml`.

Each file found by the Glob must contain a list of one or more module
definitions. For example:

//REVIEWERS: I got an error when I tried to have an empty line before the -module: mysql entry below. Shouldn't whitespace be allowed there?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uhm I think it should be allowed so we may file a bug for this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird. I was not able to reproduce this issue. Maybe I had a tab or something on that line? Anyhow...if I can't reproduce the problem, it doesn't exist, right? :-)


[source,yaml]
------------------------------------------------------------------------------
- module: apache2
access:
enabled: true
var.paths: [/var/log/apache2/access.log*]
error:
enabled: true
var.paths: [/var/log/apache2/error.log*]
- module: mysql
error:
enabled: true
var.paths: [/var/log/mysql/error.log*]
slowlog:
enabled: true
var.paths: [/var/log/mysql/mysql-slow.log*]
------------------------------------------------------------------------------