Merge pull request #4074 from dedemorton/cherrypick_docs_4.20.17

Backport: #4016, #4009, #4017, and #3890 into 5.3

filebeat/docs/reference/configuration/reload-configuration.asciidoc

@@ -3,11 +3,17 @@
 
 beta[]
 
-Reload configuration allows to dynamically reload prospector configuration files. A glob can be defined which should be watched
- for prospector configuration changes. New prospectors will be started / stopped accordingly. This is especially useful in
- container environments where 1 container is used to tail logs from services in other containers on the same host.
+You can configure Filebeat to dynamically reload prospector configuration files
+when there are changes. To do this, you specify a path 
+(https://golang.org/pkg/path/filepath/#Glob[Glob]) to watch for prospector
+configuration changes. When the files found by the Glob change, new prospectors
+are started/stopped according to changes in the configuration files.
 
-The configuration in the main filebeat.yml config file looks as following:
+This feature is especially useful in container environments where one container
+is used to tail logs for services running in other containers on the same host.
+
+To enable dynamic config reloading, you specify the `path` and `reload` options
+in the main `filebeat.yml` config file. For example: 
 
 [source,yaml]
 ------------------------------------------------------------------------------
@@ -17,11 +23,16 @@ filebeat.config.prospectors:
   reload.period: 10s
 ------------------------------------------------------------------------------
 
-A path with a glob must be defined on which files should be checked for changes. A period is set on how often
-the files are checked for changes. Do not set period below 1s as the modification time of files is often stored in seconds.
-Setting it below 1s will cause an unnecessary overhead.
+`path`:: A Glob that defines the files to check for changes. 
+`reload.enabled`:: When set to `true`, enables dynamic config reload. 
+`reload.period`:: Specifies how often the files are checked for changes. Do not
+set the `period` to less than 1s because the modification time of files is often
+stored in seconds. Setting the `period` to less than 1s will result in
+unnecessary overhead.
+
+Each file found by the Glob must contain a list of one or more prospector
+definitions. For example: 
 
-The configuration inside the files which are found by the glob look as following:
 [source,yaml]
 ------------------------------------------------------------------------------
 - input_type: log
@@ -35,7 +46,6 @@ The configuration inside the files which are found by the glob look as following
   scan_frequency: 5s
 ------------------------------------------------------------------------------
 
-Each file directly contains a list of prospectors. Each file can contain one or multiple prospector definitions.
-
-WARNING: It is critical that two running prospectors DO NOT have overlapping file paths defined. If more then one prospector
-harvests the same file at the same time, it can lead to unexpected behaviour.
+WARNING: It is critical that two running prospectors DO NOT have overlapping
+file paths defined. If more than one prospector harvests the same file at the
+same time, it can lead to unexpected behaviour.

libbeat/docs/communitybeats.asciidoc

@@ -32,6 +32,7 @@ https://github.com/icinga/icingabeat[icingabeat]:: Icingabeat ships events and s
 https://github.com/devopsmakers/iobeat[iobeat]:: Reads IO stats from /proc/diskstats on Linux.
 https://github.com/radoondas/jmxproxybeat[jmxproxybeat]:: Reads Tomcat JMX metrics exposed over 'JMX Proxy Servlet' to HTTP.
 https://github.com/mheese/journalbeat[journalbeat]:: Used for log shipping from systemd/journald based Linux systems.
+https://github.com/dearcode/kafkabeat[kafkabeat]:: read data from kafka with Consumer-groups. 
 https://github.com/eskibars/lmsensorsbeat[lmsensorsbeat]:: Collects data from lm-sensors (such as CPU temperatures, fan speeds, and voltages from i2c and smbus).
 https://github.com/consulthys/logstashbeat[logstashbeat]:: Collects data from Logstash monitoring API (v5 onwards) and indexes them in Elasticsearch.
 https://github.com/yedamao/mcqbeat[mcqbeat]:: Reads the status of queues from memcacheq.
@@ -59,7 +60,7 @@ https://github.com/hartfordfive/udplogbeat[udplogbeat]:: Accept events via local
 https://github.com/cleesmith/unifiedbeat[unifiedbeat]:: Reads records from Unified2 binary files generated by
 network intrusion detection software and indexes the records in Elasticsearch.
 https://github.com/mrkschan/uwsgibeat[uwsgibeat]:: Reads stats from uWSGI.
-https://github.com/eskibars/wmibeat[wmibeat]:: Uses WMI to grab your favorite, configurable Windows metrics.
+https://github.com/eskibars/wmibeat[wmibeat]:: Uses WMI to grab your favorite, configurable Windows metrics. 
 
 
 Have you created a Beat that's not listed? If so, add the name and description of your Beat to the source document for

libbeat/docs/dashboardsconfig.asciidoc

@@ -16,17 +16,19 @@
 beta[]
 
 The `dashboards` section of the +{beatname_lc}.yml+ config file contains options
-for the automatic loading of the sample Beats dashboards. The loading of the
-dashboards is disabled by default, but can be enabled either from the configuration
+for automatically loading the sample Beats dashboards. Automatic dashboard
+loading is disabled by default, but can be enabled either from the configuration
 file or by using the `-setup` CLI flag.
 
 If dashboard loading is enabled, {beatname_uc} attempts to configure Kibana by
 writing directly in the Elasticsearch index for the Kibana configuration (by
-default, `.kibana`). To connect to Elasticsearch, it uses the settings defined
-in the Eleasticsearch output. If the Elasticsearch output is not configured or
-not enabled, {beatname_uc} will stop with an error. Loading the dashboards is
-only attempted at the Beat start, if Elasticsearch is not available when the
-Beat starts, {beatname_uc} will stop with an error.
+default, `.kibana`). To connect to Elasticsearch, {beatname_uc} uses the
+settings defined in the Elasticsearch output. If the Elasticsearch output is
+not configured or not enabled, {beatname_uc} will stop with an error. Dashboard
+loading is only attempted at Beat startup. If Elasticsearch is not available when
+the Beat starts, {beatname_uc} will stop with an error.
+
+Here is an example configuration:
 
 [source,yaml]
 ------------------------------------------------------------------------------
@@ -40,48 +42,64 @@ You can specify the following options in the `dashboards` section of the
 
 ===== enabled
 
-If enabled, load the sample Kibana dashboards on startup. If no other options
-are set, the dashboards archive is downloaded from the elastic.co website.
+If this option is set to true, {beatname_uc} loads the sample Kibana dashboards
+automatically on startup. If no other options are set, the dashboard archive is
+downloaded from the elastic.co website.
+
+To load dashboards from a different location, you can
+configure one of the following options: <<url-option,`url`>>,
+<<directory-option,`directory`>>, or <<file-option,`file`>>.
+
+To load dashboards from a snapshot URL, use the <<snapshot-option,`snapshot`>>
+option and optionally <<snapshot-url-option,`snapshot-url`>>.
 
+[[url-option]]
 ===== url
 
-The URL from where to download the dashboards archive. By default this URL has a
-value which is computed based on the Beat name and version. For released
-versions, this URL points to the dashboard archive on the artifacts.elastic.co
+The URL to use for downloading the dashboard archive. By default this URL
+is computed based on the Beat name and version. For released versions,
+this URL points to the dashboard archive on the artifacts.elastic.co
 website.
 
+[[directory-option]]
 ===== directory
 
-The directory from where to read the dashboards. It is used instead of the URL
-when it has a value.
+The directory that contains the dashboards to load. If this option is set,
+{beatname_uc} looks for dashboards in the specified directory instead of
+downloading an archive from a URL. 
 
+[[file-option]]
 ===== file
 
-The file archive (zip file) from where to read the dashboards. It is used
-instead of the URL when it has a value.
+The file archive (zip file) that contains the dashboards to load. If this option
+is set, {beatname_uc} looks for a dashboard archive in the specified path
+instead of downloading the archive from a URL. 
 
+[[snapshot-option]]
 ===== snapshot
 
 If this option is set to true, the snapshot URL is used instead of the default
 URL.
 
+[[snapshot-url-option]]
 ===== snapshot_url
 
-The URL from where to download the snapshot version of the dashboards. By
-default this has a value which is computed based on the Beat name and version.
+The URL to use for downloading the snapshot version of the dashboards. By
+default the snapshot URL is computed based on the Beat name and version.
 
 ===== beat
 
-In case the archive contains the dashboards from multiple Beats, this lets you
-select which one to load. You can load all the dashboards in the archive by
-setting this to the empty string. The default is "{beatname_lc}".
+In case the archive contains the dashboards for multiple Beats, this setting
+lets you select the Beat for which you want to load dashboards. To load all the
+dashboards in the archive, set this option to an empty string. The default is
++"{beatname_lc}"+.
 
 ===== kibana_index
 
-The name of the Kibana index to use for setting the configuration. Default is
-".kibana"
+The name of the Kibana index to use for setting the configuration. The default
+is `".kibana"`
 
 ===== index
 
-The Elasticsearch index name. This overwrites the index name defined in the
-dashboards and index pattern. Example: "testbeat-*"
+The Elasticsearch index name. This setting overwrites the index name defined
+in the dashboards and index pattern. Example: `"testbeat-*"`

metricbeat/docs/developer-guide/creating-beat-from-metricbeat.asciidoc

@@ -8,7 +8,7 @@ own metricsets.
 === Requirements
 
 To create your own Beat, you must have Golang {go-version} or later installed, and the `$GOPATH`
-must be set up correctly. In addition, the following tools are quired:
+must be set up correctly. In addition, the following tools are required:
 
 * https://www.python.org/downloads/[python]
 * https://virtualenv.pypa.io/en/stable/[virtualenv]

metricbeat/docs/reference/configuration/reload-configuration.asciidoc

@@ -1,14 +1,22 @@
+[[metricbeat-configuration-reloading]]
 === Reload Configuration
 
 experimental[]
 
-Reload configuration allows to dynamically reload module configuration files. A glob can be defined which should be watched
- for module configuration changes. New modules will started / stopped accordingly. This is especially useful in
- container environments where 1 container is used to monitor all services in other containers on the same host.
- New containers appear and disappear dynamically which requires changes to which modules
- are needed and which hosts must be monitored.
+You can configure Metricbeat to dynamically reload configuration files when
+there are changes. To do this, you specify a path 
+(https://golang.org/pkg/path/filepath/#Glob[Glob]) to watch for module
+configuration changes. When the files found by the Glob change, new modules are
+started/stopped according to changes in the configuration files.
 
-The configuration in the main metricbeat.yml config file looks as following:
+This feature is especially useful in container environments where one container
+is used to monitor all services running in other containers on the same host.
+Because new containers appear and disappear dynamically, you may need to change
+the Metricbeat configuration frequently to specify which modules are needed and
+which hosts must be monitored. 
+
+To enable dynamic config reloading, you specify the `path` and `reload` options
+in the main `metricbeat.yml` config file. For example: 
 
 [source,yaml]
 ------------------------------------------------------------------------------
@@ -18,11 +26,16 @@ metricbeat.config.modules:
   reload.period: 10s
 ------------------------------------------------------------------------------
 
-A path with a glob must be defined on which files should be checked for changes. A period is set on how often
-the files are checked for changes. Do not set period below 1s as the modification time of files is often stored in seconds.
-Setting it below 1s will have an unnecessary overhead.
+`path`:: A Glob that defines the files to check for changes. 
+`reload.enabled`:: When set to `true`, enables dynamic config reload. 
+`reload.period`:: Specifies how often the files are checked for changes. Do not
+set the `period` to less than 1s because the modification time of files is often
+stored in seconds. Setting the `period` to less than 1s will result in
+unnecessary overhead.
+
+Each file found by the Glob must contain a list of one or more module
+definitions. For example: 
 
-The configuration inside the files which are found by the glob look as following:
 [source,yaml]
 ------------------------------------------------------------------------------
 - module: system
@@ -35,5 +48,3 @@ The configuration inside the files which are found by the glob look as following
   enabled: true
   period: 10s
 ------------------------------------------------------------------------------
-
-Each file directly contains a list of modules. Each file can contain one or multiple module definitions.