From 59b2ad59929d63609d75d0fbe9cd2cc708002cf2 Mon Sep 17 00:00:00 2001 From: DeDe Morton Date: Fri, 9 Sep 2016 21:20:12 -0700 Subject: [PATCH] Add jump lists to container topics --- filebeat/docs/configuring-howto.asciidoc | 16 ++++-- filebeat/docs/faq.asciidoc | 46 +++++++++++++++- filebeat/docs/getting-started.asciidoc | 14 ++++- filebeat/docs/index.asciidoc | 4 +- filebeat/docs/migration.asciidoc | 20 +++++-- filebeat/docs/troubleshooting.asciidoc | 52 +++---------------- libbeat/docs/config-file-format.asciidoc | 14 +++-- libbeat/docs/event-conventions.asciidoc | 2 +- libbeat/docs/gettingstarted.asciidoc | 2 +- libbeat/docs/newbeat.asciidoc | 16 +++++- libbeat/docs/newdashboards.asciidoc | 8 +++ metricbeat/docs/configuring-howto.asciidoc | 15 ++++-- metricbeat/docs/developer-guide/faq.asciidoc | 1 + .../docs/developer-guide/index.asciidoc | 11 +++- metricbeat/docs/gettingstarted.asciidoc | 14 +++-- metricbeat/docs/how-metricbeat-works.asciidoc | 5 ++ metricbeat/docs/index.asciidoc | 4 +- .../docs/metricbeat-in-a-container.asciidoc | 4 ++ metricbeat/docs/troubleshooting.asciidoc | 8 ++- packetbeat/docs/configuring-howto.asciidoc | 18 +++++-- packetbeat/docs/filtering.asciidoc | 1 + packetbeat/docs/gettingstarted.asciidoc | 12 ++++- packetbeat/docs/new_protocol.asciidoc | 3 +- packetbeat/docs/thrift.asciidoc | 1 + packetbeat/docs/troubleshooting.asciidoc | 11 +++- .../docs/visualizing-data-packetbeat.asciidoc | 6 ++- winlogbeat/docs/configuring-howto.asciidoc | 12 +++-- winlogbeat/docs/getting-started.asciidoc | 14 ++++- winlogbeat/docs/troubleshooting.asciidoc | 7 ++- 29 files changed, 251 insertions(+), 90 deletions(-) diff --git a/filebeat/docs/configuring-howto.asciidoc b/filebeat/docs/configuring-howto.asciidoc index 5143365c388..c22a30e8937 100644 --- a/filebeat/docs/configuring-howto.asciidoc +++ b/filebeat/docs/configuring-howto.asciidoc @@ -7,10 +7,20 @@ After following the <> in the Getting Started, you might want to fine tune the behavior of Filebeat. This section describes some common use cases for changing configuration options. -To configure {beatname_uc}, you edit the configuration file. For rpm and deb, you’ll find the configuration file at -+/etc/{beatname_lc}/{beatname_lc}.yml+. For mac and win, look in the archive that you extracted. +To configure {beatname_uc}, you edit the configuration file. For rpm and deb, you’ll find the default configuration file at ++/etc/{beatname_lc}/{beatname_lc}.yml+. There's also a full example configuration file at ++/etc/{beatname_lc}/{beatname_lc}.full.yml+ that shows all non-deprecated options. For mac and win, look in the archive that you extracted. -For a complete description of all Filebeat configuration options, see <>. +The following topics describe how to configure Filebeat: + +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> -- diff --git a/filebeat/docs/faq.asciidoc b/filebeat/docs/faq.asciidoc index fdf55e3ed72..21a0d3ea6dd 100644 --- a/filebeat/docs/faq.asciidoc +++ b/filebeat/docs/faq.asciidoc @@ -31,6 +31,50 @@ it's publishing events successfully: ./filebeat -c config.yml -e -d "*" ---------------------------------------------------------------------- +[float] +[[open-file-handlers]] +== Too many open file handlers? + +Filebeat keeps the file handler open in case it reaches the end of a file to read new log lines in near real time. If filebeat is harvesting a large number of files, the number of open files can be become an issue. In most environments, the number of files which are actively updated is low. The configuration `close_inactive` should be set accordingly to close files which are not active any more. + +There are 4 more configuration options which can be used to close file handlers, but all of them should be used carefully as they can side affects. The options are: + +* close_renamed +* close_removed +* close_eof +* close_timeout + +`close_renamed` and `close_removed` can be useful on Windows and issues related to file rotation, see <>. `close_eof` can be useful in environments with a large number of files with only very few entries. `close_timeout` in environments where it is more important to close file handlers then to send all log lines. More details can be found in config options, see <>. + +Before using any of these variables, make sure to study the documentation on each. + +[float] +[[reduce-registry-size]] +=== Registry file is too large? + +Filebeat keeps all states of the files and persists the states on disk in the `registry_file`. The states are used to continue file reading at a previous position in case filebeat is restarted. In case every day a large amount of new files is constantly produced, the registry file grows over time. To reduce the size of the registry file, there are two configuration variables: `clean_removed` and `close_inactive`. + +In case old files are not touched anymore and fall under `ignore_older`, it is recommended to use `clean_inactive`. If on the other size old files get removed from disk `clean_removed` can be used. + +[float] +[[inode-reuse-issue]] +=== Inode reuse causes Filebeat to skip lines? + +Filebeat uses under linux inode and device to identify files. In case a file is removed from disk, the inode can again be assigned to a new file. In the case of file rotation where and old file is removed and a new one is directly created afterwards, it can happen that the new files has the exact same inode. In this case, Filebeat assumes that the new file is the same as the old and tries to continue reading at the old position which is not correct. + +By default states are never removed from the registry file. In case of inode reuse issue it is recommended to use the `clean_*` options, especially `clean_inactive`. In case your files get rotated every 24 hours and the rotated files rotated files are not updated anymore, `ignore_older` could be set to 48 hours and `clean_inactive` 72 hours. + +`clean_removed` can be used for files that are removed from disk. Be aware that `clean_removed` also applies if during one scan a file cannot be found anymore. In case the file shows up at a later stage again, it will be sent again from scratch. + +[float] +[[windows-file-rotation]] +=== Open file handlers cause issues with Windows file rotation? + +Under Windows it can happen, that files cannot be renamed / removed as long as filebeat keeps the file handler open. This can lead to issues with the file rotating system. To reduce this issue, the options `close_removed` and `close_renamed` can be used together. + +It is important to understand, that these two options mean files are closed before the harvester finished reading the file. In case the file cannot be picked up again by the prospector and the harvester didn't finish reading the file, the missing lines will never be sent to elasticsearch. + + [float] [[filebeat-cpu]] === Filebeat is using too much CPU? @@ -38,8 +82,6 @@ it's publishing events successfully: Filebeat might be configured to scan for files too frequently. Check the setting for `scan_frequency` in the `filebeat.yml` config file. Setting `scan_frequency` to less than 1s may cause Filebeat to scan the disk in a tight loop. -//=== Why is Filebeat keeping old files open? - [float] [[dashboard-fields-incorrect-filebeat]] === Dashboard in Kibana is breaking up data fields incorrectly? diff --git a/filebeat/docs/getting-started.asciidoc b/filebeat/docs/getting-started.asciidoc index 789dde80c1e..42d91f04c41 100644 --- a/filebeat/docs/getting-started.asciidoc +++ b/filebeat/docs/getting-started.asciidoc @@ -9,7 +9,17 @@ To get started with your own Filebeat setup, install and configure these related See {libbeat}/getting-started.html[Getting Started with Beats and the Elastic Stack] for more information. -After you have installed these products, you can start <>. +After installing the Elastic Stack, read the following topics to learn how to install, configure, and run Filebeat: + +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> + [[filebeat-installation]] === Step 1: Installing Filebeat @@ -147,6 +157,7 @@ include::../../libbeat/docs/shared-logstash-config.asciidoc[] :allplatforms: include::../../libbeat/docs/shared-template-load.asciidoc[] +[[filebeat-starting]] === Step 5: Starting Filebeat Start Filebeat by issuing the appropriate command for your platform. @@ -183,6 +194,7 @@ By default, Windows log files are stored in `C:\ProgramData\filebeat\Logs`. Filebeat is now ready to send log files to your defined output. +[[filebeat-index-pattern]] === Step 6: Loading the Kibana Index Pattern We don't offer prebuilt dashboards for visualizing Filebeat data. However, to make it easier for you to explore Filebeat diff --git a/filebeat/docs/index.asciidoc b/filebeat/docs/index.asciidoc index c9a6685e919..70eb8dbef76 100644 --- a/filebeat/docs/index.asciidoc +++ b/filebeat/docs/index.asciidoc @@ -18,12 +18,12 @@ include::./overview.asciidoc[] include::./getting-started.asciidoc[] -include::./how-filebeat-works.asciidoc[] - include::./command-line.asciidoc[] include::../../libbeat/docs/shared-directory-layout.asciidoc[] +include::./how-filebeat-works.asciidoc[] + include::./configuring-howto.asciidoc[] include::./filebeat-filtering.asciidoc[] diff --git a/filebeat/docs/migration.asciidoc b/filebeat/docs/migration.asciidoc index 99cdf216195..8f4b18ba679 100644 --- a/filebeat/docs/migration.asciidoc +++ b/filebeat/docs/migration.asciidoc @@ -4,9 +4,7 @@ [partintro] -- Filebeat is based on the Logstash Forwarder source code and replaces Logstash Forwarder as the method -to use for tailing log files and forwarding them to Logstash. This guide contains -information to help you migrate from -https://github.com/elastic/logstash-forwarder[Logstash Forwarder] to Filebeat. +to use for tailing log files and forwarding them to Logstash. Filebeat introduces the following major changes: @@ -17,8 +15,18 @@ Filebeat introduces the following major changes: * Configuration options for outputs are now inherited from libbeat. For details, see the {libbeat}/index.html[Beats Platform Reference]. * The https://www.elastic.co/guide/en/logstash/current/plugins-inputs-beats.html[Beats input plugin for Logstash] is required. +The following topics describe how to migrate from +https://github.com/elastic/logstash-forwarder[Logstash Forwarder] to Filebeat: + +* <> +* <> +* <> +* <> +* <> +* <> -- +[[migration-input-plugin]] == Migrating to the Beats Input Plugin for Logstash Filebeat requires the https://www.elastic.co/guide/en/logstash/current/plugins-inputs-beats.html[Beats input plugin for Logstash]. @@ -45,6 +53,7 @@ What's required? version 2.2.8 or later. * SSL must be explicitly enabled in the Beats input plugin (`ssl => true`) because SSL is on by default with Logstash Forwarder. The SSL/TLS configs should be the same for both the Logstash Forwarder and Filebeat instances. +[[migration-registry-file]] == Updating the Registry File The registry file stores the state and location information that Filebeat uses to track @@ -68,6 +77,7 @@ Filebeat configuration file to true. Using this approach allows you to keep the old Logstash Forwarder running and then slowly migrate over to Filebeat. +[[migration-configuration]] == Migrating Your Configuration Although Filebeat is based on Logstash Forwarder, Filebeat uses YAML for its configuration @@ -284,6 +294,7 @@ output.elasticsearch: hosts: ["http://localhost:5043"] ------------------------------------------------------------------------------------- +[[migration-changed-cli]] == Changes to Command Line Options Most command line options available in Logstash Forwarder have been removed and @@ -342,7 +353,7 @@ The `config_dir` option specifies the path to the directory that contains additi |=== - +[[migration-changed-fields]] == Changes to the Output Fields In the default configuration, Filebeat structures its output documents a little @@ -403,6 +414,7 @@ The `line` field was removed. This field wasn't correct after restarts, and making it correct would have resulted in a performance penalty. We recommend using the `offset` field instead. +[[migration-other-changes]] == Other Changes The following list of implementation changes should not affect your experience migrating diff --git a/filebeat/docs/troubleshooting.asciidoc b/filebeat/docs/troubleshooting.asciidoc index 1e43dcd5344..c925d521a6f 100644 --- a/filebeat/docs/troubleshooting.asciidoc +++ b/filebeat/docs/troubleshooting.asciidoc @@ -4,59 +4,19 @@ [partintro] -- If you have issues installing or running Filebeat, read the -following tips. +following tips: + +* <> +* <> +* <> -- +[[getting-help]] == Getting Help include::../../libbeat/docs/getting-help.asciidoc[] - -== Reduce open file handlers - -Filebeat keeps the file handler open in case it reaches the end of a file to read new log lines in near real time. If filebeat is harvesting a large number of files, the number of open files can be become an issue. In most environments, the number of files which are actively updated is low. The configuration `close_inactive` should be set accordingly to close files which are not active any more. - -There are 4 more configuration options which can be used to close file handlers, but all of them should be used carefully as they can side affects. The options are: - -* close_renamed -* close_removed -* close_eof -* close_timeout - -`close_renamed` and `close_removed` can be useful on Windows and issues related to file rotation, see <>. `close_eof` can be useful in environments with a large number of files with only very few entries. `close_timeout` in environments where it is more important to close file handlers then to send all log lines. More details can be found in config options, see <>. - -Before using any of these variables, make sure to study the documentation on each. - - -[[reduce-registry-size]] -== Reduce Registry File Size - -Filebeat keeps all states of the files and persists the states on disk in the `registry_file`. The states are used to continue file reading at a previous position in case filebeat is restarted. In case every day a large amount of new files is constantly produced, the registry file grows over time. To reduce the size of the registry file, there are two configuration variables: `clean_removed` and `close_inactive`. - -In case old files are not touched anymore and fall under `ignore_older`, it is recommended to use `clean_inactive`. If on the other size old files get removed from disk `clean_removed` can be used. - -[[inode-reuse-issue]] -== Inode Reuse Issue - -Filebeat uses under linux inode and device to identify files. In case a file is removed from disk, the inode can again be assigned to a new file. In the case of file rotation where and old file is removed and a new one is directly created afterwards, it can happen that the new files has the exact same inode. In this case, Filebeat assumes that the new file is the same as the old and tries to continue reading at the old position which is not correct. - -By default states are never removed from the registry file. In case of inode reuse issue it is recommended to use the `clean_*` options, especially `clean_inactive`. In case your files get rotated every 24 hours and the rotated files rotated files are not updated anymore, `ignore_older` could be set to 48 hours and `clean_inactive` 72 hours. - -`clean_removed` can be used for files that are removed from disk. Be aware that `clean_removed` also applies if during one scan a file cannot be found anymore. In case the file shows up at a later stage again, it will be sent again from scratch. - -[[windows-file-rotation]] -== Windows File Rotation - -Under Windows it can happen, that files cannot be renamed / removed as long as filebeat keeps the file handler open. This can lead to issues with the file rotating system. To reduce this issue, the options `close_removed` and `close_renamed` can be used together. - -It is important to understand, that these two options mean files are closed before the harvester finished reading the file. In case the file cannot be picked up again by the prospector and the harvester didn't finish reading the file, the missing lines will never be sent to elasticsearch. - - - - - - [[enable-filebeat-debugging]] == Debugging diff --git a/libbeat/docs/config-file-format.asciidoc b/libbeat/docs/config-file-format.asciidoc index 17b7610a52e..221d9771a24 100644 --- a/libbeat/docs/config-file-format.asciidoc +++ b/libbeat/docs/config-file-format.asciidoc @@ -4,7 +4,7 @@ Beats config files are based on http://www.yaml.org[YAML], a file format that is easier to read and write than other common data formats like XML or JSON. -In beats all YAML files start with an dictionary, an unordered collection of +In beats all YAML files start with a dictionary, an unordered collection of name/value pairs. In addition to dictionaries, YAML also supports lists, numbers, strings, and many other data types. All members of the same list or dictionary must have the same indentation level. @@ -52,6 +52,14 @@ person: \{name: "John Doe", age: 34, country: "Canada"} colors: ["Red", "Green", "Blue"] ------------------------------------------------------------------------------ +The following topics provide more detail to help you understand and work with config files in YAML: + +* <> +* <> +* <> +* <> +* <> +* <> [[config-file-format-namespacing]] === Namespacing @@ -154,7 +162,7 @@ style supports no escaping (use `''` to quote a single quote). Only printable characters can be used when using this form. Unquoted style requires no quotes, but does not support any escaping plus care -needs to be taken to not use any symbol that has a speacial meaning in YAML. +needs to be taken to not use any symbol that has a special meaning in YAML. Note: Single-quoted style is recommended when defining regular expressions, event format strings, windows file paths, or non-alphabetical symbolic characters. @@ -313,7 +321,7 @@ when using plain references. === Command line arguments Config files to load are set using the `-c` flag on command line. If no flag is -given, a beat and OS pecific default file path will be assumed. +given, a beat and OS-specific default file path will be assumed. You can specify multiple configuration files by repeating the `-c` flag. You can use this, for example, for setting defaults in a base configuration file, and diff --git a/libbeat/docs/event-conventions.asciidoc b/libbeat/docs/event-conventions.asciidoc index 0f538066f92..9de81256781 100644 --- a/libbeat/docs/event-conventions.asciidoc +++ b/libbeat/docs/event-conventions.asciidoc @@ -13,7 +13,7 @@ Use the following naming conventions for field names: - Group related fields into subdocuments by using dot (.) notation. Groups typically have common prefixes. For example, if you have fields called `CPULoad` and `CPUSystem` in a service, you would convert them into `cpu.load` and `cpu.system` in the event. - Avoid repeating the namespace in field names. If a word or abbreviation appears in the namespace, it's not needed in the field name. For example, instead of `cpu.cpu_load`, use `cpu.load`. -- Use <> and avoid using abbreviations that aren't commonly known. +- Use <> and avoid using abbreviations that aren't commonly known. - Organise the documents from general to specific to allow for namespacing. The type, such as `.pct`, should always be last. For example, `system.core.user.pct`. - If two fields are the same, but with different units, remove the less granular one. For example, include `timeout.sec`, but don't include `timeout.min`. If a less granular value is required, you can calculate it later. - If a field name matches the namespace used for nested fields, add `.value` to the field name. For example, instead of: diff --git a/libbeat/docs/gettingstarted.asciidoc b/libbeat/docs/gettingstarted.asciidoc index 83a8d1cc199..8617380fc55 100644 --- a/libbeat/docs/gettingstarted.asciidoc +++ b/libbeat/docs/gettingstarted.asciidoc @@ -7,10 +7,10 @@ the Elastic stack for Beats. A regular _Beats setup_ consists of: - * One or more Beats. You install the Beats on your servers to capture operational data. * Elasticsearch for storage and indexing. See <>. * Logstash (optional) for inserting data into Elasticsearch. See <>. * Kibana for the UI. See <>. + * One or more Beats. You install the Beats on your servers to capture operational data. See <>. * Kibana dashboards for visualizing the data. See <>. See the https://www.elastic.co/support/matrix[Elastic Support Matrix] for information diff --git a/libbeat/docs/newbeat.asciidoc b/libbeat/docs/newbeat.asciidoc index 9690d72b227..9d02abe4621 100644 --- a/libbeat/docs/newbeat.asciidoc +++ b/libbeat/docs/newbeat.asciidoc @@ -15,6 +15,18 @@ an example Beat called Countbeat. The Beat generator creates all the files requi for a working Beat. To create your own Beat, you modify the generated files and implement the custom logic needed to collect the data you want to ship. +The following topics describe how to build a new Beat: + +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> + [[newbeat-getting-ready]] === Getting Ready @@ -42,6 +54,7 @@ When you run the command, all source files are downloaded to the `$GOPATH/src/github.com/elastic/beats` path. You can ignore the "no buildable Go source files" message because you will build the source later. +[[newbeat-overview]] === Overview At the high level, a simple Beat has two main components: @@ -99,6 +112,7 @@ The following example shows an event object in Lsbeat: Now that you have the big picture, let's dig into the code. +[[newbeat-generate]] === Generating Your Beat To generate your own Beat, you use the Beat generator available in the beats repo on GitHub. If you haven't @@ -479,7 +493,7 @@ Lsbeat periodically ships them to Elasticsearch for storage. To help you learn how to build a Beat, we've created this http://elastic.co//blog/build-your-own-beat[blog post] that describes how to build Lsbeat from scratch. You can refer to the Lsbeat implementation as a working example. - +[[newbeat-sharing]] === Sharing Your Beat with the Community When you're done with your new Beat, how about letting everyone know? Open diff --git a/libbeat/docs/newdashboards.asciidoc b/libbeat/docs/newdashboards.asciidoc index 7e0849e3398..614d85aaa65 100644 --- a/libbeat/docs/newdashboards.asciidoc +++ b/libbeat/docs/newdashboards.asciidoc @@ -18,6 +18,14 @@ recommend that you use the virtual environment under https://github.com/elastic/beats/tree/master/testing/environments[beats/testing/environments] to import, create, and export the Kibana dashboards. +The following topics provide more detail about importing and working with Beats dashboards: + +* <> +* <> +* <> +* <> +* <> + [[import-dashboards]] === Importing Existing Beat Dashboards diff --git a/metricbeat/docs/configuring-howto.asciidoc b/metricbeat/docs/configuring-howto.asciidoc index 10ef1d265d8..3cd8c7b6ba0 100644 --- a/metricbeat/docs/configuring-howto.asciidoc +++ b/metricbeat/docs/configuring-howto.asciidoc @@ -8,10 +8,19 @@ Getting Started, you might want to fine tune the behavior of {beatname_uc}. This section describes some common use cases for changing configuration options. To configure {beatname_uc}, you edit the configuration file. For rpm and deb, you’ll find the configuration file at -+/etc/{beatname_lc}/{beatname_lc}.yml+. For mac and win, look in the archive that you extracted. ++/etc/{beatname_lc}/{beatname_lc}.yml+. There's also a full example configuration file at ++/etc/{beatname_lc}/{beatname_lc}.full.yml+ that shows all non-deprecated options. +For mac and win, look in the archive that you extracted. -For a complete description of all {beatname_uc} configuration options, see -<>. +The following topics describe how to configure Metricbeat: + +* <> +* <> +* <> +* <> +* <> +* <> +* <> [float] == Basic Configuration diff --git a/metricbeat/docs/developer-guide/faq.asciidoc b/metricbeat/docs/developer-guide/faq.asciidoc index 51886e09a89..82627427fc3 100644 --- a/metricbeat/docs/developer-guide/faq.asciidoc +++ b/metricbeat/docs/developer-guide/faq.asciidoc @@ -1,3 +1,4 @@ +[[dev-faq]] == Metricbeat Developer Guide FAQ This is a list of common questions when creating a metricset and the potential answers. diff --git a/metricbeat/docs/developer-guide/index.asciidoc b/metricbeat/docs/developer-guide/index.asciidoc index 2d2179e60bd..0bc5315ed68 100644 --- a/metricbeat/docs/developer-guide/index.asciidoc +++ b/metricbeat/docs/developer-guide/index.asciidoc @@ -15,12 +15,21 @@ We recommend that you start by creating your own Beat to keep the development of independent of Metricbeat. At a later stage, if you decide to add a module to Metricbeat, you can reuse the code without making additional changes. +This following topics describe how to contribute to Metricbeat by adding metricsets, modules, and new Beats based on Metricbeat: + +* <> +* <> +* <> +* <> +* <> +* <> + If you would like to contribute to Metricbeat or the Beats project, also see the https://github.com/elastic/beats/blob/master/CONTRIBUTING.md[general contributor guide]. -- - +[[metricbeat-dev-overview]] == Overview Metricbeat consists of modules and metricsets. A Metricbeat module is typically diff --git a/metricbeat/docs/gettingstarted.asciidoc b/metricbeat/docs/gettingstarted.asciidoc index 8005e66d1fd..556047f5cf0 100644 --- a/metricbeat/docs/gettingstarted.asciidoc +++ b/metricbeat/docs/gettingstarted.asciidoc @@ -11,12 +11,17 @@ related products: * Kibana for the UI. * Logstash (optional) for inserting data into Elasticsearch. -See {libbeat}/getting-started.html[Getting Started with Beats and the Elastic -Stack] for more information. +See {libbeat}/getting-started.html[Getting Started with Beats and the Elastic Stack] for more information. -After you have installed these products, you can start -<>. +After installing the Elastic Stack, read the following topics to learn how to install, configure, and run Metricbeat: +* <> +* <> +* <> +* <> +* <> +* <> +* <> [[metricbeat-installation]] === Step 1: Installing Metricbeat @@ -170,6 +175,7 @@ instead. :allplatforms: include::../../libbeat/docs/shared-template-load.asciidoc[] +[[metricbeat-starting]] === Step 4: Starting Metricbeat Run Metricbeat by issuing the following command: diff --git a/metricbeat/docs/how-metricbeat-works.asciidoc b/metricbeat/docs/how-metricbeat-works.asciidoc index 76047d43e83..2fc920f29c3 100644 --- a/metricbeat/docs/how-metricbeat-works.asciidoc +++ b/metricbeat/docs/how-metricbeat-works.asciidoc @@ -34,6 +34,11 @@ system), it sends an event error to the specified output. This means that Metric always sends an event, even when there is a failure. This allows you to monitor for errors and see debug messages to help you diagnose what went wrong. +The following topics provide more detail about the structure of Metricbeat events: + +* <> +* <> + For more about the benefits of using Metricbeat, see <>. [[metricbeat-event-structure]] diff --git a/metricbeat/docs/index.asciidoc b/metricbeat/docs/index.asciidoc index d0e52904d2a..ef2ce915791 100644 --- a/metricbeat/docs/index.asciidoc +++ b/metricbeat/docs/index.asciidoc @@ -21,6 +21,8 @@ include::../../libbeat/docs/shared-directory-layout.asciidoc[] include::./how-metricbeat-works.asciidoc[] +include::./metricbeat-in-a-container.asciidoc[] + include::./configuring-howto.asciidoc[] include::./metricbeat-filtering.asciidoc[] @@ -35,8 +37,6 @@ include::../../libbeat/docs/yaml.asciidoc[] include::../../libbeat/docs/regexp.asciidoc[] -include::./metricbeat-in-a-container.asciidoc[] - include::./modules.asciidoc[] include::./fields.asciidoc[] diff --git a/metricbeat/docs/metricbeat-in-a-container.asciidoc b/metricbeat/docs/metricbeat-in-a-container.asciidoc index bc16cd47e2d..222fc21e9a8 100644 --- a/metricbeat/docs/metricbeat-in-a-container.asciidoc +++ b/metricbeat/docs/metricbeat-in-a-container.asciidoc @@ -1,3 +1,4 @@ +[[running-in-container]] == Running Metricbeat in a Container Elastic does not provide any official container images for Metricbeat. The @@ -9,6 +10,7 @@ containers. Let's walk-through some examples using Docker as our container orchestration tool. [float] +[[monitoring-host]] === Monitoring the Host Machine This first example highlights the changes required to make the system module @@ -46,6 +48,7 @@ flag. This is due to Linux namespacing, simply bind mounting the host's `/proc` to `/hostfs/proc` is not sufficient. [float] +[[monitoring-service]] === Monitoring a Service in Another Container Next let's look at an example of monitoring a containerized service from a @@ -80,3 +83,4 @@ metricbeat.modules: <1> The `mysql` hostname will resolve to the `some-mysql` container's address. <2> The `MYSQL_PASSWORD` variable will be evaluated at startup. If a variable is not set this will lead to an error at startup. + diff --git a/metricbeat/docs/troubleshooting.asciidoc b/metricbeat/docs/troubleshooting.asciidoc index 061a701cd7b..c8d694e6228 100644 --- a/metricbeat/docs/troubleshooting.asciidoc +++ b/metricbeat/docs/troubleshooting.asciidoc @@ -3,11 +3,15 @@ [partintro] -- -If you have issues installing or running {beatname_uc}, read the following tips. +If you have issues installing or running {beatname_uc}, read the following tips: + +* <> +* <> +* <> -- -[[getting-support]] +[[getting-help]] == Getting Help include::../../libbeat/docs/getting-help.asciidoc[] diff --git a/packetbeat/docs/configuring-howto.asciidoc b/packetbeat/docs/configuring-howto.asciidoc index 515f12e9fd1..3a0086a1ff3 100644 --- a/packetbeat/docs/configuring-howto.asciidoc +++ b/packetbeat/docs/configuring-howto.asciidoc @@ -7,10 +7,22 @@ After following the <> in the Getting Started, you might want to fine tune the behavior of Packetbeat. This section describes some common use cases for changing configuration options. -To configure {beatname_uc}, you edit the configuration file. For rpm and deb, you’ll find the configuration file at -+/etc/{beatname_lc}/{beatname_lc}.yml+. For mac and win, look in the archive that you extracted. +To configure {beatname_uc}, you edit the configuration file. For rpm and deb, you’ll find the default configuration file at ++/etc/{beatname_lc}/{beatname_lc}.yml+. There's also a full example configuration file at ++/etc/{beatname_lc}/{beatname_lc}.full.yml+ that shows all non-deprecated options. For mac and win, look in the archive +that you extracted. -For a complete description of all Packetbeat configuration options, see <>. +The following topics describe how to configure Packetbeat: + +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> -- diff --git a/packetbeat/docs/filtering.asciidoc b/packetbeat/docs/filtering.asciidoc index bb557cbba7a..d2643866313 100644 --- a/packetbeat/docs/filtering.asciidoc +++ b/packetbeat/docs/filtering.asciidoc @@ -1,3 +1,4 @@ +[[kibana-queries-filters]] == Kibana Queries and Filters In Kibana, you can filter transactions either by entering a search query or by clicking on elements within a visualization. diff --git a/packetbeat/docs/gettingstarted.asciidoc b/packetbeat/docs/gettingstarted.asciidoc index f2f455b3027..faf816cdfe6 100644 --- a/packetbeat/docs/gettingstarted.asciidoc +++ b/packetbeat/docs/gettingstarted.asciidoc @@ -12,7 +12,15 @@ To get started with your own Packetbeat setup, install and configure these relat See {libbeat}/getting-started.html[Getting Started with Beats and the Elastic Stack] for more information. -After you have installed these products, you can start <>. +After installing the Elastic Stack, read the following topics to learn how to install, configure, and run Packetbeat: + +* <> +* <> +* <> +* <> +* <> +* <> +* <> [[packetbeat-installation]] === Step 1: Installing Packetbeat @@ -190,6 +198,7 @@ TIP: To test your configuration file, run Packetbeat in the foreground with the :allplatforms: include::../../libbeat/docs/shared-template-load.asciidoc[] +[[packetbeat-starting]] === Step 4: Starting Packetbeat Run Packetbeat by issuing the following command: @@ -245,6 +254,7 @@ curl -XGET 'http://localhost:9200/packetbeat-*/_search?pretty' Make sure that you replace `localhost:9200` with the address of your Elasticsearch instance. The command should return data about the HTTP transaction you just created. +[[packetbeat-sample-dashboards]] === Step 5: Loading Sample Kibana Dashboards To make it easier for you to get application performance insights diff --git a/packetbeat/docs/new_protocol.asciidoc b/packetbeat/docs/new_protocol.asciidoc index e7da06f7ecc..f2abdf000cd 100644 --- a/packetbeat/docs/new_protocol.asciidoc +++ b/packetbeat/docs/new_protocol.asciidoc @@ -4,8 +4,7 @@ [partintro] -- -The following topics walk you through the steps needed to add a new protocol to -Packetbeat: +The following topics describe how to add a new protocol to Packetbeat: * <> * <> diff --git a/packetbeat/docs/thrift.asciidoc b/packetbeat/docs/thrift.asciidoc index e06933cc288..a4f07da9aba 100644 --- a/packetbeat/docs/thrift.asciidoc +++ b/packetbeat/docs/thrift.asciidoc @@ -1,3 +1,4 @@ +[[thrift-rpc]] == Configuring Thrift-RPC Support https://thrift.apache.org/[Apache Thrift] is a communication protocol and RPC diff --git a/packetbeat/docs/troubleshooting.asciidoc b/packetbeat/docs/troubleshooting.asciidoc index 2129c0d32fb..2dbb7cee97b 100644 --- a/packetbeat/docs/troubleshooting.asciidoc +++ b/packetbeat/docs/troubleshooting.asciidoc @@ -4,10 +4,18 @@ [partintro] -- If you have issues installing or running Packetbeat, read the -following tips. +following tips: + +* <> +* <> +* <> +* <> + + -- +[[getting-help]] == Getting Help include::../../libbeat/docs/getting-help.asciidoc[] @@ -17,6 +25,7 @@ include::../../libbeat/docs/getting-help.asciidoc[] include::../../libbeat/docs/debugging.asciidoc[] +[[recording-trace]] == Recording a Trace If you are having an issue, it's often useful to record a full network trace diff --git a/packetbeat/docs/visualizing-data-packetbeat.asciidoc b/packetbeat/docs/visualizing-data-packetbeat.asciidoc index 3a258a389d8..197de16f805 100644 --- a/packetbeat/docs/visualizing-data-packetbeat.asciidoc +++ b/packetbeat/docs/visualizing-data-packetbeat.asciidoc @@ -4,10 +4,14 @@ [partintro] -- Before trying to visualize Packetbeat data in Kibana, we recommend that you load the sample dashboards {libbeat}/visualizing-data.html[described here]. Then read the topics in this -section to learn how to work with Packetbeat data in Kibana. +section to learn how to work with Packetbeat data in Kibana: + +* <> +* <> -- +[[customizing-discover]] == Customizing the Discover Page To make it easier for you to search and discover Packetbeat data in Kibana, the sample dashboards contain diff --git a/winlogbeat/docs/configuring-howto.asciidoc b/winlogbeat/docs/configuring-howto.asciidoc index 20c11074f3b..fb8dd4f0b91 100644 --- a/winlogbeat/docs/configuring-howto.asciidoc +++ b/winlogbeat/docs/configuring-howto.asciidoc @@ -8,10 +8,16 @@ Getting Started, you might want to fine tune the behavior of Winlogbeat. This section describes some common use cases for changing configuration options. To configure {beatname_uc}, you edit the configuration file. You’ll find the configuration file, -+{beatname_lc}.yml+, in the archive that you extracted. ++{beatname_lc}.yml+, in the archive that you extracted. There's also a full example configuration file at ++/etc/{beatname_lc}/{beatname_lc}.full.yml+ that shows all non-deprecated options. -For a complete description of all Winlogbeat configuration options, see -<>. +The following topics describe how to configure Winlogbeat: + +* <> +* <> +* <> +* <> +* <> -- diff --git a/winlogbeat/docs/getting-started.asciidoc b/winlogbeat/docs/getting-started.asciidoc index 9a27a239c05..c0cd2b3124e 100644 --- a/winlogbeat/docs/getting-started.asciidoc +++ b/winlogbeat/docs/getting-started.asciidoc @@ -11,8 +11,16 @@ related products: See {libbeat}/getting-started.html[Getting Started with Beats and the Elastic Stack] for more information. -After you have installed these products, you can start -<>. +After installing the Elastic Stack, read the following topics to learn how to install, configure, and run Winlogbeat: + +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> [[winlogbeat-installation]] === Step 1: Installing Winlogbeat @@ -120,6 +128,7 @@ include::../../libbeat/docs/shared-logstash-config.asciidoc[] include::../../libbeat/docs/shared-template-load.asciidoc[] +[[winlogbeat-starting]] === Step 5: Starting Winlogbeat Start the Winlogbeat service with the following command: @@ -150,6 +159,7 @@ Stop the Winlogbeat service with the following command: PS C:\Program Files\Winlogbeat> Stop-Service winlogbeat ---------------------------------------------------------------------- +[[winlogbeat-sample-dashboards]] === Step 6: Loading Sample Kibana Dashboards To make it easier for you to start monitoring your servers in Kibana, diff --git a/winlogbeat/docs/troubleshooting.asciidoc b/winlogbeat/docs/troubleshooting.asciidoc index 2e616bdea17..cf3807f0c79 100644 --- a/winlogbeat/docs/troubleshooting.asciidoc +++ b/winlogbeat/docs/troubleshooting.asciidoc @@ -3,10 +3,15 @@ [partintro] -- -If you have issues installing or running Winlogbeat, read the following tips. +If you have issues installing or running Winlogbeat, read the following tips: + +* <> +* <> +* <> -- +[[getting-help]] == Getting Help include::../../libbeat/docs/getting-help.asciidoc[]