Skip to content

Commit

Permalink
Fix intro language about processors
Browse files Browse the repository at this point in the history
  • Loading branch information
dedemorton committed Dec 6, 2016
1 parent a8862b8 commit e4c041f
Show file tree
Hide file tree
Showing 2 changed files with 34 additions and 18 deletions.
35 changes: 24 additions & 11 deletions filebeat/docs/filebeat-filtering.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,31 @@
== Filtering and Enhancing the Exported Data

Your use case might require only a subset of the data exported by Filebeat, or
you might need to add metadata. Filebeat provides a couple of options for
filtering and enhancing exported data. You can:
you might need to enhance the exported data (for example, by adding metadata).
Filebeat provides a couple of options for filtering and enhancing exported
data. You can:

* <<filebeat-filtering-overview,Use simple filtering based on pattern matching>>
* <<defining-processors,Define processors for more complex processing requirements>>
* <<filebeat-filtering-overview,Define filters at the prospector level>> to
configure each prospector to include or exclude specific lines or files.
* <<defining-processors,Define processors>> to configure global processing
across all data exported by Filebeat.

[float]
[[filebeat-filtering-overview]]
=== Filtering Based on Pattern Matching
=== Filtering at the Prospector Level

You can specify configuration options in the `filebeat` section of the config file to define regular expressions that
match the lines you want to include and/or exclude from the output. The supported options are <<include-lines,`include_lines`>>, <<exclude-lines,`exclude_lines`>>, and <<exclude-files,`exclude_files`>>.
You can specify filtering options at the prospector level to configure which
lines or files are included or excluded in the output. This allows you to
specify different filtering criteria for each prospector.

For example, you can use the `include_lines` option to export any lines that start with "ERR" or "WARN":
You configure prospector-level filtering in the `filebeat.prospectors` section
of the config file by specifying regular expressions that match the lines you
want to include and/or exclude from the output. The supported options are
<<include-lines,`include_lines`>>, <<exclude-lines,`exclude_lines`>>, and
<<exclude-files,`exclude_files`>>.

For example, you can use the `include_lines` option to export any lines that
start with "ERR" or "WARN":

[source,yaml]
-------------------------------------------------------------------------------------
Expand All @@ -26,9 +37,11 @@ filebeat.prospectors:
include_lines: ["^ERR", "^WARN"]
-------------------------------------------------------------------------------------

The disadvantage of this approach is that you need to implement a configuration option for each filtering criteria that you need.
The disadvantage of this approach is that you need to implement a
configuration option for each filtering criteria that you need.

See <<configuration-filebeat-options,Filebeat configuration options>> for more information about each option.
See <<configuration-filebeat-options,Filebeat configuration options>> for more
information about each option.

[float]
[[defining-processors]]
Expand Down Expand Up @@ -85,7 +98,7 @@ filebeat.prospectors:
processors:
- decode_json_fields:
fields: ["inner"]
fields: ['inner']
output.console.pretty: true
-----------------------------------------------------
Expand Down
17 changes: 10 additions & 7 deletions libbeat/docs/processors.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,18 @@
//// include::../../libbeat/docs/filtering.asciidoc[]
//////////////////////////////////////////////////////////////////////////

You can define processors in your configuration to process events before they are sent to the configured output.
The libbeat library provides processors for reducing the number of exported fields, and processors for
enhancing events with additional metadata. Each processor receives an event, applies a defined action to the event,
and returns the event. If you define a list of processors, they are executed in the order they are defined in the
configuration file.
You can define processors in your configuration to process events before they
are sent to the configured output.The libbeat library provides processors for:

* reducing the number of exported fields
* enhancing events with additional metadata
* performing additional processing and decoding
Each processor receives an event, applies a defined action to the event, and
returns the event. If you define a list of processors, they are executed in the
order they are defined in the {beatname_uc} configuration file.

[source,yaml]
-------
event -> processor 1 -> event1 -> processor 2 -> event2 ...
-------

The processors are defined in the {beatname_uc} configuration file.

0 comments on commit e4c041f

Please sign in to comment.