Skip to content

Commit

Permalink
Add module usage info
Browse files Browse the repository at this point in the history
Fixes #7826
  • Loading branch information
dedemorton committed Jul 28, 2017
1 parent c967bc5 commit 15b75e3
Show file tree
Hide file tree
Showing 2 changed files with 130 additions and 5 deletions.
14 changes: 9 additions & 5 deletions docs/index-shared2.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,15 @@ include::static/managing-multiline-events.asciidoc[]
:edit_url: https://github.com/elastic/logstash/edit/master/docs/static/glob-support.asciidoc
include::static/glob-support.asciidoc[]

// Working with Logstash Modules

include::static/modules.asciidoc[]

// Working with Filebeat Modules

:edit_url: https://github.com/elastic/logstash/edit/master/docs/static/filebeat-modules.asciidoc
include::static/filebeat-modules.asciidoc[]

// Data resiliency

:edit_url: https://github.com/elastic/logstash/edit/master/docs/static/resiliency.asciidoc
Expand All @@ -37,11 +46,6 @@ include::static/persistent-queues.asciidoc[]
:edit_url: https://github.com/elastic/logstash/edit/master/docs/static/dead-letter-queues.asciidoc
include::static/dead-letter-queues.asciidoc[]

// Working with Filebeat Modules

:edit_url: https://github.com/elastic/logstash/edit/master/docs/static/filebeat-modules.asciidoc
include::static/filebeat-modules.asciidoc[]

// Transforming Data

:edit_url: https://github.com/elastic/logstash/edit/master/docs/static/transforming-data.asciidoc
Expand Down
121 changes: 121 additions & 0 deletions docs/static/modules.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
[[logstash-modules]]
== Working with Logstash Modules

Logstash modules provide a quick way to get started using Logstash.

Each module comes pre-packaged with configurations, Kibana dashboards, and
other meta files that make it easier for you to set up the Elastic stack for
specific use cases or data sources.

[float]
[[running-logstash-modules]]
=== Running modules

When you run a module, Logstash creates and loads the pipeline configurations
required to read and parse the data. It also loads the index pattern,
field definitions, searches, visualizations, and dashboards required to
visualize your data in Kibana.

To run a module, you use the `--modules` option:

[source,shell]
----
bin/logstash --modules MODULE_NAME [-M CONFIG_SETTINGS]
----


//REVIEWERS: Can users run multiple modules like they can in Filebeat? This topic assumes "no" but I haven't tested it yet.

Where `MODULE_NAME` is the name of Logstash module and `CONFIG_SETTINGS`
is one or more optional configuration settings. `CONFIG_SETTINGS` are only
required when the default configuration doesn't meet your needs, or you need to
override settings specified in the `logstash.yml` settings file.

For example, the following command runs the Netflow module with the default
settings:

[source,shell]
----
bin/logstash --modules netflow
----

The following command runs the Netflow module and overrides the `host` setting:

[source,shell]
----
bin/logstash --modules netflow -M "netflow.var.elasticsearch.host=es.mycloud.com"
----


//REVIEWERS: I'm mentioning the overrides here because I've had some feedback from Tanya about making it clear in the docs that the var overrides are available. Figured this is relevant for LS too.

See <<overriding-logstash-module-settings>> for more info about overriding settings.

[float]
[[configuring-logstash-modules]]
=== Configuring modules

//REVIEWERS: How will users know when the defaults are OK and when they need to change the config?

To configure a module, you can either
<<setting-logstash-module-config,specify configuration settings>> in the
`logstash.yml` <<logstash-settings-file,settings file>>, or use command-line overrides to
<<overriding-logstash-module-settings,specify settings at the command line>>.

[float]
[[setting-logstash-module-config]]
==== Specify module settings in `logstash.yml`

To specify module settings in the `logstash.yml`
<<logstash-settings-file,settings file>> file, you add a module definition to
the modules array. Each module definition begins with a dash (-) and is followed
by `name: module_name` then a series of name/value pairs that specify module
settings. For example:

[source,shell]
----
modules:
- name: netflow
var.output.elasticsearch.host: "es.mycloud.com"
var.output.elasticsearch.user: "foo"
var.output.elasticsearch.password: "password"
var.input.tcp.port: 5606
----

For a list of available module settings, see the documentation for the module.

[float]
[[overriding-logstash-module-settings]]
==== Specify module settings at the command line

You can override module settings by specifying one or more configuration
overrides when you start Logstash. To specify an override, you use the `-M`
command line option:

[source,shell]
----
-M MODULE_NAME.var.PLUGINTYPE1.PLUGINNAME1.KEY1=VALUE
----

Notice that the fully-qualified setting name includes the module name.

You can specify multiple overrides. Each override must start with `-M`.

The following command runs the Netflow module and sets the `tcp.port` to `5606`:

//REVIEWERS: It would be better here to show an example that overrides multiple settings. Any suggestions for a realistic example?

[source,shell]
----
bin/logstash --modules netflow -M "netflow.var.tcp.port=5606"
----

Any settings defined in the command line are ephemeral and will not persist across
subsequent runs of Logstash. If you want to persist a configuration, you need to
set it in the `logstash.yml` <<logstash-settings-file,settings file>>.

Settings that you specify at the command line are merged with any settings
specified in the `logstash.yml` file. If an option is set in both
places, the value specified at the command line takes precedence.


0 comments on commit 15b75e3

Please sign in to comment.