-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Docs] Add module usage info #7826
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,118 @@ | ||
[[logstash-modules]] | ||
== Working with Logstash Modules | ||
|
||
Logstash modules provide a quick, end-to-end solution for ingesting data and | ||
visualizing it with purpose-built dashboards. | ||
|
||
Each module comes pre-packaged with Logstash configurations, Kibana dashboards, | ||
and other meta files that make it easier for you to set up the Elastic Stack for | ||
specific use cases or data sources. | ||
|
||
[float] | ||
[[running-logstash-modules]] | ||
=== Running modules | ||
|
||
When you run a module, Logstash creates and loads the pipeline configurations | ||
required to read and parse the data. It also loads the index pattern, | ||
field definitions, searches, visualizations, and dashboards required to | ||
visualize your data in Kibana. | ||
|
||
To run a module, you use the `--modules` option: | ||
|
||
[source,shell] | ||
---- | ||
bin/logstash --modules MODULE_NAME [-M CONFIG_SETTINGS] | ||
---- | ||
|
||
|
||
//TODO: For 6.0, show how to run mutliple modules | ||
|
||
Where `MODULE_NAME` is the name of Logstash module and `CONFIG_SETTINGS` | ||
is one or more optional configuration settings. `CONFIG_SETTINGS` are only | ||
required when the default configuration doesn't meet your needs, or you need to | ||
override settings specified in the `logstash.yml` settings file. | ||
|
||
For example, the following command runs the Netflow module with the default | ||
settings: | ||
|
||
[source,shell] | ||
---- | ||
bin/logstash --modules netflow | ||
---- | ||
|
||
The following command runs the Netflow module and overrides the Elasticsearch | ||
`host` setting: | ||
|
||
[source,shell] | ||
---- | ||
bin/logstash --modules netflow -M "netflow.var.elasticsearch.host=es.mycloud.com" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @suyograo missed this one initially, but this should be plural "hosts" right? and its also missing the port? |
||
---- | ||
|
||
|
||
See <<overriding-logstash-module-settings>> for more info about overriding settings. | ||
|
||
[float] | ||
[[configuring-logstash-modules]] | ||
=== Configuring modules | ||
|
||
To configure a module, you can either | ||
<<setting-logstash-module-config,specify configuration settings>> in the | ||
`logstash.yml` <<logstash-settings-file,settings file>>, or use command-line overrides to | ||
<<overriding-logstash-module-settings,specify settings at the command line>>. | ||
|
||
[float] | ||
[[setting-logstash-module-config]] | ||
==== Specify module settings in `logstash.yml` | ||
|
||
To specify module settings in the `logstash.yml` | ||
<<logstash-settings-file,settings file>> file, you add a module definition to | ||
the modules array. Each module definition begins with a dash (-) and is followed | ||
by `name: module_name` then a series of name/value pairs that specify module | ||
settings. For example: | ||
|
||
[source,shell] | ||
---- | ||
modules: | ||
- name: netflow | ||
var.output.elasticsearch.host: "es.mycloud.com" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @suyograo same here as the comment above... |
||
var.output.elasticsearch.user: "foo" | ||
var.output.elasticsearch.password: "password" | ||
var.input.tcp.port: 5606 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @suyograo also, does netflow has a tcp setting? i thought it was udp. |
||
---- | ||
|
||
For a list of available module settings, see the documentation for the module. | ||
|
||
[float] | ||
[[overriding-logstash-module-settings]] | ||
==== Specify module settings at the command line | ||
|
||
You can override module settings by specifying one or more configuration | ||
overrides when you start Logstash. To specify an override, you use the `-M` | ||
command line option: | ||
|
||
[source,shell] | ||
---- | ||
-M MODULE_NAME.var.PLUGINTYPE1.PLUGINNAME1.KEY1=VALUE | ||
---- | ||
|
||
Notice that the fully-qualified setting name includes the module name. | ||
|
||
You can specify multiple overrides. Each override must start with `-M`. | ||
|
||
The following command runs the Netflow module and overrides both the | ||
Elasticsearch `host` setting and the `udp.port` setting: | ||
|
||
[source,shell] | ||
---- | ||
bin/logstash --modules netflow -M "netflow.var.input.udp.port=3555" -M "netflow.var.elasticseach.host=my-es-cloud" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @suyograo same comment here as above. |
||
---- | ||
|
||
Any settings defined in the command line are ephemeral and will not persist across | ||
subsequent runs of Logstash. If you want to persist a configuration, you need to | ||
set it in the `logstash.yml` <<logstash-settings-file,settings file>>. | ||
|
||
Settings that you specify at the command line are merged with any settings | ||
specified in the `logstash.yml` file. If an option is set in both | ||
places, the value specified at the command line takes precedence. | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
index pattern and field mappings will be directly on ES, should we also make that explicit?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My inclination is to keep this statement simple here. If we start seeing questions about this on the forum, we can clarify.