Packetbeat protocol analyzer configuration enhancements #3518
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
urso
added
discuss
urso
force-pushed
the
enh/packetbeat-modules-config
branch
from
17ff13f
to
ae6c95f
Compare
|
I just reused already available functionality... |
monicasarbu
approved these changes
Feb 16, 2017
|
@urso I suggest we change our current default config to the array style and deprecate the old style. This will mean we have the same structure across all beats. |
urso
force-pushed
the
enh/packetbeat-modules-config
branch
from
ae6c95f
to
b76b503
Compare
urso
added
docs
review
and removed
discuss
dedemorton
reviewed
Feb 21, 2017
CHANGELOG.asciidoc
Outdated
| @@ -92,6 +92,8 @@ https://github.com/elastic/beats/compare/v5.1.1...master[Check the HEAD diff] | |||
| - The HAProxy module is now GA, instead of experimental. {pull}3525[3525] | |||
|
|
|||
| *Packetbeat* | |||
| - Add `fields` and `fields_under_root` to packetbeat protocols configurations. {pull}3518[3518] | |||
| - Add list style packetbeat protocols configurations. This change support different configurations of same protocol analyzers to be active. {pull]3518[3518] | |||
There was a problem hiding this comment.
Would change this to say, "Add list style Packetbeat protocol configurations. This change supports different configurations of the same protocol analyzer to be active." The second sentence is a bit confusing even with the edit. How about this instead? "This change supports specifying multiple configurations of the same protocol analyzer." [saying "to be active" makes the sentence harder to parse and isn't IMO necessary.]
- Optionally configure protocol analyzers using dictionary and/or list
- Add 'fields', 'fields_under_root' and 'tags' settings to every
protocol analyzer
- update sample config file to use list style configuration
- add deprecated warning if dictionary style configuration is used
- update docs
This change allows for configuring packetbeat protocols in 2 different
styles... both styles can be used at the same time,.
1.) (deprecated) dictionary style:
```
packetbeat.protocols.http:
...
packetbeat.protocols.dns:
...
```
2.) array style:
```
packetbeat.protocols:
- type: http
...
- type: dns
...
```
Examples (1) and (2) are equivalent. But array style allows to configure a
protocol analyzer multiple times: e.g.
(3) array style with multiple instances of http protocol analyzer:
```
packetbeat.protocols:
- type: http
ports: [80]
fields.service: nginx
- type: http
ports: [9200]
fields.service: elasticsearch
```
4) mixed style:
```
packetbeat.protocols.http:
...
packetbeat.protocols:
- type: dns
...
```
Limitations:
a) due to limitations in yaml parser, only capturing the last 'name' in a
dictionary the key name `packetbeat.protocols` must not be used multiple times.
e.g. this will result in an incompletely processed config (only DNS will be
configured):
```
packetbeat.protocols:
http:
...
packetbeat.protocols:
- type: dns
```
b) Reusing port numbers (overlapping) might result in one module not seeing any
packets (this is already the case if any 2 protocols shall listen on same port
number).
urso
force-pushed
the
enh/packetbeat-modules-config
branch
from
d78e421
to
7dfa8f0
Compare
|
One step closer to have beats configs unified 🎉 About defining a port multiple times. This is kind of similar to defining a file multiple times in filebeat, just easier to detect. If there would be a registry of ports, we could return an error in case this happens. Having protocols as list will also make it easier to implement config reloading. |
monicasarbu
changed the title
10 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
protocol analyzer
This change allows for configuring packetbeat protocols in 2 different styles... both styles can be used at the same time.
1.) dictionary style:
2.) array style:
Examples (1) and (2) are equivalent. But array style allows to configure a protocol analyzer multiple times: e.g.
(3) array style with multiple instances of http protocol analyzer:
Limitations:
a) due to limitations in yaml parser, only capturing the last 'name' in a dictionary the key name
packetbeat.protocolsmust not be used multiple times. e.g. this will result in an incompletely processed config (only DNS will be configured):b) Reusing port numbers (overlapping) might result in one module not seeing any packets (this is already the case if any 2 protocols shall listen on same port number). e.g.: