Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Configuration refactoring #1417

Closed
14 tasks done
tsg opened this issue Apr 19, 2016 · 1 comment
Closed
14 tasks done

Configuration refactoring #1417

tsg opened this issue Apr 19, 2016 · 1 comment
Labels
in progress Pull request is currently in progress. meta

Comments

@tsg
Copy link
Contributor

tsg commented Apr 19, 2016 •
edited
Loading

This is a meta issue for reorganizing our default configuration files for 5.0. The problems we're trying to solve are:

  • Getting whitespace wrong in YAML is our number one source of issues on discuss. Now the Beats supports dotted notation (e.g. output.elasticsearch.hosts) in the configuration file, which greatly reduces the indentation levels required.
  • Inconsistent namespacing around Beats. Filebeat has all its options under filebeat, Topbeat has/had input, Packetbeat has several options toplevel. Correct namespacing is a prerequisite for building more Beats in the same binary/process.
  • The Libbeat options are not well organized, we have some stuff top level and some other stuff under beat.
  • Because we put all available options in the configuration file, the default files have gotten really large and hard to modify for the common options.

TODOs:
@tsg tsg mentioned this issue Apr 19, 2016
Closed
tsg pushed a commit to tsg/beats that referenced this issue Apr 20, 2016
If confdir is relative, make it relative to the config path
a3516d1 
This should hopefully  be the least surprise for the user. Part of elastic#1417.
@tsg tsg mentioned this issue Apr 20, 2016
Merged
andrewkroh pushed a commit that referenced this issue Apr 20, 2016
@tsg @andrewkroh
If confdir is relative, make it relative to the config path (#1435)
5399141 
This should hopefully  be the least surprise for the user. Part of #1417.
@monicasarbu monicasarbu added the in progress Pull request is currently in progress. label Apr 21, 2016
@ruflin ruflin mentioned this issue Apr 26, 2016
Merged
ruflin added a commit to ruflin/beats that referenced this issue Apr 27, 2016
@ruflin
Migrate packetbeat config options to namespace
26627ff 
* Fix docs by adding indentation
* Remove singleton config

Relates to elastic#1417
tsg pushed a commit that referenced this issue Apr 27, 2016
@ruflin @tsg
Migrate packetbeat config options to namespace (#1490)
9aeac72 
* Fix docs by adding indentation
* Remove singleton config

Relates to #1417
@dedemorton dedemorton mentioned this issue Apr 28, 2016
Merged
@monicasarbu monicasarbu mentioned this issue May 2, 2016
Merged
5 tasks
@tsg tsg mentioned this issue May 4, 2016
Merged
tsg pushed a commit to tsg/beats that referenced this issue May 9, 2016
Move "beat" (ex-shipper) config settings to be top level
6b7dd17 
Part of elastic#1417. Done in a non-BWC way, because I think it's not worth
complicating the code. I'd rather have a script that migrates the
configuration.
@tsg tsg mentioned this issue May 9, 2016
Closed
ruflin pushed a commit that referenced this issue May 9, 2016
@tsg @ruflin
Move "beat" (ex-"shipper") config settings to be top level (#1570)
d51884b 
* Move "beat" (ex-shipper) config settings to be top level

Part of #1417. Done in a non-BWC way, because I think it's not worth
complicating the code. I'd rather have a script that migrates the
configuration.

* Make winlogbeat accept the new top-level settings

The list has grown quite long, switching to ucfg validation for this would be nice.
Also added explicit key names to config.
tsg pushed a commit to tsg/beats that referenced this issue May 12, 2016
Configuration files restyling
e8b04f0 
Little changes content wise, but some style changes:

* Use ###, ===, and --- headers to indicate hierarchy. This makes it
  easier to skim the config file. Idea stolen from elasticsearch.yml
* Unindent filebeat prospectors and metricbeat modules by one level.
  This is still valid yaml, and fewer spaces make it harder to mess up the
  white spaces.
* Reorganized a bit the logging config
* Moved the "general" libbeat section before the outputs
* Other fairly minor changes to the beats yaml files.

This is a follow up of elastic#1544 and part of elastic#1417.
@tsg tsg mentioned this issue May 12, 2016
Merged
ruflin pushed a commit that referenced this issue May 12, 2016
@tsg @ruflin
Configuration files restyling (#1624)
2180852 
Little changes content wise, but some style changes:

* Use ###, ===, and --- headers to indicate hierarchy. This makes it
  easier to skim the config file. Idea stolen from elasticsearch.yml
* Unindent filebeat prospectors and metricbeat modules by one level.
  This is still valid yaml, and fewer spaces make it harder to mess up the
  white spaces.
* Reorganized a bit the logging config
* Moved the "general" libbeat section before the outputs
* Other fairly minor changes to the beats yaml files.

This is a follow up of #1544 and part of #1417.

* Added a new line in front of the libbeat options
* Make the styling changes for Metricbeat in automation
@tsg tsg mentioned this issue May 13, 2016
Merged
@tsg tsg mentioned this issue May 20, 2016
Merged
tsg pushed a commit to tsg/beats that referenced this issue May 20, 2016
Updated configuration templates used in tests
dd69284 
The goal was to bring them closer to the new conventions we
use in the configuration files, so what we test is closer to what
we ship in terms of configuration.

I also made steps in unifying the libbeat portion of these templates so
we could remove the duplication in the future.

Part of elastic#1417.
ruflin pushed a commit that referenced this issue May 23, 2016
@tsg @ruflin
Updated configuration templates used in tests (#1688)
0a3505a 
The goal was to bring them closer to the new conventions we
use in the configuration files, so what we test is closer to what
we ship in terms of configuration.

I also made steps in unifying the libbeat portion of these templates so
we could remove the duplication in the future.

Part of #1417.
This was referenced May 23, 2016
Merged
Merged
@tsg
Copy link
Contributor Author

tsg commented May 24, 2016

All done. 😎

@tsg tsg closed this as completed May 24, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
in progress Pull request is currently in progress. meta
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tsg @monicasarbu