Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add condition to the config file template for add_kubernetes_metadata #14056

Merged
merged 5 commits into from
Oct 15, 2019
Merged

Add condition to the config file template for add_kubernetes_metadata #14056

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
a8df199
Add condition to the config file template for add_kubernetes_metadata
ChrsMark Oct 15, 2019
c30efd1
fix format
ChrsMark Oct 15, 2019
05c4ea3
Remove blank lines
ChrsMark Oct 15, 2019
04ae96c
Add changelog entry
ChrsMark Oct 15, 2019
32d797a
Merge remote-tracking branch 'upstream/master' into config_condititio…
ChrsMark Oct 15, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion auditbeat/auditbeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,11 +145,12 @@ output.elasticsearch:

# Configure processors to enhance or manipulate events generated by the beat.


processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~


#================================ Logging =====================================

Expand Down
23 changes: 12 additions & 11 deletions dev-tools/mage/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,17 +105,18 @@ func Config(types ConfigFileType, args ConfigFileParams, targetDir string) error
}

params := map[string]interface{}{
"GOOS": EnvOr("DEV_OS", "linux"),
"GOARCH": EnvOr("DEV_ARCH", "amd64"),
"Reference": false,
"Docker": false,
"ExcludeConsole": false,
"ExcludeFileOutput": false,
"ExcludeKafka": false,
"ExcludeLogstash": false,
"ExcludeRedis": false,
"UseObserverProcessor": false,
"ExcludeDashboards": false,
"GOOS": EnvOr("DEV_OS", "linux"),
"GOARCH": EnvOr("DEV_ARCH", "amd64"),
"Reference": false,
"Docker": false,
"ExcludeConsole": false,
"ExcludeFileOutput": false,
"ExcludeKafka": false,
"ExcludeLogstash": false,
"ExcludeRedis": false,
"UseObserverProcessor": false,
"UseKubernetesMetadataProcessor": false,
"ExcludeDashboards": false,
}
for k, v := range args.ExtraVars {
params[k] = v
Expand Down
2 changes: 2 additions & 0 deletions filebeat/filebeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,12 +173,14 @@ output.elasticsearch:

# Configure processors to enhance or manipulate events generated by the beat.


processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~


#================================ Logging =====================================

# Sets log level. The default log level is info.
Expand Down
3 changes: 3 additions & 0 deletions filebeat/scripts/mage/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,9 @@ func configFileParams(moduleDirs ...string) devtools.ConfigFileParams {
devtools.OSSBeatDir("_meta/beat.docker.yml"),
devtools.LibbeatDir("_meta/config.docker.yml"),
},
ExtraVars: map[string]interface{}{
"UseKubernetesMetadataProcessor": true,
},
}
}

Expand Down
3 changes: 2 additions & 1 deletion journalbeat/journalbeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,11 +142,12 @@ output.elasticsearch:

# Configure processors to enhance or manipulate events generated by the beat.


processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~


#================================ Logging =====================================

Expand Down
7 changes: 7 additions & 0 deletions libbeat/_meta/config.yml.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,12 +90,19 @@ output.elasticsearch:
{{if not .UseObserverProcessor}}
# Configure processors to enhance or manipulate events generated by the beat.

{{if .UseKubernetesMetadataProcessor}}
processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
{{else}}
processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
{{end}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could this change be just adding the kubernetes processor within the existing list? Something like:

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
{{if .UseKubernetesMetadataProcessor}}
  - add_kubernetes_metadata: ~
{{end}}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was what I tried first, however this adds extra blank lines because of the template conditionals like:

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~

  - add_kubernetes_metadata: ~

If this is ok, I can change it.

Copy link
Contributor

@exekias exekias Oct 15, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gotcha, try with {{- if ... -}}!

{{else}}
processors:
- add_observer_metadata:
# Optional, but recommended geo settings for the location {{ .BeatName | title }} is running in
Expand Down
2 changes: 2 additions & 0 deletions metricbeat/metricbeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,12 +117,14 @@ output.elasticsearch:

# Configure processors to enhance or manipulate events generated by the beat.


processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~


#================================ Logging =====================================

# Sets log level. The default log level is info.
Expand Down
3 changes: 3 additions & 0 deletions metricbeat/scripts/mage/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,9 @@ func configFileParams(moduleDirs ...string) devtools.ConfigFileParams {
devtools.OSSBeatDir("_meta/beat.docker.yml"),
devtools.LibbeatDir("_meta/config.docker.yml"),
},
ExtraVars: map[string]interface{}{
"UseKubernetesMetadataProcessor": true,
},
}
}

Expand Down
3 changes: 2 additions & 1 deletion packetbeat/packetbeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,11 +199,12 @@ output.elasticsearch:

# Configure processors to enhance or manipulate events generated by the beat.


processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~


#================================ Logging =====================================

Expand Down
3 changes: 2 additions & 1 deletion winlogbeat/winlogbeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,11 +124,12 @@ output.elasticsearch:

# Configure processors to enhance or manipulate events generated by the beat.


processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~


#================================ Logging =====================================

Expand Down
3 changes: 2 additions & 1 deletion x-pack/auditbeat/auditbeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,11 +167,12 @@ output.elasticsearch:

# Configure processors to enhance or manipulate events generated by the beat.


processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~


#================================ Logging =====================================

Expand Down
2 changes: 2 additions & 0 deletions x-pack/filebeat/filebeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,12 +173,14 @@ output.elasticsearch:

# Configure processors to enhance or manipulate events generated by the beat.


processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~


#================================ Logging =====================================

# Sets log level. The default log level is info.
Expand Down
3 changes: 2 additions & 1 deletion x-pack/functionbeat/functionbeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -341,11 +341,12 @@ output.elasticsearch:

# Configure processors to enhance or manipulate events generated by the beat.


processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~


#================================ Logging =====================================

Expand Down
2 changes: 2 additions & 0 deletions x-pack/metricbeat/metricbeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,12 +117,14 @@ output.elasticsearch:

# Configure processors to enhance or manipulate events generated by the beat.


processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~


#================================ Logging =====================================

# Sets log level. The default log level is info.
Expand Down
3 changes: 2 additions & 1 deletion x-pack/winlogbeat/winlogbeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,11 +136,12 @@ output.elasticsearch:

# Configure processors to enhance or manipulate events generated by the beat.


processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~


#================================ Logging =====================================

Expand Down