Skip to content

Commit

Permalink
Avoid mapping issues in kubernetes module (elastic#8487)
Browse files Browse the repository at this point in the history
* Avoid mapping issues in kubernetes module

This contains 2 changes:

 * Do not report `start_time` when it's empty, as that will throw a
 mapping exception
 * Fix `fields.yml`, as some fields were out of place for `event`
 metricset

(cherry picked from commit 3c363a0)
  • Loading branch information
exekias authored and Carlos Pérez-Aradros Herce committed Oct 1, 2018
1 parent f0b5fe8 commit 86e060b
Show file tree
Hide file tree
Showing 8 changed files with 56 additions and 20 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ https://github.com/elastic/beats/compare/v6.4.0...6.x[Check the HEAD diff]
- Fixed the location of the modules.d dir in Deb and RPM packages. {issue}8104[8104]
- Add docker diskio stats on Windows. {issue}6815[6815] {pull}8126[8126]
- Fix incorrect type conversion of average response time in Haproxy dashboards {pull}8404[8404]
- Fix dropwizard module parsing of metric names. {issue}8365[8365] {pull}6385[8385]
- Avoid mapping issues in kubernetes module. {pull}8487[8487]

*Packetbeat*

Expand Down
21 changes: 21 additions & 0 deletions metricbeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -8239,6 +8239,27 @@ type: long
Count field records the number of times the particular event has occurred
--
*`kubernetes.event.timestamp.first_occurrence`*::
+
--
type: date
Timestamp of first occurrence of event
--
*`kubernetes.event.timestamp.last_occurrence`*::
+
--
type: date
Timestamp of last occurrence of event
--
*`kubernetes.event.message`*::
Expand Down
7 changes: 5 additions & 2 deletions metricbeat/module/kubernetes/container/data.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,7 @@ func eventMapping(content []byte, perfMetrics *util.PerfMetricsCache) ([]common.
},
},

"name": container.Name,
"start_time": container.StartTime,
"name": container.Name,

"cpu": common.MapStr{
"usage": common.MapStr{
Expand Down Expand Up @@ -114,6 +113,10 @@ func eventMapping(content []byte, perfMetrics *util.PerfMetricsCache) ([]common.
},
}

if container.StartTime != "" {
containerEvent.Put("start_time", container.StartTime)
}

if nodeCores > 0 {
containerEvent.Put("cpu.usage.node.pct", float64(container.CPU.UsageNanoCores)/1e9/nodeCores)
}
Expand Down
21 changes: 10 additions & 11 deletions metricbeat/module/kubernetes/event/_meta/fields.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,18 +9,17 @@
type: long
description: >
Count field records the number of times the particular event has occurred
- name: timestamp
type: group
fields:
- name: timestamp
type: group
fields:
- name: first_occurrence
type: date
description: >
Timestamp of first occurrence of event
- name: last_occurrence
type: date
description: >
Timestamp of last occurrence of event
- name: first_occurrence
type: date
description: >
Timestamp of first occurrence of event
- name: last_occurrence
type: date
description: >
Timestamp of last occurrence of event
- name: message
type: keyword
description: >
Expand Down
2 changes: 1 addition & 1 deletion metricbeat/module/kubernetes/fields.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 6 additions & 2 deletions metricbeat/module/kubernetes/node/data.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,7 @@ func eventMapping(content []byte) (common.MapStr, error) {

node := summary.Node
nodeEvent := common.MapStr{
"name": node.NodeName,
"start_time": node.StartTime,
"name": node.NodeName,

"cpu": common.MapStr{
"usage": common.MapStr{
Expand Down Expand Up @@ -105,5 +104,10 @@ func eventMapping(content []byte) (common.MapStr, error) {
},
},
}

if node.StartTime != "" {
nodeEvent.Put("start_time", node.StartTime)
}

return nodeEvent, nil
}
7 changes: 5 additions & 2 deletions metricbeat/module/kubernetes/pod/data.go
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,7 @@ func eventMapping(content []byte, perfMetrics *util.PerfMetricsCache) ([]common.
"name": node.NodeName,
},
},
"name": pod.PodRef.Name,
"start_time": pod.StartTime,
"name": pod.PodRef.Name,

"cpu": common.MapStr{
"usage": common.MapStr{
Expand All @@ -85,6 +84,10 @@ func eventMapping(content []byte, perfMetrics *util.PerfMetricsCache) ([]common.
},
}

if pod.StartTime != "" {
podEvent.Put("start_time", pod.StartTime)
}

if coresLimit > nodeCores {
coresLimit = nodeCores
}
Expand Down
8 changes: 6 additions & 2 deletions metricbeat/module/kubernetes/system/data.go
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,7 @@ func eventMapping(content []byte) ([]common.MapStr, error) {
"name": node.NodeName,
},
},
"container": syscontainer.Name,
"start_time": syscontainer.StartTime,
"container": syscontainer.Name,
"cpu": common.MapStr{
"usage": common.MapStr{
"nanocores": syscontainer.CPU.UsageNanoCores,
Expand All @@ -68,6 +67,11 @@ func eventMapping(content []byte) ([]common.MapStr, error) {
"majorpagefaults": syscontainer.Memory.MajorPageFaults,
},
}

if syscontainer.StartTime != "" {
containerEvent.Put("start_time", syscontainer.StartTime)
}

events = append(events, containerEvent)
}

Expand Down

0 comments on commit 86e060b

Please sign in to comment.