Add Kubernetes StatefulSet metricset #6236
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| //// | ||
| This file is generated! See scripts/docs_collector.py | ||
| //// | ||
|
|
||
| [[metricbeat-metricset-kubernetes-state_statefulset]] | ||
| === Kubernetes state_statefulset metricset | ||
|
|
||
| include::../../../module/kubernetes/state_statefulset/_meta/docs.asciidoc[] | ||
|
|
||
|
|
||
| ==== Fields | ||
|
|
||
| For a description of each field in the metricset, see the | ||
| <<exported-fields-kubernetes,exported fields>> section. | ||
|
|
||
| Here is an example document generated by this metricset: | ||
|
|
||
| [source,json] | ||
| ---- | ||
| include::../../../module/kubernetes/state_statefulset/_meta/data.json[] | ||
| ---- |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| { | ||
| "@timestamp": "2017-05-10T16:46:37.821Z", | ||
| "beat": { | ||
| "hostname": "X1", | ||
| "name": "X1", | ||
| "version": "6.0.0-alpha1" | ||
| }, | ||
| "kubernetes": { | ||
| "namespace": "jenkins", | ||
| "statefulset": { | ||
| "name": "wise-lynx-jenkins-1616735317", | ||
| "created": 123454, | ||
| "replicas": { | ||
| "desired": 1, | ||
| "observed": 1, | ||
| }, | ||
| "generation": { | ||
| "desired": 1, | ||
| "observed": 1, | ||
| } | ||
| } | ||
| }, | ||
| "metricset": { | ||
| "host": "192.168.99.100:18080", | ||
| "module": "kubernetes", | ||
| "name": "state_statefulset", | ||
| "namespace": "statefulset", | ||
| "rtt": 6719 | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| This is the `state_statefulset` metricset of the Kubernetes module. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| - name: statefulset | ||
| type: group | ||
| description: > | ||
| kubernetes stateful set metrics | ||
| release: ga | ||
| fields: | ||
| - name: name | ||
| type: keyword | ||
| description: > | ||
| Kubernetes stateful set name | ||
| - name: created | ||
| type: long | ||
| description: > | ||
| The creation timestamp (epoch) for StatefulSet | ||
| - name: replicas | ||
| type: group | ||
| description: > | ||
| Kubernetes stateful set replicas status | ||
| fields: | ||
| - name: observed | ||
| type: long | ||
| description: > | ||
| The number of observed replicas per StatefulSet | ||
| - name: desired | ||
| type: long | ||
| description: > | ||
| The number of desired replicas per StatefulSet | ||
| - name: generation | ||
| type: group | ||
| description: > | ||
| Kubernetes stateful set generation information | ||
| fields: | ||
| - name: observed | ||
| type: long | ||
| description: > | ||
| The observed generation per StatefulSet | ||
| - name: desired | ||
| type: long | ||
| description: > | ||
| The desired generation per StatefulSet |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| package state_statefulset | ||
|
|
||
| import ( | ||
| "github.com/elastic/beats/libbeat/common" | ||
| "github.com/elastic/beats/metricbeat/mb" | ||
| "github.com/elastic/beats/metricbeat/module/kubernetes/util" | ||
|
|
||
| dto "github.com/prometheus/client_model/go" | ||
| ) | ||
|
|
||
| func eventMapping(families []*dto.MetricFamily) ([]common.MapStr, error) { | ||
| eventsMap := map[string]common.MapStr{} | ||
| for _, family := range families { | ||
| for _, metric := range family.GetMetric() { | ||
| statefulset := util.GetLabel(metric, "statefulset") | ||
| if statefulset == "" { | ||
| continue | ||
| } | ||
| namespace := util.GetLabel(metric, "namespace") | ||
| statefulsetKey := namespace + "::" + statefulset | ||
| event, ok := eventsMap[statefulsetKey] | ||
| if !ok { | ||
| event = common.MapStr{} | ||
| eventsMap[statefulsetKey] = event | ||
| } | ||
| switch family.GetName() { | ||
| case "kube_statefulset_metadata_generation": | ||
| event.Put(mb.ModuleDataKey+".namespace", util.GetLabel(metric, "namespace")) | ||
| event.Put(mb.NamespaceKey, "statefulset") | ||
| event.Put("name", util.GetLabel(metric, "statefulset")) | ||
| event.Put("generation.desired", metric.GetGauge().GetValue()) | ||
| case "kube_statefulset_status_observed_generation": | ||
| event.Put("generation.observed", metric.GetGauge().GetValue()) | ||
| case "kube_statefulset_created": | ||
| event.Put("created", metric.GetGauge().GetValue()) | ||
| case "kube_statefulset_replicas": | ||
| event.Put("replicas.desired", metric.GetGauge().GetValue()) | ||
| case "kube_statefulset_status_replicas": | ||
| event.Put("replicas.observed", metric.GetGauge().GetValue()) | ||
| default: | ||
| // Ignore unknown metric | ||
| continue | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // initialize, populate events array from values in eventsMap | ||
| events := make([]common.MapStr, 0, len(eventsMap)) | ||
| for _, event := range eventsMap { | ||
| events = append(events, event) | ||
| } | ||
| return events, nil | ||
| } | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| package state_statefulset | ||
|
There was a problem hiding this comment. don't use an underscore in package name |
||
|
|
||
| import ( | ||
| "github.com/elastic/beats/libbeat/common" | ||
| "github.com/elastic/beats/metricbeat/helper" | ||
| "github.com/elastic/beats/metricbeat/mb" | ||
| "github.com/elastic/beats/metricbeat/mb/parse" | ||
| ) | ||
|
|
||
| const ( | ||
| defaultScheme = "http" | ||
| defaultPath = "/metrics" | ||
| ) | ||
|
|
||
| var ( | ||
| hostParser = parse.URLHostParserBuilder{ | ||
| DefaultScheme: defaultScheme, | ||
| DefaultPath: defaultPath, | ||
| }.Build() | ||
| ) | ||
|
|
||
| // init registers the MetricSet with the central registry. | ||
| // The New method will be called after the setup of the module and before starting to fetch data | ||
| func init() { | ||
| if err := mb.Registry.AddMetricSet("kubernetes", "state_statefulset", New, hostParser); err != nil { | ||
| panic(err) | ||
| } | ||
| } | ||
|
|
||
| // MetricSet type defines all fields of the MetricSet | ||
| // As a minimum it must inherit the mb.BaseMetricSet fields, but can be extended with | ||
| // additional entries. These variables can be used to persist data or configuration between | ||
| // multiple fetch calls. | ||
| type MetricSet struct { | ||
| mb.BaseMetricSet | ||
| prometheus *helper.Prometheus | ||
| } | ||
|
|
||
| // New create a new instance of the MetricSet | ||
| // Part of new is also setting up the configuration by processing additional | ||
| // configuration entries if needed. | ||
| func New(base mb.BaseMetricSet) (mb.MetricSet, error) { | ||
| prometheus, err := helper.NewPrometheusClient(base) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| return &MetricSet{ | ||
| BaseMetricSet: base, | ||
| prometheus: prometheus, | ||
| }, nil | ||
| } | ||
|
|
||
| // Fetch methods implements the data gathering and data conversion to the right format | ||
| // It returns the event which is then forward to the output. In case of an error, a | ||
| // descriptive error must be returned. | ||
| func (m *MetricSet) Fetch() ([]common.MapStr, error) { | ||
| families, err := m.prometheus.GetFamilies() | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| return eventMapping(families) | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
don't use an underscore in package name
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Following same pattern as other classes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, please ignore the bot on this.