Add events metricset for kubernetes metricbeat module #4315
Conversation
vjsamuel
changed the title
|
@ruflin, vendoring needs to be decided as im getting these info messages: |
|
Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run. |
1 similar comment
|
Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run. |
monicasarbu
added
in progress
| // Part of new is also setting up the configuration by processing additional | ||
| // configuration entries if needed. | ||
| func New(base mb.BaseMetricSet) (mb.MetricSet, error) { | ||
| logp.Warn("EXPERIMENTAL: The kubernetes events metricset is experimental") |
There was a problem hiding this comment.
Done. We would need to update the metricset generator as this was generated code.
| return nil, fmt.Errorf("fail to unpack the kubernetes events configuration: %s", err) | ||
| } | ||
|
|
||
| err = validate(config) |
There was a problem hiding this comment.
go-ucfg allows a config type to implement a Validate() error function that is invoked automatically on when unpacking.
| m.watcher.Stop() | ||
| return | ||
| case msg := <-m.watcher.eventQueue: | ||
| // Ignore events that are deleted |
There was a problem hiding this comment.
Why are you ignoring deletion events? I guess someone could be interested on them, perhaps we should offer some fine grained filter for desired events?
There was a problem hiding this comment.
delete, in case of events is to just delete the API Event object. it carries no significance from monitoring perspective.
vjsamuel
changed the title
vjsamuel
force-pushed
the
kubernetes_events
branch
2 times, most recently
from
2e53faf to
c87167f
Compare
There was a problem hiding this comment.
Thanks for the PR. I left a few comments mainly related to naming conventions.
One thing that seems to be missing is adding it to the config file.
From which of the two services which we connect to in the kubernetes module at the moment will this info be fetched? What is the installation pattern?
metricbeat/docs/fields.asciidoc
Outdated
| @@ -6040,6 +6040,194 @@ Used inodes | |||
|
|
|||
|
|
|||
| [float] | |||
| == events Fields | |||
There was a problem hiding this comment.
The metricset should probably be called event and not events to be in line with our naming convention.
metricbeat/docs/fields.asciidoc
Outdated
|
|
||
|
|
||
| [float] | ||
| === kubernetes.events.firstOccurrenceTimestamp |
There was a problem hiding this comment.
I would suggest to change this to *.event.timestamp.first_occurenceand below to *.event.timestamp.last_occurence.
metricbeat/docs/fields.asciidoc
Outdated
|
|
||
|
|
||
| [float] | ||
| === kubernetes.events.metadata.creationTimestamp |
There was a problem hiding this comment.
timestamp.created, timestamp.deleted
metricbeat/docs/fields.asciidoc
Outdated
|
|
||
|
|
||
| [float] | ||
| === kubernetes.events.metadata.resourceVersion |
There was a problem hiding this comment.
resource_version or version.resource
metricbeat/docs/fields.asciidoc
Outdated
|
|
||
|
|
||
| [float] | ||
| === kubernetes.events.metadata.selfLink |
metricbeat/docs/fields.asciidoc
Outdated
|
|
||
|
|
||
| [float] | ||
| === kubernetes.events.tags.host |
metricbeat/docs/fields.asciidoc
Outdated
|
|
||
|
|
||
| [float] | ||
| === kubernetes.events.tags.pod |
There was a problem hiding this comment.
What is this under tags. Don't we have a common namespace for these?
There was a problem hiding this comment.
these are dimensions that can be used to better query information. base implementation off of which I went with is here:
https://github.com/kubernetes/heapster/blob/master/events/sinks/elasticsearch/driver.go#L76
| ) | ||
|
|
||
| // EventWatcher is a controller that synchronizes Pods. | ||
| type EventWatcher struct { |
There was a problem hiding this comment.
You could call this just Watcher as it is already in the event package.
| if err != nil { | ||
| //if listing fails try again after sometime | ||
| logp.Err("kubernetes: List API error %v", err) | ||
| time.Sleep(time.Second) |
There was a problem hiding this comment.
Based on what did you select this timeout? Perhaps a comment could be helpful here.
| //watch events failures should be logged and gracefully failed over as metadata retrieval | ||
| //should never stop. | ||
| logp.Err("kubernetes: Watching API eror %v", err) | ||
| time.Sleep(time.Second) |
vjsamuel
force-pushed
the
kubernetes_events
branch
from
c87167f to
463299c
Compare
metricbeat/metricbeat.full.yml
Outdated
| - module: kubernetes | ||
| metricsets: | ||
| - event | ||
| kube_config: ${HOME}/.kube/config |
There was a problem hiding this comment.
- Is
${HOME}always set in these environments? - You left out the hosts: [...] part. Is that on purpose?
There was a problem hiding this comment.
as per @exekias we can default in_cluster to true. this metricset doesnt rely on hosts by default. we can add documentation on various ways to configure it similar to the kubernetes processor.
| }, | ||
| "message": "Created pod: prometheus-2552087900-9fxh6", | ||
| "metadata": { | ||
| "annotations": null, |
There was a problem hiding this comment.
Would be nice to not send fields if they are null
| # Kubernetes events | ||
| - module: kubernetes | ||
| metricsets: | ||
| - event |
There was a problem hiding this comment.
I remember @exekias added somewhere that all state_* prefixed metricset must be installed once in contrast to the other ones. If this is installed global (my assumption) should this also have a state_ prefix?
NOTE: In general I think we need to find a better solution for this in the long run but in the short term we should try to be consistent.
There was a problem hiding this comment.
I think event (or events) is ok in this case, this is something unrelated to kube-state-metrics, state_ metricsets get their name from it
| metricsets: | ||
| - event | ||
| kube_config: ${HOME}/.kube/config | ||
| in_cluster: false |
There was a problem hiding this comment.
perhaps we should leave in_cluster: true as default? I guess it will be the most common case
There was a problem hiding this comment.
will default to true
| # Kubernetes events | ||
| - module: kubernetes | ||
| metricsets: | ||
| - event |
There was a problem hiding this comment.
I think event (or events) is ok in this case, this is something unrelated to kube-state-metrics, state_ metricsets get their name from it
| "deleted": eve.Metadata.DeletionTimestamp, | ||
| }, | ||
| "name": eve.Metadata.Name, | ||
| "namespace": eve.Metadata.Namespace, |
There was a problem hiding this comment.
this one can ve moved to kubernetes.namespace, using mb.ModuleKey. WDYT?
There was a problem hiding this comment.
we would need to do this across all metricsets to establish consistency. lets do it in a follow-up PR all in one go.
vjsamuel
force-pushed
the
kubernetes_events
branch
from
463299c to
a5e3f48
Compare
vjsamuel
force-pushed
the
kubernetes_events
branch
from
a5e3f48 to
3267969
Compare
|
jenkins, test it |
| SyncPeriod time.Duration `config:"sync_period"` | ||
| } | ||
|
|
||
| type Enabled struct { |
There was a problem hiding this comment.
[golint] reported by reviewdog 🐶
exported type Enabled should have comment or be unexported
| Enabled bool `config:"enabled"` | ||
| } | ||
|
|
||
| type PluginConfig []map[string]common.Config |
There was a problem hiding this comment.
[golint] reported by reviewdog 🐶
exported type PluginConfig should have comment or be unexported
|
|
||
| import "time" | ||
|
|
||
| type ObjectMeta struct { |
There was a problem hiding this comment.
[golint] reported by reviewdog 🐶
exported type ObjectMeta should have comment or be unexported
| UID string `json:"uid"` | ||
| } | ||
|
|
||
| type Event struct { |
There was a problem hiding this comment.
[golint] reported by reviewdog 🐶
exported type Event should have comment or be unexported
|
|
||
| } | ||
|
|
||
| func (w *Watcher) Run() { |
There was a problem hiding this comment.
[golint] reported by reviewdog 🐶
exported method Watcher.Run should have comment or be unexported
|
|
||
| } | ||
|
|
||
| func (w *Watcher) Stop() { |
There was a problem hiding this comment.
[golint] reported by reviewdog 🐶
exported method Watcher.Stop should have comment or be unexported
monicasarbu
added
review
and removed
in progress
This PR adds a kubernetes events metricset to the kubernetes metricbeat module. Events are polled from the kubernetes API server and are written into the outputs interface.