Add `bearer_token_file` paramter to HTTP helper #7527

exekias · 2018-07-06T11:16:52Z

This change allows to load bearer tokens from files in modules using
the HTTP helper. This is especially useful for Kubernetes and Prometheus, as some deployments enforce SSL access (like OpenShift):

- module: kubernetes
  metricsets:
    - pod
  bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
  ssl.certificate_authorities:
    - /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt

Closes #7518

exekias · 2018-07-06T11:17:20Z

@vjsamuel, I think this should deprecate the kubernetes.token appender (in 7.0) in favor of this + config appender, anything against that?

andrewkroh

Are these HTTP options documented in Asciidoc somewhere? It would be nice if we could point people to bearer_token_file on the website if needed.

andrewkroh · 2018-07-08T14:48:00Z

metricbeat/module/prometheus/_meta/config.yml

@@ -7,3 +7,8 @@
  #namespace: example
  #username: "user"
  #password: "secret"
+
+  # This can be used for service account based authorization:


I think it would be useful to say what it does. Something like

It reads the contents of the file once at initialization and then uses the value in an HTTP Authorization header.

Makes sense, I updated docs to include this longer explanation

ruflin · 2018-07-09T09:02:49Z

Needs a changelog entry?

exekias · 2018-07-09T09:57:12Z

Thanks for the reviews, I have added more docs & changelog entry

exekias · 2018-07-09T10:50:42Z

metricbeat/helper/http_test.go

@@ -0,0 +1,54 @@
+package helper


I realize now this is a new file without license headers, but CI is not complaining. Are we checking that somewhere @andrewkroh?

@exekias Was also looking into this this morning. We check but return 0 after the check. Not sure if that was on purpose (probably not).

In the meanwhile, I've pushed the header to this file

exekias · 2018-07-09T11:27:34Z

Latest K8s version (1.11, just released) makes the safe port mandatory, so Metricbeat kubernetes module cannot work without this. I'm adding the needs_backport label here. I think it would be nice to have this in 6.3.2

vjsamuel · 2018-07-09T16:55:53Z

metricbeat/autodiscover/appender/kubernetes/token/token.go

@@ -41,7 +41,7 @@ type tokenAppender struct {
 // NewTokenAppender creates a token appender that can append a bearer token required to authenticate with
 // protected endpoints
 func NewTokenAppender(cfg *common.Config) (autodiscover.Appender, error) {
-	cfgwarn.Beta("The token appender is beta")
+	cfgwarn.Deprecate("7.0.0", "token appender is deprecated in favor of bearer_token_file config parameter")


Do you suggest moving to config appender for use cases that require a bearer token then? I wouldnt do a blanket token path on all configs with the hints builder.

Yeah, my thought was to use the config appender to set bearer_token_file wherever you were using the token appender before. No need to create new hints for this.
Would that work for you?

im ok with that. we will do the needful once this is merged.

ruflin · 2018-07-10T11:18:01Z

metricbeat/module/prometheus/_meta/config.yml

@@ -7,3 +7,8 @@
  #namespace: example
  #username: "user"
  #password: "secret"
+
+  # This can be used for service account based authorization:


Should there also be an update to the reference config files? Also applies to the k8s one.

For the paths here for the prometheus config: Should we have k8s as default ones?

I think Prometheus is mostly used in k8s scenarios, it should not harm other use cases anyway? I've pushed a commit to include it in

This change allows to load barer tokens from files in modules using the HTTP helper. For instance: ``` - module: kubernetes metricsets: - pod bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token ssl.certificate_authorities: - /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt ```

ruflin · 2018-07-11T11:27:04Z

Seems like something in prometheus breaks the doc build :-(

exekias · 2018-07-11T14:22:53Z

ups, my bad, should be ok now, sorry

This change allows to load bearer tokens from files in modules using the HTTP helper. This is especially useful for Kubernetes and Prometheus, as some deployments enforce SSL access (like OpenShift): ``` - module: kubernetes metricsets: - pod bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token ssl.certificate_authorities: - /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt ``` Closes elastic#7518 (cherry picked from commit 1d3109f)

…lper (#7577) * Add `bearer_token_file` paramter to HTTP helper (#7527) This change allows to load bearer tokens from files in modules using the HTTP helper. This is especially useful for Kubernetes and Prometheus, as some deployments enforce SSL access (like OpenShift): ``` - module: kubernetes metricsets: - pod bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token ssl.certificate_authorities: - /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt ``` Closes #7518 (cherry picked from commit 1d3109f) * Update CHANGELOG.asciidoc

…HTTP helper (elastic#7577) * Add `bearer_token_file` paramter to HTTP helper (elastic#7527) This change allows to load bearer tokens from files in modules using the HTTP helper. This is especially useful for Kubernetes and Prometheus, as some deployments enforce SSL access (like OpenShift): ``` - module: kubernetes metricsets: - pod bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token ssl.certificate_authorities: - /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt ``` Closes elastic#7518 (cherry picked from commit 7b90836) * Update CHANGELOG.asciidoc

exekias added enhancement in progress Pull request is currently in progress. review Metricbeat Metricbeat labels Jul 6, 2018

exekias removed the in progress Pull request is currently in progress. label Jul 6, 2018

andrewkroh reviewed Jul 8, 2018

View reviewed changes

exekias force-pushed the bearer-token branch from b51b0c1 to 588dc72 Compare July 9, 2018 09:58

exekias commented Jul 9, 2018

View reviewed changes

exekias added needs_backport PR is waiting to be backported to other branches. v6.4.0 v6.3.2 labels Jul 9, 2018

vjsamuel reviewed Jul 9, 2018

View reviewed changes

ruflin reviewed Jul 10, 2018

View reviewed changes

exekias force-pushed the bearer-token branch from 77938b7 to c126027 Compare July 11, 2018 09:32

Carlos Pérez-Aradros Herce added 5 commits July 11, 2018 11:32

Add documentation

4134f37

Add docs

404c4fc

Update CHANGELOG.asciidoc

1019312

Add missing license header

3965a1d

exekias force-pushed the bearer-token branch from c126027 to 0fb7af1 Compare July 11, 2018 09:33

Update reference configs

5bd726e

exekias force-pushed the bearer-token branch 2 times, most recently from 2c5c381 to bac35e7 Compare July 11, 2018 14:52

fix docs

833d0a5

exekias force-pushed the bearer-token branch from bac35e7 to 833d0a5 Compare July 11, 2018 15:07

ruflin merged commit 1d3109f into elastic:master Jul 12, 2018

exekias mentioned this pull request Jul 12, 2018

Cherry-pick #7527 to 6.3: Add bearer_token_file paramter to HTTP helper #7577

Merged

exekias added v6.3.2 and removed needs_backport PR is waiting to be backported to other branches. labels Jul 12, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `bearer_token_file` paramter to HTTP helper #7527

Add `bearer_token_file` paramter to HTTP helper #7527

exekias commented Jul 6, 2018 •

edited

Loading

exekias commented Jul 6, 2018 •

edited

Loading

andrewkroh left a comment

andrewkroh Jul 8, 2018

exekias Jul 9, 2018

ruflin commented Jul 9, 2018

exekias commented Jul 9, 2018

exekias Jul 9, 2018

ruflin Jul 9, 2018

exekias Jul 9, 2018

exekias commented Jul 9, 2018

vjsamuel Jul 9, 2018

exekias Jul 9, 2018

vjsamuel Jul 9, 2018

ruflin Jul 10, 2018

exekias Jul 10, 2018

ruflin commented Jul 11, 2018

exekias commented Jul 11, 2018

Add bearer_token_file paramter to HTTP helper #7527

Add bearer_token_file paramter to HTTP helper #7527

Conversation

exekias commented Jul 6, 2018 • edited Loading

exekias commented Jul 6, 2018 • edited Loading

andrewkroh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ruflin commented Jul 9, 2018

exekias commented Jul 9, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

exekias commented Jul 9, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ruflin commented Jul 11, 2018

exekias commented Jul 11, 2018

Add `bearer_token_file` paramter to HTTP helper #7527

Add `bearer_token_file` paramter to HTTP helper #7527

exekias commented Jul 6, 2018 •

edited

Loading

exekias commented Jul 6, 2018 •

edited

Loading