Metrics Service Support #220

ramaraochavali · 2017-10-31T04:39:52Z

Signed-off-by: Rama ramaraochavali@gmail.com

Signed-off-by: Rama <ramaraochavali@gmail.com>

ramaraochavali · 2017-10-31T05:30:47Z

@mattklein123 Sorry. Had to create a new PR to get around the DCO issues. Here is the original PR #210. Please go through the PR for more details

Signed-off-by: Rama <ramaraochavali@gmail.com>

htuch

Great, thanks for making this happen.

htuch · 2017-10-31T14:20:26Z

bazel/repositories.bzl

+    visibility = ["//visibility:public"],
+)
+
+py_proto_library(


I would skip the Python rules unless someone is going to use them for now. Bazel really should get native py_proto_library support that builds on proto_library to make this cleaner, see bazelbuild/bazel#2626 and bazelbuild/bazel#3935.

How do I skip? Initially I did not have. Then I got into some py related error. So I thought it is mandatory to have py and added this.

htuch · 2017-10-31T14:20:51Z

api/metrics_service.proto

+import "google/api/annotations.proto";
+import "metrics.proto";
+
+//Service to fetch metrics from Proxy. This uses Promotheus data model to represent metrics returned.


Nit: // Service.

Will change

htuch · 2017-10-31T14:22:28Z

api/metrics_service.proto

+service MetricsService {
+  rpc FetchMetrics (MetricsRequest) returns (MetricsResponse) {
+    option (google.api.http) = {
+      post: "/v2/metrics"


FYI, unlike the other services in data-plane-api, this is the first to be served by Envoy rather than have Envoy act as client. I'm wondering if we somehow want to distinguish this namespace wise. Also, can you clarify if this is intended to be a gRPC service, REST or combination?

I do not have strong opinion on putting it in different name space. I was originally thinking of it as a gRPC service. For REST, I think the admin end point with ?format=json is sufficient I guess.

Signed-off-by: Rama <ramaraochavali@gmail.com>

mattklein123 · 2017-10-31T15:56:37Z

@ramaraochavali thanks for bearing with us here. :)

I still think think is a major decision point. Is this
a) A push API that Envoy will call to stream metrics (I would actually vote for this).
b) A pull API that some metrics puller will call to get metrics (what is written).

We should be clear about what we are going after here.

If we do b), I would prefer to do this in the context of https://github.com/envoyproxy/data-plane-api/issues/158 and to define a new admin namespace per @htuch where we can start specifying protos for all the admin output.

Signed-off-by: Rama <ramaraochavali@gmail.com>

ramaraochavali · 2017-10-31T16:04:22Z

@htuch I made the py optional only for metrics and build passes.

ramaraochavali · 2017-10-31T16:13:46Z

@mattklein123 No issues. I see your point. I think a) is more valuable in data-plane-api context. While b) may be good - it is in the context of admin. I think we should go after a). I am actually modifying the service def accordingly. I am just setting bazel stuff trying to compile with promotheus proto with basic service def.

Signed-off-by: Rama <ramaraochavali@gmail.com>

ramaraochavali · 2017-10-31T16:55:58Z

@mattklein123 I have changed it to suit the definition for a). Please review. I also think b) might also have some value but we can do it under #158 may be if needed.

mattklein123

LGTM. I think this is a useful addition. @rshriram et al, does this look ok from Istio side? Anyone else have any thoughts on this?

mattklein123 · 2017-10-31T21:09:21Z

api/metrics_service.proto

+// Service for streaming metrics to connected end point.
+service MetricsService {
+  rpc StreamMetrics(stream StreamMetricsMessage) returns (StreamMetricsResponse) {
+


mega nit: I would del newline.

rshriram

This is generally useful. Would like some opinion from @douglas-reid or @ZackButcher

rshriram · 2017-10-31T22:07:35Z

api/metrics_service.proto

+import "metrics.proto";
+
+// Service for streaming metrics to connected end point.
+service MetricsService {


Does it make sense to explicitly call out PrometheusMetricsService ? After all the format is for Prometheus. If not this, atleast have a type field or something in streammessage that says what type of metric is coming out of Envoy.

I think we discussed it in the PR #210 that we do not want to call this PrometheusMetricsService. We picked Promotheus model because it is more comprehensive representation of metrics model. We tried to look at OpenMetrics but since it is far off and any way is heavily inspired from Promotheus model. So we decided to go with this model for now.

rshriram · 2017-10-31T22:08:19Z

api/metrics_service.proto

+message StreamMetricsResponse {}
+
+message StreamMetricsMessage {
+  repeated io.prometheus.client.MetricFamily proxy_metrics = 1;


envoy_metrics. We don’t care if this is proxy or sidecar or LB or anything :).

will change

douglas-reid · 2017-10-31T23:05:00Z

@rshriram et al. I will look tonight. Am currently OOO. On Oct 31, 2017 3:09 PM, "Shriram Rajagopalan" <notifications@github.com> wrote: *@rshriram* approved this pull request. This is generally useful. Would like some opinion from @douglas-reid <https://github.com/douglas-reid> or @ZackButcher <https://github.com/zackbutcher> ------------------------------ In api/metrics_service.proto <#220 (comment)> :

@@ -0,0 +1,19 @@

+syntax = "proto3"; + +package envoy.api.v2; + +import "google/api/annotations.proto"; +import "metrics.proto"; + +// Service for streaming metrics to connected end point. +service MetricsService { Does it make sense to explicitly call out PrometheusMetricsService ? After all the format is for Prometheus. If not this, atleast have a type field or something in streammessage that says what type of metric is coming out of Envoy. ------------------------------ In api/metrics_service.proto <#220 (comment)> :

+package envoy.api.v2;

+ +import "google/api/annotations.proto" ; +import "metrics.proto"; + +// Service for streaming metrics to connected end point. +service MetricsService { + rpc StreamMetrics(stream StreamMetricsMessage) returns (StreamMetricsResponse) { + + } +} + +message StreamMetricsResponse {} + +message StreamMetricsMessage { + repeated io.prometheus.client.MetricFamily proxy_metrics = 1; envoy_metrics. We don’t care if this is proxy or sidecar or LB or anything :). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#220 (review)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AUKx3c-kWdxHz0ENqXKTT16HQPGZWz9lks5sx5qUgaJpZM4QMMtw> .

Signed-off-by: Rama <ramaraochavali@gmail.com>

ramaraochavali · 2017-11-01T04:00:11Z

@mattklein123 do we need identifier like Node to be sent along with this so that server identifies the node from which the metrics are coming from?
Another couple of points

I skipped python rules for this as per @htuch comments above.
I also did not add this to go_protos and go_grpc? Do I have to? If I have to add, I need to find a way to reference "promotheus" proto there.

ramaraochavali · 2017-11-01T06:06:09Z

I enabled go compilation and am running in to the following build problem
ERROR: /home/rama.rao/git/data-plane-api/api/BUILD:38:1: GoCompile api/~~normal~~go_default_library~/api.a failed (Exit 1)
2017/11/01 11:29:55 missing strict dependencies:
bazel-out/local-fastbuild/bin/api/go_default_library/api/metrics_service.pb.go: import of ., which is not a direct dependency
INFO: Elapsed time: 16.925s, Critical Path: 3.11s
FAILED: Build did NOT complete successfully
and in the metrics_service.pb.go has the following import which is causing trouble
import io_prometheus_client "."
Any ideas on how to get around this?
@htuch do you know how to get around this? It looks a bazel rule issue with protoc generator with go..

htuch

Yes, I think we should Node as an optional field to be included in the first field, for consistency with other APIs. Looks good other than remaining comments.

htuch · 2017-11-01T13:46:38Z

api/metrics_service.proto

+import "google/api/annotations.proto";
+import "metrics.proto";
+
+// Service for streaming metrics to connected end point.


Can you please change the wording here to describe in more detail what the endpoint is, and also update README.md in the root directory to describe this new service that we are connecting to?

Will change..

Signed-off-by: Rama <ramaraochavali@gmail.com>

douglas-reid

This looks reasonable for Istio usage.

douglas-reid · 2017-11-01T16:09:28Z

bazel/repositories.bzl

@@ -1,4 +1,5 @@
 GOOGLEAPIS_SHA = "5c6df0cd18c6a429eab739fb711c27f6e1393366" # May 14, 2017
+PROMETHEUS_SHA = "6f3806018612930941127f2a7c6c453ba2c527d2"


nit: might be worth annotating with date.

mattklein123 · 2017-11-01T17:07:43Z

api/metrics_service.proto

+import "google/api/annotations.proto";
+import "metrics.proto";
+
+// Service for streaming metrics to server that consumes the metrics data. It uses Prometheus metric data model as a standard to represent metrics information.


nit: please line break around 100 cols

mattklein123 · 2017-11-01T17:08:10Z

api/metrics_service.proto

+
+message StreamMetricsMessage {
+  // The node sending the metric messages over the stream.
+  Node node = 1;


Can you take a look at my access log API and copy what we did there with Identifier? We only need to send node once.

mattklein123 · 2017-11-01T17:08:35Z

api/metrics_service.proto

+import "metrics.proto";
+
+// Service for streaming metrics to server that consumes the metrics data. It uses Prometheus metric data model as a standard to represent metrics information.
+service MetricsService {


You will also need to provide a config struct with at least cluster in it. See access log API.

Signed-off-by: Rama <ramaraochavali@gmail.com>

ramaraochavali · 2017-11-02T04:04:55Z

@mattklein123 I think addressed all comments and should be good to go. Can you please look at it?

mattklein123

LGTM. Thank you! Any final comments from anyone else?

rshriram · 2017-11-02T22:04:12Z

Lgtm.

…

On Thu, Nov 2, 2017 at 12:29 PM Matt Klein ***@***.***> wrote: ***@***.**** approved this pull request. LGTM. Thank you! Any final comments from anyone else? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#220 (review)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH0qd4H1zGcTE0rv6SLGoOnacK6hBtHRks5sye34gaJpZM4QMMtw> .

htuch · 2017-11-03T02:50:54Z

api/metrics_service.proto

@@ -0,0 +1,39 @@
+syntax = "proto3";
+
+package envoy.api.v2;


Nit: blank line after package.

htuch · 2017-11-03T02:52:05Z

api/metrics_service.proto

+  }
+
+  // Identifier data that will only be sent in the first message on the stream. This is effectively
+  // structured metadata and is a performance optimization.


Nit: this should probably be two sentences, the conjunction of the description of metadata and then the performance optimization note doesn't make sense.

Signed-off-by: Rama <ramaraochavali@gmail.com>

initial metrics_service proto

f0e2359

Signed-off-by: Rama <ramaraochavali@gmail.com>

ramaraochavali added 4 commits October 31, 2017 11:01

added tests

ac39c33

Signed-off-by: Rama <ramaraochavali@gmail.com>

added support for bazel build of promotheus proto

2380719

Signed-off-by: Rama <ramaraochavali@gmail.com>

added support for bazel build of promotheus proto

0edcce8

Signed-off-by: Rama <ramaraochavali@gmail.com>

added support for bazel build of promotheus proto

3875eab

Signed-off-by: Rama <ramaraochavali@gmail.com>

htuch reviewed Oct 31, 2017

View reviewed changes

ramaraochavali added 8 commits October 31, 2017 20:07

added support for bazel build of promotheus proto

c4fef88

Signed-off-by: Rama <ramaraochavali@gmail.com>

added support for bazel build of promotheus proto

5123296

Signed-off-by: Rama <ramaraochavali@gmail.com>

removed proto py rule

5f5eabb

Signed-off-by: Rama <ramaraochavali@gmail.com>

skip py proto

92c0331

Signed-off-by: Rama <ramaraochavali@gmail.com>

fully qualified MetricsFamily

beae9e5

Signed-off-by: Rama <ramaraochavali@gmail.com>

add dep for go_grpc

a868271

Signed-off-by: Rama <ramaraochavali@gmail.com>

removed go support

f15c7da

Signed-off-by: Rama <ramaraochavali@gmail.com>

removed go support

7804dc6

Signed-off-by: Rama <ramaraochavali@gmail.com>

make py optional only for metrics

bf2238c

Signed-off-by: Rama <ramaraochavali@gmail.com>

changed to streaming

e473c29

Signed-off-by: Rama <ramaraochavali@gmail.com>

mattklein123 reviewed Oct 31, 2017

View reviewed changes

rshriram approved these changes Oct 31, 2017

View reviewed changes

review comments

e3bfd40

Signed-off-by: Rama <ramaraochavali@gmail.com>

htuch reviewed Nov 1, 2017

View reviewed changes

review comments

1519e6b

Signed-off-by: Rama <ramaraochavali@gmail.com>

douglas-reid approved these changes Nov 1, 2017

View reviewed changes

mattklein123 reviewed Nov 1, 2017

View reviewed changes

review comments

9040f93

Signed-off-by: Rama <ramaraochavali@gmail.com>

mattklein123 approved these changes Nov 2, 2017

View reviewed changes

htuch approved these changes Nov 3, 2017

View reviewed changes

review comments

d753f43

Signed-off-by: Rama <ramaraochavali@gmail.com>

mattklein123 approved these changes Nov 3, 2017

View reviewed changes

mattklein123 merged commit c54d0de into envoyproxy:master Nov 3, 2017

htuch mentioned this pull request Mar 7, 2018

Documentation for hystrix dashboard support feature (issue #1244) #523

Closed

		@@ -1,4 +1,5 @@
		GOOGLEAPIS_SHA = "5c6df0cd18c6a429eab739fb711c27f6e1393366" # May 14, 2017
		PROMETHEUS_SHA = "6f3806018612930941127f2a7c6c453ba2c527d2"

Metrics Service Support #220

Metrics Service Support #220

Conversation

ramaraochavali commented Oct 31, 2017

ramaraochavali commented Oct 31, 2017

htuch left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mattklein123 commented Oct 31, 2017

ramaraochavali commented Oct 31, 2017

ramaraochavali commented Oct 31, 2017

ramaraochavali commented Oct 31, 2017

mattklein123 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rshriram left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

douglas-reid commented Oct 31, 2017 via email

ramaraochavali commented Nov 1, 2017

ramaraochavali commented Nov 1, 2017 • edited Loading

htuch left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

douglas-reid left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ramaraochavali commented Nov 2, 2017

mattklein123 left a comment

Choose a reason for hiding this comment

rshriram commented Nov 2, 2017 via email

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ramaraochavali commented Nov 1, 2017 •

edited

Loading