Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Metrics Service Support #220

Merged
merged 19 commits into from
Nov 3, 2017
Merged

Metrics Service Support #220

Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
f0e2359
initial metrics_service proto
ramaraochavali Oct 31, 2017
ac39c33
added tests
ramaraochavali Oct 31, 2017
2380719
added support for bazel build of promotheus proto
ramaraochavali Oct 31, 2017
0edcce8
added support for bazel build of promotheus proto
ramaraochavali Oct 31, 2017
3875eab
added support for bazel build of promotheus proto
ramaraochavali Oct 31, 2017
c4fef88
added support for bazel build of promotheus proto
ramaraochavali Oct 31, 2017
5123296
added support for bazel build of promotheus proto
ramaraochavali Oct 31, 2017
5f5eabb
removed proto py rule
ramaraochavali Oct 31, 2017
92c0331
skip py proto
ramaraochavali Oct 31, 2017
beae9e5
fully qualified MetricsFamily
ramaraochavali Oct 31, 2017
a868271
add dep for go_grpc
ramaraochavali Oct 31, 2017
f15c7da
removed go support
ramaraochavali Oct 31, 2017
7804dc6
removed go support
ramaraochavali Oct 31, 2017
bf2238c
make py optional only for metrics
ramaraochavali Oct 31, 2017
e473c29
changed to streaming
ramaraochavali Oct 31, 2017
e3bfd40
review comments
ramaraochavali Nov 1, 2017
1519e6b
review comments
ramaraochavali Nov 1, 2017
9040f93
review comments
ramaraochavali Nov 2, 2017
d753f43
review comments
ramaraochavali Nov 3, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ Unless otherwise stated, the APIs with the same names as v1 APIs have a similar
endpoints. The health check subset may not be a subset of the Envoy instance's
EDS endpoints.
* [Listener Discovery Service (LDS)](api/lds.proto). This new API supports dynamic discovery of the listener configuration (which ports to bind to, TLS details, filter chains, etc.).
* [Metric Service (MS)](api/metrics_service.proto). This new API allows Envoy to push (stream) metrics forever for servers to consume.
* [Rate Limit Service (RLS)](api/rls.proto)
* [Route Discovery Service (RDS)](api/rds.proto).
* [Secret Discovery Service (SDS)](api/sds.proto).
Expand Down
11 changes: 11 additions & 0 deletions api/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,17 @@ api_proto_library(
],
)

api_proto_library(
name = "metrics",
srcs = ["metrics_service.proto"],
has_services = 1,
deps = [
":base",
"@promotheus_metrics_model//:client_model_protos_lib",
],
require_py = 0,
)

api_proto_library(
name = "protocol",
srcs = ["protocol.proto"],
Expand Down
23 changes: 23 additions & 0 deletions api/metrics_service.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
syntax = "proto3";

package envoy.api.v2;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: blank line after package.

import "api/base.proto";

import "google/api/annotations.proto";
import "metrics.proto";

// Service for streaming metrics to server that consumes the metrics data. It uses Prometheus metric data model as a standard to represent metrics information.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: please line break around 100 cols

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will do

service MetricsService {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to explicitly call out PrometheusMetricsService ? After all the format is for Prometheus. If not this, atleast have a type field or something in streammessage that says what type of metric is coming out of Envoy.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we discussed it in the PR #210 that we do not want to call this PrometheusMetricsService. We picked Promotheus model because it is more comprehensive representation of metrics model. We tried to look at OpenMetrics but since it is far off and any way is heavily inspired from Promotheus model. So we decided to go with this model for now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You will also need to provide a config struct with at least cluster in it. See access log API.

// Envoy will connect and send StreamMetricsMessage messages forever. It does not expect any
// response to be sent as nothing would be done in the case of failure.
rpc StreamMetrics(stream StreamMetricsMessage) returns (StreamMetricsResponse) {
}
}

message StreamMetricsResponse {}

message StreamMetricsMessage {
// The node sending the metric messages over the stream.
Node node = 1;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you take a look at my access log API and copy what we did there with Identifier? We only need to send node once.

repeated io.prometheus.client.MetricFamily envoy_metrics = 2;
}
5 changes: 3 additions & 2 deletions bazel/api_build_system.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ def api_py_proto_library(name, srcs = [], deps = [], has_services = 0):

# TODO(htuch): has_services is currently ignored but will in future support
# gRPC stub generation.
def api_proto_library(name, srcs = [], deps = [], has_services = 0):
def api_proto_library(name, srcs = [], deps = [], has_services = 0, require_py = 1):
native.proto_library(
name = name,
srcs = srcs,
Expand All @@ -43,7 +43,8 @@ def api_proto_library(name, srcs = [], deps = [], has_services = 0):
deps = [name],
visibility = ["//visibility:public"],
)
api_py_proto_library(name, srcs, deps, has_services)
if (require_py == 1):
api_py_proto_library(name, srcs, deps, has_services)

def api_cc_test(name, srcs, proto_deps):
native.cc_test(
Expand Down
29 changes: 29 additions & 0 deletions bazel/repositories.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
GOOGLEAPIS_SHA = "5c6df0cd18c6a429eab739fb711c27f6e1393366" # May 14, 2017
PROMETHEUS_SHA = "6f3806018612930941127f2a7c6c453ba2c527d2"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: might be worth annotating with date.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will do


def api_dependencies():
native.new_http_archive(
Expand Down Expand Up @@ -44,3 +45,31 @@ py_proto_library(
)
""",
)

native.new_http_archive(
name = "promotheus_metrics_model",
strip_prefix = "client_model-" + PROMETHEUS_SHA,
url = "https://github.com/prometheus/client_model/archive/" + PROMETHEUS_SHA + ".tar.gz",
build_file_content = """

filegroup(
name = "client_model_protos_src",
srcs = [
"metrics.proto",
],
visibility = ["//visibility:public"],
)

proto_library(
name = "client_model_protos_lib",
srcs = [":client_model_protos_src"],
visibility = ["//visibility:public"],
)

cc_proto_library(
name = "client_model_protos",
deps = [":client_model_protos_lib"],
visibility = ["//visibility:public"],
)
""",
)
1 change: 1 addition & 0 deletions test/build/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ api_cc_test(
"//api:eds",
"//api:hds",
"//api:lds",
"//api:metrics",
"//api:rds",
"//api:rls",
],
Expand Down
1 change: 1 addition & 0 deletions test/build/build_test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ int main(int argc, char *argv[]) {
"envoy.api.v2.HealthDiscoveryService.StreamHealthCheck",
"envoy.api.v2.ListenerDiscoveryService.FetchListeners",
"envoy.api.v2.ListenerDiscoveryService.StreamListeners",
"envoy.api.v2.MetricsService.StreamMetrics",
"envoy.api.v2.RouteDiscoveryService.FetchRoutes",
"envoy.api.v2.RouteDiscoveryService.StreamRoutes",
"envoy.api.v2.RateLimitService.ShouldRateLimit",
Expand Down