Merge branch 'master' into redacted_protobuf

Signed-off-by: Dan Rosen <mergeconflict@google.com>
envoyproxy · Jan 15, 2020 · 358bac9 · 358bac9
2 parents 5f1211e + 156d7c9
commit 358bac9
Show file tree

Hide file tree

Showing 95 changed files with 447 additions and 218 deletions.
diff --git a/api/API_VERSIONING.md b/api/API_VERSIONING.md
@@ -76,7 +76,7 @@ implementations within a major version should set explicit values for these fiel
 
 # API lifecycle
 
-The API lifecycle follows a calendar clock. At the end of Q3 each year, a major API version
+The API lifecycle follows a calendar clock. At the end of Q4 each year, a major API version
 increment may occur for any Envoy API package, in concert with the quarterly Envoy release.
 
 Envoy will support at most three major versions of any API package at all times:
@@ -134,17 +134,9 @@ methods, depending on whether the change is mechanical or manual.
 
 ## Mechanical breaking changes
 
-Field deprecations, renames, etc. are mechanical changes that will be supported by the
+Field deprecations, renames, etc. are mechanical changes that are supported by the
 [protoxform](https://github.com/envoyproxy/envoy/tree/master/tools/protoxform) tool. These are
-guided by annotations in protobuf.
-* Deprecations are specified with the built-in protobuf deprecated option set on a message, enum,
-  field or enum value. No field may be marked as deprecated unless a replacement for this
-  functionality exists and the corresponding Envoy implementation is production ready.
-
-* Renames are specified with a `[(udpa.annotations.field_migrate).rename = "<new name>"]` annotation.
-
-* We anticipate that `protoxform` will also support `oneof` promotion, package movement, etc. via
-  similar annotations.
+guided by [annotations](STYLE.md#api-annotations).
 
 ## Manual breaking changes
 

diff --git a/api/STYLE.md b/api/STYLE.md
@@ -113,3 +113,67 @@ for services and configs should be `//docs` (proto documentation tool).
 
 Extensions should use the regular hierarchy. For example, configuration for network filters belongs
 in a package under `envoy.config.filter.network`.
+
+## Adding an extension configuration to the API
+
+Extensions must currently be added as v2 APIs following the [package
+organization](#package-organization) above.
+To add an extension config to the API, the steps below should be followed:
+
+1. Place the v2 extension configuration `.proto` in `api/config`, e.g.
+   `api/config/filter/http/foobar/v2/foobar.proto` together with an initial BUILD file:
+   ```
+   load("@envoy_api//bazel:api_build_system.bzl", "api_proto_package")
+
+   licenses(["notice"])  # Apache 2
+
+   api_proto_package(
+       deps = ["@com_github_cncf_udpa//udpa/annotations:pkg"],
+       )
+   ```
+1. Add to the v2 extension config proto `import "udpa/annotations/migrate.proto";`
+2. Add to the v2 extension config proto a package level `option (udpa.annotations.file_migrate).move_to_package = "envoy.extensions.filters.http.foobar.v3alpha";`.
+   This places the filter in the correct [v3 package hierarchy](#package-organization).
+3. Add a reference to the v2 extension config in (1) in [api/docs/BUILD](docs/BUILD).
+4. Run `./tools/proto_format fix`. This should regenerate the `BUILD` file,
+   reformat `foobar.proto` as needed and also generate the v3 extension config,
+   together with shadow API protos.
+4. `git add api/ generated_api_shadow/` to add any new files to your Git index.
+
+## API annotations
+
+A number of annotations are used in the Envoy APIs to provide additional API
+metadata. We describe these annotations below by category.
+
+### Field level
+* `[deprecated = true]` to denote fields that are deprecated in a major version.
+  These fields are slated for removal at the next major cycle and follow the
+  [breaking change policy](../CONTRIBUTING.md#breaking-change-policy).
+* `[envoy.annotations.disallowed_by_default = true]` to denote fields that have
+  been disallowed by default as per the [breaking change policy](../CONTRIBUTING.md#breaking-change-policy).
+* `[(udpa.annotations.field_migrate).rename = "<new field name>"]` to denote that
+  the field will be renamed to a given name in the next API major version.
+* `[(udpa.annotations.field_migrate).oneof_promotion = "<oneof name>"]` to denote that
+  the field will be promoted to a given `oneof` in the next API major version.
+* `[(udpa.annotations.sensitive) = true]` to denote sensitive fields that
+  should be redacted in output such as logging or configuration dumps.
+* [PGV annotations](https://github.com/envoyproxy/protoc-gen-validate) to denote field
+  value constraints.
+
+### Enum value level
+* `[(udpa.annotations.enum_value_migrate).rename = "new enum value name"]` to denote that
+  the enum value will be renamed to a given name in the next API major version.
+
+### Message level
+* `option (udpa.annotations.versioning).previous_message_type = "<message type
+  name>";` to denote the previous type name for an upgraded message. You should
+  never have to write these manually, they are generated by `protoxform`.
+
+### Service level
+* `option (envoy.annotations.resource).type = "<resource type name>";` to denote
+  the resource type for an xDS service definition.
+
+### File level
+* `option (udpa.annotations.file_migrate).move_to_package = "<package name>";`
+  to denote that in the next major version of the API, the file will be moved to
+  the given package. This is consumed by `protoxform`.
diff --git a/api/envoy/admin/v2alpha/BUILD b/api/envoy/admin/v2alpha/BUILD
@@ -6,6 +6,7 @@ licenses(["notice"])  # Apache 2
 
 api_proto_package(
     deps = [
+        "//envoy/annotations:pkg",
         "//envoy/api/v2/core:pkg",
         "//envoy/config/bootstrap/v2:pkg",
         "//envoy/service/tap/v2alpha:pkg",

diff --git a/api/envoy/admin/v2alpha/server_info.proto b/api/envoy/admin/v2alpha/server_info.proto
@@ -4,6 +4,8 @@ package envoy.admin.v2alpha;
 
 import "google/protobuf/duration.proto";
 
+import "envoy/annotations/deprecation.proto";
+
 option java_package = "io.envoyproxy.envoy.admin.v2alpha";
 option java_outer_classname = "ServerInfoProto";
 option java_multiple_files = true;
@@ -128,9 +130,10 @@ message CommandLineOptions {
   Mode mode = 19;
 
   // max_stats and max_obj_name_len are now unused and have no effect.
-  uint64 max_stats = 20 [deprecated = true];
+  uint64 max_stats = 20 [deprecated = true, (envoy.annotations.disallowed_by_default) = true];
 
-  uint64 max_obj_name_len = 21 [deprecated = true];
+  uint64 max_obj_name_len = 21
+      [deprecated = true, (envoy.annotations.disallowed_by_default) = true];
 
   // See :option:`--disable-hot-restart` for details.
   bool disable_hot_restart = 22;

diff --git a/api/envoy/admin/v3alpha/BUILD b/api/envoy/admin/v3alpha/BUILD
@@ -7,6 +7,7 @@ licenses(["notice"])  # Apache 2
 api_proto_package(
     deps = [
         "//envoy/admin/v2alpha:pkg",
+        "//envoy/annotations:pkg",
         "//envoy/config/bootstrap/v3alpha:pkg",
         "//envoy/config/core/v3alpha:pkg",
         "//envoy/config/tap/v3alpha:pkg",

diff --git a/api/envoy/admin/v3alpha/server_info.proto b/api/envoy/admin/v3alpha/server_info.proto
@@ -6,6 +6,8 @@ import "google/protobuf/duration.proto";
 
 import "udpa/annotations/versioning.proto";
 
+import "envoy/annotations/deprecation.proto";
+
 option java_package = "io.envoyproxy.envoy.admin.v3alpha";
 option java_outer_classname = "ServerInfoProto";
 option java_multiple_files = true;

diff --git a/api/envoy/annotations/deprecation.proto b/api/envoy/annotations/deprecation.proto
@@ -0,0 +1,21 @@
+syntax = "proto3";
+
+package envoy.annotations;
+
+import "google/protobuf/descriptor.proto";
+
+// Allows tagging proto fields as fatal by default. One Envoy release after
+// deprecation, deprecated fields will be disallowed by default, a state which
+// is reversible with :ref:`runtime overrides <config_runtime_deprecation>`.
+
+// Magic number in this file derived from top 28bit of SHA256 digest of
+// "envoy.annotation.disallowed_by_default"
+extend google.protobuf.FieldOptions {
+  bool disallowed_by_default = 189503207;
+}
+
+// Magic number in this file derived from top 28bit of SHA256 digest of
+// "envoy.annotation.disallowed_by_default_enum"
+extend google.protobuf.EnumValueOptions {
+  bool disallowed_by_default_enum = 70100853;
+}
diff --git a/api/envoy/api/v2/core/BUILD b/api/envoy/api/v2/core/BUILD
@@ -6,6 +6,7 @@ licenses(["notice"])  # Apache 2
 
 api_proto_package(
     deps = [
+        "//envoy/annotations:pkg",
         "//envoy/type:pkg",
         "//envoy/type/matcher:pkg",
         "@com_github_cncf_udpa//udpa/annotations:pkg",

diff --git a/api/envoy/api/v2/core/config_source.proto b/api/envoy/api/v2/core/config_source.proto
@@ -7,6 +7,7 @@ import "envoy/api/v2/core/grpc_service.proto";
 import "google/protobuf/duration.proto";
 import "google/protobuf/wrappers.proto";
 
+import "envoy/annotations/deprecation.proto";
 import "udpa/annotations/migrate.proto";
 import "validate/validate.proto";
 
@@ -40,7 +41,8 @@ message ApiConfigSource {
   enum ApiType {
     // Ideally this would be 'reserved 0' but one can't reserve the default
     // value. Instead we throw an exception if this is ever used.
-    UNSUPPORTED_REST_LEGACY = 0 [deprecated = true];
+    UNSUPPORTED_REST_LEGACY = 0
+        [deprecated = true, (envoy.annotations.disallowed_by_default_enum) = true];
 
     // REST-JSON v2 API. The `canonical JSON encoding
     // <https://developers.google.com/protocol-buffers/docs/proto3#json>`_ for

diff --git a/api/envoy/api/v2/route/BUILD b/api/envoy/api/v2/route/BUILD
@@ -6,6 +6,7 @@ licenses(["notice"])  # Apache 2
 
 api_proto_package(
     deps = [
+        "//envoy/annotations:pkg",
         "//envoy/api/v2/core:pkg",
         "//envoy/type:pkg",
         "//envoy/type/matcher:pkg",

diff --git a/api/envoy/api/v2/route/route_components.proto b/api/envoy/api/v2/route/route_components.proto
@@ -14,6 +14,7 @@ import "google/protobuf/duration.proto";
 import "google/protobuf/struct.proto";
 import "google/protobuf/wrappers.proto";
 
+import "envoy/annotations/deprecation.proto";
 import "udpa/annotations/migrate.proto";
 import "validate/validate.proto";
 
@@ -494,7 +495,8 @@ message CorsPolicy {
     //
     //   **This field is deprecated**. Set the
     //   :ref:`filter_enabled<envoy_api_field_route.CorsPolicy.filter_enabled>` field instead.
-    google.protobuf.BoolValue enabled = 7 [deprecated = true];
+    google.protobuf.BoolValue enabled = 7
+        [deprecated = true, (envoy.annotations.disallowed_by_default) = true];
 
     // Specifies the % of requests for which the CORS filter is enabled.
     //
@@ -558,7 +560,7 @@ message RouteAction {
     //   **This field is deprecated**. Set the
     //   :ref:`runtime_fraction
     //   <envoy_api_field_route.RouteAction.RequestMirrorPolicy.runtime_fraction>` field instead.
-    string runtime_key = 2 [deprecated = true];
+    string runtime_key = 2 [deprecated = true, (envoy.annotations.disallowed_by_default) = true];
 
     // If both :ref:`runtime_key
     // <envoy_api_field_route.RouteAction.RequestMirrorPolicy.runtime_key>` and this field are not

diff --git a/api/envoy/config/bootstrap/v2/BUILD b/api/envoy/config/bootstrap/v2/BUILD
@@ -6,6 +6,7 @@ licenses(["notice"])  # Apache 2
 
 api_proto_package(
     deps = [
+        "//envoy/annotations:pkg",
         "//envoy/api/v2:pkg",
         "//envoy/api/v2/auth:pkg",
         "//envoy/api/v2/core:pkg",

diff --git a/api/envoy/config/bootstrap/v2/bootstrap.proto b/api/envoy/config/bootstrap/v2/bootstrap.proto
@@ -16,6 +16,7 @@ import "google/protobuf/duration.proto";
 import "google/protobuf/struct.proto";
 import "google/protobuf/wrappers.proto";
 
+import "envoy/annotations/deprecation.proto";
 import "validate/validate.proto";
 
 option java_package = "io.envoyproxy.envoy.config.bootstrap.v2";
@@ -119,7 +120,7 @@ message Bootstrap {
   // Configuration for the runtime configuration provider (deprecated). If not
   // specified, a “null” provider will be used which will result in all defaults
   // being used.
-  Runtime runtime = 11 [deprecated = true];
+  Runtime runtime = 11 [deprecated = true, (envoy.annotations.disallowed_by_default) = true];
 
   // Configuration for the runtime configuration provider. If not
   // specified, a “null” provider will be used which will result in all defaults

diff --git a/api/envoy/config/bootstrap/v3alpha/BUILD b/api/envoy/config/bootstrap/v3alpha/BUILD
@@ -6,6 +6,7 @@ licenses(["notice"])  # Apache 2
 
 api_proto_package(
     deps = [
+        "//envoy/annotations:pkg",
         "//envoy/config/bootstrap/v2:pkg",
         "//envoy/config/cluster/v3alpha:pkg",
         "//envoy/config/core/v3alpha:pkg",

diff --git a/api/envoy/config/bootstrap/v3alpha/bootstrap.proto b/api/envoy/config/bootstrap/v3alpha/bootstrap.proto
@@ -18,6 +18,7 @@ import "google/protobuf/wrappers.proto";
 
 import "udpa/annotations/versioning.proto";
 
+import "envoy/annotations/deprecation.proto";
 import "validate/validate.proto";
 
 option java_package = "io.envoyproxy.envoy.config.bootstrap.v3alpha";

diff --git a/api/envoy/config/core/v3alpha/BUILD b/api/envoy/config/core/v3alpha/BUILD
@@ -6,6 +6,7 @@ licenses(["notice"])  # Apache 2
 
 api_proto_package(
     deps = [
+        "//envoy/annotations:pkg",
         "//envoy/api/v2/core:pkg",
         "//envoy/type/matcher/v3alpha:pkg",
         "//envoy/type/v3alpha:pkg",

diff --git a/api/envoy/config/core/v3alpha/config_source.proto b/api/envoy/config/core/v3alpha/config_source.proto
@@ -9,6 +9,7 @@ import "google/protobuf/wrappers.proto";
 
 import "udpa/annotations/versioning.proto";
 
+import "envoy/annotations/deprecation.proto";
 import "validate/validate.proto";
 
 option java_package = "io.envoyproxy.envoy.config.core.v3alpha";
@@ -42,7 +43,8 @@ message ApiConfigSource {
   enum ApiType {
     // Ideally this would be 'reserved 0' but one can't reserve the default
     // value. Instead we throw an exception if this is ever used.
-    DEPRECATED_AND_UNAVAILABLE_DO_NOT_USE = 0 [deprecated = true];
+    DEPRECATED_AND_UNAVAILABLE_DO_NOT_USE = 0
+        [deprecated = true, (envoy.annotations.disallowed_by_default_enum) = true];
 
     // REST-JSON v2 API. The `canonical JSON encoding
     // <https://developers.google.com/protocol-buffers/docs/proto3#json>`_ for

diff --git a/api/envoy/config/filter/fault/v2/BUILD b/api/envoy/config/filter/fault/v2/BUILD
@@ -6,6 +6,7 @@ licenses(["notice"])  # Apache 2
 
 api_proto_package(
     deps = [
+        "//envoy/annotations:pkg",
         "//envoy/type:pkg",
         "@com_github_cncf_udpa//udpa/annotations:pkg",
     ],

diff --git a/api/envoy/config/filter/fault/v2/fault.proto b/api/envoy/config/filter/fault/v2/fault.proto
@@ -6,6 +6,7 @@ import "envoy/type/percent.proto";
 
 import "google/protobuf/duration.proto";
 
+import "envoy/annotations/deprecation.proto";
 import "udpa/annotations/migrate.proto";
 import "validate/validate.proto";
 
@@ -35,7 +36,7 @@ message FaultDelay {
   reserved 2;
 
   // Unused and deprecated. Will be removed in the next release.
-  FaultDelayType type = 1 [deprecated = true];
+  FaultDelayType type = 1 [deprecated = true, (envoy.annotations.disallowed_by_default) = true];
 
   oneof fault_delay_secifier {
     option (validate.required) = true;

diff --git a/api/envoy/config/filter/http/ext_authz/v2/BUILD b/api/envoy/config/filter/http/ext_authz/v2/BUILD
@@ -6,6 +6,7 @@ licenses(["notice"])  # Apache 2
 
 api_proto_package(
     deps = [
+        "//envoy/annotations:pkg",
         "//envoy/api/v2/core:pkg",
         "//envoy/type:pkg",
         "//envoy/type/matcher:pkg",

diff --git a/api/envoy/config/filter/http/ext_authz/v2/ext_authz.proto b/api/envoy/config/filter/http/ext_authz/v2/ext_authz.proto
@@ -8,6 +8,7 @@ import "envoy/api/v2/core/http_uri.proto";
 import "envoy/type/http_status.proto";
 import "envoy/type/matcher/string.proto";
 
+import "envoy/annotations/deprecation.proto";
 import "udpa/annotations/migrate.proto";
 import "validate/validate.proto";
 
@@ -50,7 +51,7 @@ message ExtAuthz {
   // useful when transitioning from alpha to release versions assuming that both definitions are
   // semantically compatible. Deprecation note: This field is deprecated and should only be used for
   // version upgrade. See release notes for more details.
-  bool use_alpha = 4 [deprecated = true];
+  bool use_alpha = 4 [deprecated = true, (envoy.annotations.disallowed_by_default) = true];
 
   // Enables filter to buffer the client request body and send it within the authorization request.
   // A ``x-envoy-auth-partial-body: false|true`` metadata header will be added to the authorization

diff --git a/api/envoy/config/filter/network/redis_proxy/v2/BUILD b/api/envoy/config/filter/network/redis_proxy/v2/BUILD
@@ -6,6 +6,7 @@ licenses(["notice"])  # Apache 2
 
 api_proto_package(
     deps = [
+        "//envoy/annotations:pkg",
         "//envoy/api/v2/core:pkg",
         "@com_github_cncf_udpa//udpa/annotations:pkg",
     ],

diff --git a/api/envoy/config/filter/network/redis_proxy/v2/redis_proxy.proto b/api/envoy/config/filter/network/redis_proxy/v2/redis_proxy.proto
@@ -7,6 +7,7 @@ import "envoy/api/v2/core/base.proto";
 import "google/protobuf/duration.proto";
 import "google/protobuf/wrappers.proto";
 
+import "envoy/annotations/deprecation.proto";
 import "udpa/annotations/migrate.proto";
 import "validate/validate.proto";
 
@@ -162,7 +163,8 @@ message RedisProxy {
     //   This field is deprecated. Use a :ref:`catch_all
     //   route<envoy_api_field_config.filter.network.redis_proxy.v2.RedisProxy.PrefixRoutes.catch_all_route>`
     //   instead.
-    string catch_all_cluster = 3 [deprecated = true];
+    string catch_all_cluster = 3
+        [deprecated = true, (envoy.annotations.disallowed_by_default) = true];
 
     // Optional catch-all route to forward commands that doesn't match any of the routes. The
     // catch-all route becomes required when no routes are specified.
@@ -181,7 +183,7 @@ message RedisProxy {
   //   This field is deprecated. Use a :ref:`catch_all
   //   route<envoy_api_field_config.filter.network.redis_proxy.v2.RedisProxy.PrefixRoutes.catch_all_route>`
   //   instead.
-  string cluster = 2 [deprecated = true];
+  string cluster = 2 [deprecated = true, (envoy.annotations.disallowed_by_default) = true];
 
   // Network settings for the connection pool to the upstream clusters.
   ConnPoolSettings settings = 3 [(validate.rules).message = {required: true}];

diff --git a/api/envoy/config/route/v3alpha/BUILD b/api/envoy/config/route/v3alpha/BUILD
@@ -6,6 +6,7 @@ licenses(["notice"])  # Apache 2
 
 api_proto_package(
     deps = [
+        "//envoy/annotations:pkg",
         "//envoy/api/v2:pkg",
         "//envoy/api/v2/route:pkg",
         "//envoy/config/core/v3alpha:pkg",

diff --git a/api/envoy/config/route/v3alpha/route_components.proto b/api/envoy/config/route/v3alpha/route_components.proto
@@ -16,6 +16,7 @@ import "google/protobuf/wrappers.proto";
 
 import "udpa/annotations/versioning.proto";
 
+import "envoy/annotations/deprecation.proto";
 import "validate/validate.proto";
 
 option java_package = "io.envoyproxy.envoy.config.route.v3alpha";