Separate direct and hierarchical policy types

Signed-off-by: Nick Young <nick@isovalent.com>

geps/gep-713.md

@@ -5,35 +5,43 @@
 
 ## TLDR
 
-This GEP aims to standardize attachment of one Kubernetes object to another in
-Gateway API by establishing a pattern which defines how _metaresources_ such as
-`Policy` API types can have their relevant effects applied to network traffic.
-_Metaresources_ are a type of object that _augments_ the behavior of an object
-in a standard way. ReferenceGrant is an example of this general type of metaresource.
-
-"Policy Attachment" is a specific type of _metaresource_ that controls how policy
-can flow _across_ multiple objects in a hierarchy.
-Individual policy APIs (e.g. `TimeoutPolicy`, `RetryPolicy`, etc) must include a
-common `TargetRef` field in their specification to identify how and where to
-apply that policy, along with a consistent set of behaviors around defaults and
-overrides.
-
-This will be important for providing a consistent experience across
-implementations of the API, even for configuration details that may not be fully
-portable.
+This GEP aims to standardize terminology and process around using one Kubernetes
+object modify the functions of one or more other objects.
+
+Generally, a Kubernetes object that that _augments_ the behavior of an object
+in a standard way is called a _Metaresource_ are a type of object. ReferenceGrant
+is an example of this general type of metaresource, but it is far from the only
+one.
+
+This document proposes controlling the creation of configuration in the underlying
+Gateway data plane using two types of Policy Attachment.
+A "Policy Attachment" is a specific type of _metaresource_ that can affect specific
+settings across either one object (this is "Direct Policy Attachment"), or objects
+in a hierarchy (this is "Hierarchical Policy Attachment").
+
+Individual policy APIs must
+- be their own CRDs (e.g. `TimeoutPolicy`, `RetryPolicy` etc),
+- can be included in the Gateway API API group and installation or be defined by
+  implementations
+- and must include a common `TargetRef` struct in their specification to identify how and where to
+  apply that policy, along with a field that indicates if the reference is `Direct`
+  or `Hierarchical`. (Shorter name suggestions welcomed)
+
+For Hierarchical Policies, this GEP also describes a set of expected behaviors
+for how settings can flow across a defined hierarchy.
+
 
 ## Goals
 
-* Clearly communicate what metaresources are, and that policy resources are a
-  specific _type_ of metaresource
-* Establish a pattern for metaresources which will be used for any metaresources
-  or policies included in the Gateway API spec
-* Establish a pattern for metaresources and policy attachment which should be
-  used for any implementation specific policies used with Gateway API resources
+* Establish a pattern for Policy objects which will be used for any policies
+  included in the Gateway API spec
+* Establish a pattern for Policy attachment, whether Direct or Hierarchical,
+  which must be used for any implementation specific policies used with
+  Gateway API resources
 * Provide a way to distinguish between required and default values for all
   policy API implementations
 * Enable policy attachment at all relevant scopes, including Gateways, Routes,
-  Backends
+  Backends, along with how values should flow across a hierarchy if necessary
 * Ensure the policy attachment specification is generic and forward thinking
   enough that it could be easily adapted to other grouping mechanisms like
   Namespaces in the future
@@ -54,8 +62,9 @@ When designing Gateway API, one of the things we’ve found is that we often nee
 able change the behavior of objects without being able to make changes to the spec
 of those objects. Sometimes, this is because we can’t change the spec of the object
 to hold the information we need ( ReferenceGrant, from GEP-709, affecting Secrets
-and Services is an example), and sometimes it’s because we want the behavior change
-to flow across multiple objects (this is what Policy Attachment is for).
+and Services is an example, as is Direct Policy Attachment), and sometimes it’s
+because we want the behavior change to flow across multiple objects
+(this is what Hierarchical Policy Attachment is for).
 
 To put this another way, sometimes we need ways to be able to affect how an object
 is interpreted in the API, without representing the description of those effects
@@ -64,81 +73,110 @@ inside the spec of the object.
 This document describes the ways we design objects to meet these two use cases,
 and why you might choose one or the other.
 
-We use the term “metaresource” to describe both of these use cases, since the
-object is in both cases a superset of its settings and the settings of a wrapped
-object (or objects). “Meta” here is used in its Greek sense of “more comprehensive”
+We use the term “metaresource” to describe the class of objects that _only_ augment
+the behavior of another Kubernetes object, regardless of what they are targeting.
+
+“Meta” here is used in its Greek sense of “more comprehensive”
 or “transcending”, and “resource” rather than “object” because “metaresource”
 is more pronounceable than “metaobject”. Additionally, a single word is better
 than a phrase like “wrapper object” or “wrapper resource” overall, although both
 of those terms are effectively synonymous with “metaresource”.
 
-All ReferenceGrants are metaresources, and all objects that use Policy Attachment
-are metaresources, but metaresources can be more than just those two things.
-
-### Targeted metaresources
-
-ReferenceGrant is an example of a targeted metaresource. It’s tightly bound to a
-particular Kind within a single namespace, and only modifies the behavior of the
-objects that match its binding (by allowing references to them from outside their
-namespace).
-
-We’ve been discussing the idea of having a new object that holds extra capabilities
-for a backend Service (something like BackendCapabilities or similar, name TBD),
-as a way to implement GEP-1282 (Backend Properties). A new resource here that
-referenced a single service and had extra properties that we can’t put into the
-Service object is also an example of a targeted metaresource. The intent is to
-augment the behavior of the bound Service object with the structured data in the
-Spec of BackendCapabilities, since it’s not easy to make changes to a GA upstream
-object like Service. (And arguably, Service is already a pretty overloaded
-resource, so there will need to be good justification for any changes).
-
-With these two examples in mind, here are some guidelines for when to consider
-using a targeted metaresource:
-* The number or scope of objects to be modified is limited or singular. Targeted
-  metaresources should target one specific object (preferred), or a tightly-scoped
-  set of objects (like ReferenceGrant).
+A "Policy Attachment" is a metaresource that affects the fields in existing objects
+(like Gateway or Routes), or influences the configuration that's generated in an
+underlying data plane.
+
+"Direct Policy Attachment" is when a Policy object references a single object _only_,
+and only modifies the fields of or the configuration associated with that object.
+
+"Hierarchical Policy Attachment" is when a Policy object references a single object
+_and any child objects of that object_ (according to some defined hierarchy), and
+modifies fields of the child objects, or configuration associated with the child
+objects.
+
+### Direct Policy Attachment
+
+A Direct Policy Attachment is tightly bound to one or more instances of a particular
+Kind within a single namespace, and only modifies the behavior of the object or
+objects that match its binding.
+
+As an example, one use case that Gateway API currently does not support is how
+to configure details of the TLS required to connect to a backend (in other words,
+if the process running inside the backend workload expects TLS, not that some
+automated infrastructure layer is provisioning TLS as in the Mesh case).
+
+A hypothetical TLSConnectionPolicy that targets a Service could be used for this,
+using the functionality of the Service as describing a set of endpoints. (It
+should be also noted this is not the only way to solve this problem, just an
+example to illustrate Direct Policy Attachment.)
+
+The TLSConnectionPolicy would look something like this:
+
+```yaml
+apiVersion: gateway.networking.k8s.io/v1alpha2
+kind: TLSConnectionPolicy
+metadata:
+  name: tlsport8443
+  namespace: foo
+spec:
+  targetRef: # This struct is defined as part of Gateway API
+    type: Direct
+    group: "" # Empty string means core - this is a standard convention
+    kind: Service
+    name: fooService
+  tls:
+    certificateAuthorityRefs:
+      - name: CAcert
+    port: 8443
+
+```
+
+All this does is tell an implementation, that for connecting to port `8443` on the
+Service `fooService`, it should assume that the connection is TLS, and expect the
+service's certificate to be validated by the chain in the `CAcert` Secret.
+
+Importantly, this would apply to _every_ usage of that Service across any HTTPRoutes
+in that namespace, which could be useful for a Service that is reused in a lot of
+HTTPRoutes.
+
+With this two examples in mind, here are some guidelines for when to consider
+using Direct Policy Attachment:
+* The number or scope of objects to be modified is limited or singular. Direct
+  Policy Attachments should target one specific object (preferred), or a tightly-scoped
+  set of objects (like all Services in a namespace).
 * The modifications to be made to the objects don’t have any transitive information -
   that is, the modifications only affect the single object that the targeted
   metaresource is bound to, and don’t have ramifications that flow beyond that
-  object. ReferenceGrant and BackendCapabilities are good examples here because
-  they don’t expect the metaresource to modify the behavior of more than one
-  object. (Even in the ReferenceGrant case, the ability to set a Kind rather
-  than an individual object is syntactic sugar for having to prevent needing to
-  have lots of ReferenceGrants if you have lots of wrapped objects).
+  object.
 * In terms of status, it should be reasonably easy for a user to understand that
   everything is working - basically, as long as the targeted object exists, and
   the modifications are valid, the metaresource is valid, and this should be
   straightforward to communicate in one or two Conditions. Note that at the time
   of writing, this is *not* completed.
-* Be extremely careful in using metaresources to target things in other namespaces.
-  In general, this is a big security problem, unless the referent resources have
-  some way to decline. Most of the time, it’s better to have the resource target
-  something in its own namespace (that is, the targeted resource describes what
-  should have its behavior modified), with other, similar but distinct fields
-  then describe what to target for further modifications. An example:
-  * A metaresource that constrains client behavior when reaching out to other
-    services should use a targetRef-like field to choose the clients, and have
-    another field that sets what the external targets of the constrained behavior
-    are. (The external targets may conceivably be “all external targets”).
-  In general, metaresources that modify the behavior of things outside their own
-  namespace should be avoided unless it uses a handshake of some sort, where the
-  things outside the namespace can opt–out of the behavior. (Notably, this is the
-  design that we used for ReferenceGrant).)
-
-### Policy Attachment: It's all about the defaults and overrides
-
-Because a Policy is a metaresource, it targets some other resource and _augments_
-its behavior.
-
-But why have this distinct from other types of metaresource? Because Policy
-resources are designed to have a way for settings to flow down a hierarchy.
+* Direct Policy Attachment _should_ only be used to target objects in the same
+  namespace as the Policy object. Allowing cross-namespace references brings in
+  significant security concerns, and/or difficulties about merging cross-namespace
+  policy objects. Notably, Mesh use cases may need to do something like this for
+  consumer policies, but in general, Policy objects that modify the behavior of
+  things outside their own namespace should be avoided unless it uses a handshake
+  of some sort, where the things outside the namespace can opt–out of the behavior.
+  (Notably, this is the design that we used for ReferenceGrant).)
+
+### Hierarchical Policy Attachment: It's all about the defaults and overrides
+
+Because a Hierarchical Policy is a metaresource, it targets some other resource
+and _augments_ its behavior.
+
+But why have this distinct from other types of metaresource? Because Hierarchical
+Policy resources are designed to have a way for settings to flow down a hierarchy.
+
 Defaults set the default value for something, and can be overridden by the
 “lower” objects (like a connection timeout default policy on a Gateway being
 overridable inside a HTTPRoute), and Overrides cannot be overridden by “lower”
 objects (like setting a maximum client timeout to some non-infinite value at the
 Gateway level to stop HTTPRoute owners from leaking connections over time).
 
-Here are some guidelines for when to consider using a Policy object:
+Here are some guidelines for when to consider using a Hierarchical Policy object:
 * The settings or configuration are bound to one containing object, but affect
   other objects attached to that one (for example, affecting HTTPRoutes attached
   to a single Gateway, or all HTTPRoutes in a GatewayClass).
@@ -153,7 +191,6 @@ Here are some guidelines for when to consider using a Policy object:
   is not, and needs to be carefully designed to avoid fanout apiserver load.
   (This is not built at all in the current design either).
 
-
 ## API
 
 This approach is building on concepts from all of the alternatives discussed
@@ -162,16 +199,18 @@ but also borrows some concepts from the [ServicePolicy
 proposal](https://github.com/kubernetes-sigs/gateway-api/issues/611).
 
 ### Policy Attachment for Ingress
-Attaching policy to Gateway resources for ingress use cases is relatively
-straightforward. A policy can reference the resource it wants to apply to.
+Attaching a Directly Attached Policy to Gateway resources for ingress use cases
+is relatively straightforward. A policy can reference the resource it wants to
+apply to.
+
 Access is granted with RBAC - anyone that has access to create a RetryPolicy in
 a given namespace can attach it to any resource within that namespace.
 
 ![Simple Ingress Example](images/713-ingress-simple.png)
 
-To build on that example, it’s possible to attach policies to more resources.
-Each policy applies to the referenced resource and everything below it in terms
-of hierarchy. Although this example is likely more complex than many real world
+A Hierarchical Policy can attach to a parent resource, and then each policy
+applies to the referenced resource and everything below it in terms of hierarchy.
+Although this example is likely more complex than many real world
 use cases, it helps demonstrate how policy attachment can work across
 namespaces.
 
@@ -216,6 +255,13 @@ The `targetRef` field MUST have the following structure:
 ```go
 // PolicyTargetReference identifies an API object to apply policy to.
 type PolicyTargetReference struct {
+    // Type is the type of the policy attachment.
+    // This can be either Direct or Hierarchical.
+    //
+    // +kubebuilder:validation:MinLength=1
+    // +kubebuilder:validation:MaxLength=253
+    Type string `json:"type"`
+
     // Group is the group of the target resource.
     //
     // +kubebuilder:validation:MinLength=1
@@ -340,6 +386,7 @@ spec:
         includeProtocol: true
         includeQueryString: true
   targetRef:
+    type: hierarchical
     kind: Gateway
     name: example
 ---
@@ -350,6 +397,7 @@ spec:
       cachePolicy:
         includeQueryString: false
   targetRef:
+    type: direct
     kind: HTTPRoute
     name: example
 ```
@@ -393,6 +441,7 @@ spec:
   default:
     maxRetries: 5
   targetRef:
+    type: direct
     group: networking.acme.io
     kind: ExternalService
     name: foo.com
@@ -405,6 +454,7 @@ the same resource _and_ have an identical field specified with different values,
 precedence MUST be determined in order of the following criteria, continuing on
 ties:
 
+* Hierarchical Policies override Direct Policies.
 * The oldest Policy based on creation timestamp. For example, a Policy with a
   creation timestamp of "2021-07-15 01:02:03" is given precedence over a Policy
   with a creation timestamp of "2021-07-15 01:02:04".