Skip to content

Commit

Permalink
Add clarifications and examples of different interactions
Browse files Browse the repository at this point in the history 
Signed-off-by: Nick Young <nick@isovalent.com>
  • Loading branch information
@youngnick
youngnick committed Mar 16, 2023
1 parent 32817ca commit ec345cf
Show file tree
Hide file tree
Showing 2 changed files with 730 additions and 33 deletions.
310 changes: 302 additions & 8 deletions geps/gep-713.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,10 @@ _and any child objects of that object_ (according to some defined hierarchy), an
modifies fields of the child objects, or configuration associated with the child
objects.

In either case, a Policy may either affect an object by controlling the value
of one of the existing _fields_ in the `spec` of an object, or it may add
additional fields that are _not_ in the `spec` of the object.

### Direct Policy Attachment

A Direct Policy Attachment is tightly bound to one or more instances of a particular
Expand Down Expand Up @@ -194,6 +198,50 @@ Here are some guidelines for when to consider using a Hierarchical Policy object
is not, and needs to be carefully designed to avoid fanout apiserver load.
(This is not built at all in the current design either).

When multiple Hierarchical Policies are used, they can interact in various ways,
which are governed by the following rules, which will be expanded on later in
in this document.

* If an Policy does not affect an object's fields directly, then the resultant
Policy should be the set of all distinct fields inside the relevant Policy objects,
as set out by the rules below.
* For Policies that affect an object's existing fields, multiple instances of the
same Policy Kind affecting an object's fields will be evaluated as
though only a single Policy "wins" the right to affect each field. This operation
is performed on a _per-distinct-field_ basis.
* Settings in `overrides` stanzas will win over the same setting in a `defaults`
stanza.
* `overrides` settings operate in a "less specific beats more specific" fashion -
Policies attached _higher_ up the hierarchy will beat the same type of Policy
attached further down the hierarchy.
* `defaults` settings operate in a "more specific beats less specific" fashion -
Policies attached _lower down_ the hierarchy will beat the same type of Policy
attached further _up_ the hierarchy.
* For `defaults`, the _most specific_ value is the one _inside the object_ that
the Policy applies to; that is, if a Policy specifies a `default`, and an object
specifies a value, the _object's_ value will win.
* Policies interact with the fields they are controlling in a "replace value"
fashion.
* For fields where the `value` is a scalar, (like a string or a number)
should have their value _replaced_ by the value in the Policy if it wins.
Notably, this means that a `default` will only ever replace an empty or unset
value in an object.
* For fields where the value is an object, the Policy should include the fields
in the object in its definition, so that the replacement can be on simple fields
rather than complex ones.
* For fields where the final value is non-scalar, but is not an _object_ with
fields of its own, the value should be entirely replaced, _not_ merged. This
means that lists of strings or lists of ints specified in a Policy will overwrite
the empty list (in the case of a `default`) or any specified list (in the case
of an `override`). The same applies to `map[string]string` fields. An example
here would be a field that stores a map of annotations - specifying a Policy
that overrides annotations will mean that a final object specifying those
annotations will have its value _entirely replaced_ by an `override` setting.
* In the case that two Policies of the same type specify different fields, then
_all_ of the specified fields should take effect on the affected object.

Examples to further illustrate these rules are given below.

## API

This approach is building on concepts from all of the alternatives discussed
Expand Down Expand Up @@ -369,7 +417,7 @@ precedence over Routes and Services below it. On the other hand, an app owner
may want to set a default timeout for their Service. That would have precedence
over defaults attached at higher levels such as Route or Gateway.

If using defaults and overrides, each policy resource MUST include 2 structs
If using defaults _and_ overrides, each policy resource MUST include 2 structs
within the spec. One with override values and the other with default values.

In the following example, the policy attached to the Gateway requires cdn to
Expand Down Expand Up @@ -410,6 +458,9 @@ precedence over the default drainTimeout value attached to the Route. At the
same time, we can see that the default connectionTimeout attached to the Route
has precedence over the default attached to the Gateway.
Also note how the different resources interact - fields that are not common across
objects _may_ both end up affecting the final object.
![Hierarchical Policy Example](images/713-policy-hierarchy.png)
#### Supported Resources
Expand All @@ -433,7 +484,7 @@ used to set defaults and requirements for an entire GatewayClass.
### Targeting External Services
In some cases (likely limited to mesh) we may want to apply policies to requests
to external services. To accomplish this, implementations can choose to support
a refernce to a virtual resource type:
a reference to a virtual resource type:
```yaml
apiVersion: networking.acme.io/v1alpha1
Expand All @@ -450,13 +501,13 @@ spec:
name: foo.com
```
### Merge semantics
### Merging into existing `spec` fields

It's possible (even likely) that configuration in a Policy may need to be merged
into an existing object somehow, particularly for Hierarchical policies.
into an existing object's fields somehow, particularly for Hierarchical policies.

In general, Policy objects should merge values at a scalar level, not at a
whole-struct level.
When merging into an existing fields inside an object, Policy objects should
merge values at a scalar level, not at a structor object level.

For example, in the `GKEServicePolicy` example above, the `cdn` struct contains
a `cachePolicy` struct that contains fields. If an implementation was merging
Expand All @@ -474,16 +525,23 @@ In the case that the field in the Policy affects a struct that is a member of a
each existing item in the list in the affected object should have each of its
fields compared to the corresponding fields in the Policy.

For non-scalar field _values_, like a list of strings, or a `map[string]string`
value, the _entire value_ must be overwritten by the value from the Policy. No
merging should take place. This mainly applies to `overrides`, since for
`defaults`, there should be no value present in a field on the final object.

### Conflict Resolution
It is possible for multiple policies to target the same resource. When this
happens, merging is the preferred outcome. If multiple policy resources target
It is possible for multiple policies to target the same object _and_ the same
fields inside that object. If multiple policy resources target
the same resource _and_ have an identical field specified with different values,
precedence MUST be determined in order of the following criteria, continuing on
ties:

* Direct Policies override Hierarchical Policies. If preventing settings from
being overwritten is important, implementations should only use Hierarchical
Policies, and the `override` stanza that implies.
* Inside Hierarchical Policies, the same setting in `overrides` beats the one in
`defaults`.
* The oldest Policy based on creation timestamp. For example, a Policy with a
creation timestamp of "2021-07-15 01:02:03" is given precedence over a Policy
with a creation timestamp of "2021-07-15 01:02:04".
Expand Down Expand Up @@ -725,6 +783,242 @@ type RouteRule struct {
### Disadvantages
* May be difficult to understand which policies apply to a request

## Examples

This section provides some examples of various types of Policy objects, and how
merging, `defaults`, `overrides`, and other interactions work.

### Direct Policy Attachment

The following Policy sets the minimum TLS version required on a Gateway Listener:
```yaml
apiVersion: networking.example.io/v1alpha1
kind: TLSMinimumVersionPolicy
metadata:
name: minimum12
namespace: default
spec:
minimumTLSVersion: 1.2
targetRef:
name: internet
group: gateway.networking.k8s.io
kind: Gateway
```
Note that because there is no version controlling the minimum TLS version in the
Gateway `spec`, this is an example of a non-field Policy.

### Hierarchical Policy Attachment

It also could be useful to be able to _default_ the `minimumTLSVersion` setting
across multiple Gateways.

This version of the above Policy allows this:
```yaml
apiVersion: networking.example.io/v1alpha1
kind: TLSMinimumVersionPolicy
metadata:
name: minimum12
namespace: default
spec:
defaults:
minimumTLSVersion: 1.2
targetRef:
name: default
group: ""
kind: namespace
```

This Hierarchical Policy is using the implicit hierarchy that all resources belong
to a namespace, so attaching a Policy to a namespace means affecting all possible
resources in a namespace. Multiple hierarchies are possible, even within Gateway
API, for example Gateway -> Route, Gateway -> Route -> Backend, Gateway -> Route
-> Service. GAMMA Policies could conceivably use a hierarchy of Service -> Route
as well.

Note that this will not be very discoverable for Gateway owners in the absence of
a solution to the Policy status problem. This is being worked on and this GEP will
be updated once we have a design.

Conceivably, a security or admin team may want to _force_ Gateways to have at least
a minimum TLS version of `1.2` - that would be a job for `overrides`, like so:

```yaml
apiVersion: networking.example.io/v1alpha1
kind: TLSMinimumVersionPolicy
metadata:
name: minimum12
namespace: default
spec:
overrides:
minimumTLSVersion: 1.2
targetRef:
name: default
group: ""
kind: namespace
```

This will make it so that _all Gateways_ in the `default` namespace _must_ use
a minimum TLS version of `1.2`, and this _cannot_ be changed by Gateway owners.
Only the Policy owner can change this Policy.

### Handling non-scalar values

In this example, we will assume that at some future point, HTTPRoute has grown
fields to configure retries, including a field called `retryOn` that reflects
the HTTP status codes that should be retried. The _value_ of this field is a
list of strings, being the HTTP codes that must be retried. The `retryOn` field
has no defaults in the field definitions (which is probably a bad design, but we
need to show this interaction somehow!)

We also assume that a Hierarchical `RetryOnPolicy` exists that allows both
defaulting and overriding of the `retryOn` field.

A full `RetryOnPolicy` to default the field to the codes `501`, `502`, and `503`
would look like this:
```yaml
apiVersion: networking.example.io/v1alpha1
kind: RetryOnPolicy
metadata:
name: retryon5xx
namespace: default
spec:
defaults:
retryOn:
- "501"
- "502"
- "503"
targetRef:
kind: Gateway
group: gateway.networking.k8s.io
name: WeLoveRetries
```

This means that, for HTTPRoutes that do _NOT_ explicitly set this field to something
else, (in other words, they contain an empty list), then the field will be set to
a list containing `501`, `502`, and `503`. (Notably, because of Go zero values, this
would also occur if the user explicitly set the value to the empty list.)

However, if a HTTPRoute owner sets any value other than the empty list, then that
value will remain, and the Policy will have _no effect_. These values are _not_
merged.

If the Policy used `overrides` instead:
```yaml
apiVersion: networking.example.io/v1alpha1
kind: RetryOnPolicy
metadata:
name: retryon5xx
namespace: default
spec:
overrides:
retryOn:
- "501"
- "502"
- "503"
targetRef:
kind: Gateway
group: gateway.networking.k8s.io
name: YouMustRetry
```

### Interactions between defaults, overrides, and field values

All HTTPRoutes that attach to the `YouMustRetry` Gateway will have any value
_overwritten_ by this policy. The empty list, or any number of values, will all
be replaced with `501`, `502`, and `503`.

Now, let's also assume that we use the Namespace -> Gateway hierarchy on top of
the Gateway -> HTTPRoute hierarchy, and allow attaching a `RetryOnPolicy` to a
_namespace_. The expectation here is that this will affect all Gateways in a namespace
and all HTTPRoutes that attach to those Gateways. (Note that the HTTPRoutes
themselves may not necessarily be in the same namespace though.)

If we apply the default policy from earlier to the namespace:
```yaml
apiVersion: networking.example.io/v1alpha1
kind: RetryOnPolicy
metadata:
name: retryon5xx
namespace: default
spec:
defaults:
retryOn:
- "501"
- "502"
- "503"
targetRef:
kind: Namespace
group: ""
name: default
```

Then this will have the same effect as applying that Policy to every Gateway in
the `default` namespace - namely that every HTTPRoute that attaches to every
Gateway will have its `retryOn` field set to `501`, `502`, `503`, _if_ there is no
other setting in the HTTPRoute itself.

With two layers in the hierarchy, we have a more complicated set of interactions
possible.

Let's look at some tables for a particular HTTPRoute, assuming that it does _not_
configure the `retryOn` field, for various types of Policy at different levels.

#### Overrides interacting with defaults for RetryOnPolicy, empty list in HTTPRoute

||None|Namespace override|Gateway override|HTTPRoute override|
|----|-----|-----|----|----|
|No default|Empty list|Namespace override| Gateway override Policy| HTTPRoute override|
|Namespace default| Namespace default| Namespace override | Gateway override | HTTPRoute override |
|Gateway default| Gateway default | Namespace override | Gateway override | HTTPRoute override |
|HTTPRoute default| HTTPRoute default | Namespace override | Gateway override | HTTPRoute override|

#### Overrides interacting with other overrides for RetryOnPolicy, empty list in HTTPRoute
||No override|Namespace override|Gateway override|HTTPRoute override|
|----|-----|-----|----|----|
|No override|Empty list|Namespace override| Gateway override| HTTPRoute override|
|Namespace override| Namespace override| Namespace override<br />first created wins<br />otherwise first alphabetically | Namespace override | Namespace override |
|Gateway override| Gateway override | Namespace override | Gateway override<br />first created wins<br />otherwise first alphabetically | Gateway override |
|HTTPRoute override| HTTPRoute override | Namespace override | Gateway override | HTTPRoute override<br />first created wins<br />otherwise first alphabetically|

#### Defaults interacting with other defaults for RetryOnPolicy, empty list in HTTPRoute
||No default|Namespace default|Gateway default|HTTPRoute default|
|----|-----|-----|----|----|
|No default|Empty list|Namespace default| Gateway default| HTTPRoute default|
|Namespace default| Namespace default| Namespace default<br />first created wins<br />otherwise first alphabetically | Gateway default | HTTPRoute default |
|Gateway default| Gateway default | Gateway default | Gateway default<br />first created wins<br />otherwise first alphabetically | HTTPRoute default |
|HTTPRoute default| HTTPRoute default | HTTPRoute default | HTTPRoute default | HTTPRoute default<br />first created wins<br />otherwise first alphabetically|


Now, if the HTTPRoute _does_ specify a RetryPolicy,
it's a bit easier, because we can basically disregard all defaults:

#### Overrides interacting with defaults for RetryOnPolicy, value in HTTPRoute

||None|Namespace override|Gateway override|HTTPRoute override|
|----|-----|-----|----|----|
|No default| Value in HTTPRoute|Namespace override| Gateway override Policy| HTTPRoute override|
|Namespace default| Value in HTTPRoute| Namespace override | Gateway override | HTTPRoute override |
|Gateway default| Value in HTTPRoute | Namespace override | Gateway override | HTTPRoute override |
|HTTPRoute default| Value in HTTPRoute | Namespace override | Gateway override | HTTPRoute override|

#### Overrides interacting with other overrides for RetryOnPolicy, value in HTTPRoute
||No override|Namespace override|Gateway override|HTTPRoute override|
|----|-----|-----|----|----|
|No override|Value in HTTPRoute|Namespace override| Gateway override| HTTPRoute override|
|Namespace override| Namespace override| Namespace override<br />first created wins<br />otherwise first alphabetically | Namespace override | Namespace override |
|Gateway override| Gateway override | Namespace override | Gateway override<br />first created wins<br />otherwise first alphabetically | Gateway override |
|HTTPRoute override| HTTPRoute override | Namespace override | Gateway override | HTTPRoute override<br />first created wins<br />otherwise first alphabetically|

#### Defaults interacting with other defaults for RetryOnPolicy, value in HTTPRoute
||No default|Namespace default|Gateway default|HTTPRoute default|
|----|-----|-----|----|----|
|No default|Value in HTTPRoute|Value in HTTPRoute|Value in HTTPRoute|Value in HTTPRoute|
|Namespace default|Value in HTTPRoute|Value in HTTPRoute|Value in HTTPRoute|Value in HTTPRoute|
|Gateway default|Value in HTTPRoute|Value in HTTPRoute|Value in HTTPRoute|Value in HTTPRoute|
|HTTPRoute default|Value in HTTPRoute|Value in HTTPRoute|Value in HTTPRoute|Value in HTTPRoute|


## Removing BackendPolicy
BackendPolicy represented the initial attempt to cover policy attachment for
Gateway API. Although this proposal ended up with a similar structure to
Expand Down
Loading

0 comments on commit ec345cf

Please sign in to comment.