diff --git a/rfcs/0000-rlp-v2.md b/rfcs/0000-rlp-v2.md index aba47bc9..41add040 100644 --- a/rfcs/0000-rlp-v2.md +++ b/rfcs/0000-rlp-v2.md @@ -34,15 +34,15 @@ Furthermore, the also related [`Limit`](https://pkg.go.dev/github.com/kuadrant/k ### Highlights -- `spec.rateLimits` replaced by `spec.limits` -- `spec.rateLimits.limits` replaced by `spec.limits.rates` -- `spec.rateLimits.limits.maxValue` replaced by `spec.limits.rates.limit` -- `spec.rateLimits.limits.seconds` replaced by `spec.limits.rates.duration` + `spec.limits.rates.unit` -- `spec.rateLimits.limits.conditions` replaced by `spec.limits.when`, structured field based on _well-known selectors_, mainly for expressing conditions not related to the HTTP route (although not exclusively) -- `spec.rateLimits.limits.variables` replaced by `spec.limits.counters`, based on _well-known selectors_ -- `spec.rateLimits.rules` replaced by `spec.limits.matches`, where each match is a perfect match to one of the `HTTPRouteMatch`es of the targeted `HTTPRoute` (`httproute.spec.rules.matches`) – complies with [architecture#4](https://github.com/Kuadrant/architecture/pull/4) -- `spec.rateLimits.configurations` removed – `spec.rateLimits.configurations.actions` generated from `spec.limits.when.selector` ∪ `spec.limits.counters` and `spec.limits.matches` -- Limitador conditions composed of `spec.limits.when` + unique hash identifier - ensures the limit is enforced only for the targeted `HTTPRouteMatch` +- `spec.rateLimits` replaced with `spec.limits` +- `spec.rateLimits.limits` replaced with `spec.limits.rates` +- `spec.rateLimits.limits.maxValue` replaced with `spec.limits.rates.limit` +- `spec.rateLimits.limits.seconds` replaced with `spec.limits.rates.duration` + `spec.limits.rates.unit` +- `spec.rateLimits.limits.conditions` replaced with `spec.limits.when`, structured field based on _well-known selectors_, mainly for expressing conditions not related to the HTTP route (although not exclusively) +- `spec.rateLimits.limits.variables` replaced with `spec.limits.counters`, based on _well-known selectors_ +- `spec.rateLimits.rules` replaced with `spec.limits.rules`, for "sub-targeting" HTTPRouteRules within an HTTPRoute +- `spec.rateLimits.configurations` removed – `spec.rateLimits.configurations.actions` generated from `spec.limits.when.selector` ∪ `spec.limits.counters` and `spec.limits.rules` +- Limitador conditions composed of `spec.limits.when` + unique hash identifier - ensures the limit is enforced only for the targeted HTTPRouteRules For detailed differences between current and vew RLP API, see [Comparison to current RateLimitPolicy](#comparison-to-current-ratelimitpolicy). @@ -87,13 +87,26 @@ spec: backendRefs: - name: toystore port: 80 + - matches: + - path: + type: PathPrefix + value: "/assets/" + backendRefs: + - name: toystore + port: 80 + filters: + - type: ResponseHeaderModifier + responseHeaderModifier: + set: + - name: Cache-Control + value: max-age=31536000, immutable ``` -The following are examples of RLPs targeting the route and the gateway. +The following are examples of RLPs targeting the route and the gateway. Each example is independent from the other. #### Example 1. Minimal example - network resource targeted entirely without filtering, unconditional and unqualified rate limiting -In this example, all traffic to `*.toystore.com` will be limited to 5 rps, unconditionally and regardless of path, method or any other attribute of the request, across all consumers of the API. +In this example, all traffic to `*.toystore.com` will be limited to 5rps, unconditionally and regardless of path, method or any other attribute of the request, across all consumers of the API. ```yaml apiVersion: kuadrant.io/v2beta1 @@ -126,42 +139,57 @@ spec: configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-simple-infra-rl/0/e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07" + descriptor_value: "toystore/toystore-simple-infra-rl/0/d9edf43707d9a99b4f499055aa59ef6848e2346a638021944bd8f1efce22a8b3" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}},{"matches":{"path":{"type":"PathPrefix","value":"/assets/"}}}] + - rules: + - paths: ["/assets/*"] + methods: ["*"] + hosts: ["*.toystore.com"] + configurations: + - generic_key: + descriptor_key: "context.request.http.route_matcher" + descriptor_value: "toystore/toystore-simple-infra-rl/0/d9edf43707d9a99b4f499055aa59ef6848e2346a638021944bd8f1efce22a8b3" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}},{"matches":{"path":{"type":"PathPrefix","value":"/assets/"}}}] ``` ```yaml limits: - conditions: - - context.request.http.route_matcher == "toystore/toystore-simple-infra-rl/0/e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07" + - context.request.http.route_matcher == "toystore/toystore-simple-infra-rl/0/d9edf43707d9a99b4f499055aa59ef6848e2346a638021944bd8f1efce22a8b3" max_value: 5 seconds: 1 namespace: "*.toystore.com" ``` -#### Example 2. Specific route matching rule targeted, conditions, counter qualifiers and multiple rates +#### Example 2. Targeting specific route rules, plus conditions, counter qualifiers and multiple rates -In this example, a distinct limit is associated to each individual matching rule of the targeted HTTPRoute, using the `matches` field for fine-grained filtering. In both cases, the rate limits are to be enforced per username and only in case the user is not an admin. +In this example, a distinct limit will be associated to each individual rule of the targeted HTTPRoute, using the `rules` field for fine-grained filtering (or "sub-targeting"). +- `GET|POST /toys*` → 50rpm, enforced per username (counter qualifier) and only in case the user is not an admin (condition). +- `/assets/*` → 5rpm / 100rp12h -For each matching rule in the RLP, the RLP controller will try to find a matching rule in the HTTPRoute that is an identical match to it and bind the two matching rules together; the first identical match prevails. In case there is no identical match in the HTTPRoute, the RLP is considered invalid. In case there is more than one matching rule specified in the RLP that is an identical match to the same matching rule in the HTTPRoute, the first matching rule on the list in the RLP is bound to its identical match in the HTTPRoute, thus "shadowing" any other rule on the list that is also an identical match. +Each rule in the RLP must perfectly match a rule in the HTTPRoute. In case there is no identical match amongst the HTTPRoute rules for a given rule defined in the RLP, the RLP is considered invalid. In case there is more than one rule specified in the RLP that is an identical match to the same rule in the HTTPRoute, the first matching rule on the list in the RLP is bound to its identical match in the HTTPRoute, thus "shadowing" any other rule on the list that is also an identical match. ```yaml apiVersion: kuadrant.io/v2beta1 kind: RateLimitPolicy metadata: - name: toystore-per-endpoint-per-user + name: toystore-per-endpoint spec: targetRef: group: gateway.networking.k8s.io kind: HTTPRoute name: toystore limits: - - name: readers - matches: - - path: - type: PathPrefix - value: "/toys" - method: GET + - name: toys + rules: + - matches: # matches the 1st rule in the HTTPRoute + - path: + type: PathPrefix + value: "/toys" + method: GET + - path: + type: PathPrefix + value: "/toys" + method: POST rates: - limit: 50 duration: 1 @@ -173,12 +201,12 @@ spec: operator: neq value: admin - - name: writers - matches: - - path: - type: PathPrefix - value: "/toys" - method: POST + - name: assets + rules: + - matches: # matches the 2nd rule in the HTTPRoute + - path: + type: PathPrefix + value: "/assets/" rates: - limit: 5 duration: 1 @@ -186,12 +214,6 @@ spec: - limit: 100 duration: 12 unit: hour - counters: - - auth.identity.username - when: - - selector: auth.identity.group - operator: neq - value: admin ```
@@ -203,10 +225,13 @@ spec: - paths: ["/toys*"] methods: ["GET"] hosts: ["*.toystore.com"] + - paths: ["/toys*"] + methods: ["POST"] + hosts: ["*.toystore.com"] configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-per-endpoint-per-user/readers/2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05" + descriptor_value: "toystore/toystore-per-endpoint/toys/c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}}] - metadata: descriptor_key: "auth.identity.group" metadata_key: @@ -226,37 +251,19 @@ spec: - segment: key: "username" - rules: - - paths: ["/toys*"] - methods: ["POST"] + - paths: ["/assets/*"] + methods: ["*"] hosts: ["*.toystore.com"] configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-per-endpoint-per-user/writers/4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d" - - metadata: - descriptor_key: "auth.identity.group" - metadata_key: - key: "envoy.filters.http.ext_authz" - path: - - segment: - key: "identity" - - segment: - key: "group" - - metadata: - descriptor_key: "auth.identity.username" - metadata_key: - key: "envoy.filters.http.ext_authz" - path: - - segment: - key: "identity" - - segment: - key: "username" + descriptor_value: "toystore/toystore-per-endpoint/assets/643f8d429ff65b62bf9d69bf201461ce3bf5f47f0a5d54fd519d118fa91cce66" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/assets/"}}}] ``` ```yaml limits: - conditions: - - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/readers/2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05" + - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/toys/c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f" - auth.identity.group != "admin" variables: - auth.identity.username @@ -264,18 +271,12 @@ spec: seconds: 60 namespace: "*.toystore.com" - conditions: - - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/writers/4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d" - - auth.identity.group != "admin" - variables: - - auth.identity.username + - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/assets/643f8d429ff65b62bf9d69bf201461ce3bf5f47f0a5d54fd519d118fa91cce66" max_value: 5 seconds: 60 namespace: "*.toystore.com" - conditions: - - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/writers/4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d" - - auth.identity.group != "admin" - variables: - - auth.identity.username + - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/assets/643f8d429ff65b62bf9d69bf201461ce3bf5f47f0a5d54fd519d118fa91cce66" max_value: 100 seconds: 43200 # 12 hours namespace: "*.toystore.com" @@ -284,9 +285,11 @@ spec: #### Example 3. Route filtering using `when` conditions -`when` conditions are preferably to be used for special cases of conditional filtering based on values other than attributes of the route that could otherwise be specified using the `matches` field. However, in some cases such as where the condition is not related to the HTTPRoute (e.g. see [Example 2](#example-2-specific-route-matching-rule-targeted-conditions-counter-qualifiers-and-multiple-rates) above; filterring based on hostname, etc) or a temporary measure while the target object misses the desired matching rule, the `when` conditions remains an option. +This is a similar RLP to the one from [Example 2](#example-2-targeting-specific-route-rules-plus-conditions-counter-qualifiers-and-multiple-rates) regarding the use case of applying specific limits to portions of an HTTPRoute. Whereas in the previous example the route rules map perfectly for the limits to be applied, in this example we use `when` conditions to apply the limit to a subpath without having a corresponding HTTPRouteRule that could be referred using the `rules` field. -In this example, a special limit with one rate limit of 150 rps is set for `GET` requests to the `/toys/special` path. Because the `toystore` HTTPRoute is unaltered, the special case is defined using the `when` conditions field. +`when` conditions are to be used preferably for special cases of conditional filtering based on values other than attributes of the HTTP request that otherwise could be specified using the `rules` field of the RLP. However, in some cases such as where the conditions are not related to the HTTPRoute (e.g. filterring based on hostname, filterring based on metadata) or while the targeted object (HTTPRoute or Gateway) misses the desired rule and cannot be changed, the `when` conditions remain an option. + +In this example, a special limit with one rate limit of 150rps is set for `GET /toys/special`. Ideally, an additional HTTPRoute rule would be created and targeted using the `rules` field, with resulting benefits for optimization, status reporting and matching. But because the `toystore` HTTPRoute is unaltered, the special case is implemented using the `when` conditions field instead. ```yaml apiVersion: kuadrant.io/v2beta1 @@ -300,25 +303,35 @@ spec: name: toystore limits: - name: read-special - matches: - - path: - type: PathPrefix - value: "/toys" - method: GET + rules: + - matches: + - path: + type: PathPrefix + value: "/toys" + method: GET + - path: # must be included as well or it wouldn't match the http route rule + type: PathPrefix + value: "/toys" + method: POST rates: - limit: 150 unit: second when: + - selector: context.request.http.method + operator: eq + value: GET - selector: context.request.http.path operator: eq value: /toys/special ``` -By using the route rule as-is, simply `GET /toys*` instead of more specific `GET /toys/special`, the rate limit filter (wasm filter) will continue invoking the rate limit service (Limitador) for all requests that match HTTP method equal to `GET` and path prefix equal to `/toys`, including requests to paths other than `/toys/special`, but the limit will only be enforced when the path fully matches `/toys/special`, as specified in the `when` condition. +By not targeting a specific HTTPRoute rule added for `GET /toys/special`, the rate limit filter (wasm shim) will continue invoking the rate limit service (Limitador) for all requests that match the HTTP the path prefix `/toys` (methods `GET` and `POST`), as targeted HTTPRoute rule. This includes requests to paths other than `GET /toys/special` nevertheless. However, the limit will be ensured to be enforced in Limitador only when the request matches `GET /toys/special`. + +Technically, the `rules` field did not have to be included in the RLP, and the `when` would still ensure the limit applies only to requests to `GET /toys/special`. However, in this case the other HTTPRoute rule (`/assets/*`) would also be initally targeted by the limit, though never effectivelly triggered. -On one hand, this makes it easy to set limits when either the HTTPRoute cannot be modified to include a special matching rule or as a shortcut in cases where multiple HTTPRoutes would have to be touched, such when targeting a Gateway with special conditions based on attributes of the route. On the other hand however, users might miss information in the status in a scenario where the status of rate limiting is reported at the level of the route rule or rule matcher. Effectively, the route rule `GET /toys*` might be reported as rate limited to '150 rps' when that is actually only the case of requests to `GET /toys/special`. These special conditions of the limit definition need therefore to be included in the status. +On one hand, using `when` conditions for route filterring makes it easy to set limits when either the HTTPRoute cannot be modified to include the special rule or as a shortcut in cases where multiple HTTPRoutes would have to be touched (e.g., when targeting a Gateway with special conditions based on attributes of the HTTP request). On the other hand however, users might miss information in the status in a scenario where the status of rate limiting is reported at the level of the route rule. Effectively, the route rule for `GET|POST /toys*` in the above example might be reported as rate limited to '150rps' with additional details that this is in fact only when requests match `GET /toys/special`. -In the case of existing other limit definitions targeting the `GET /toys*` matching rule of the `toystore` HTTPRoute, because a merge strategy is expect to take into account the `matches` field as part of the qualification of the limit, there should be no problem of multiple simultaneous limits enforced to the same route rules, differenciated only by special conditions. Nevertheless, to avoid dealing of complex status reports including too many special conditions associated with a limit, users are instead encouraged to favor altering the HTTPRoutes for additional route rules that can be referred in the `matches` field preferably. +In the case of existing other limit definitions targeting the `GET|POST /toys*` rule of the `toystore` HTTPRoute, because any merge strategy is expected to take into account the `rules` field as part of the qualification of the limit, all limits are guaranteed to be enforced, and only differenciated by the special "soft" the condition. Nevertheless, to avoid dealing with complex status reports including too many special conditions associated with a limit, **users are encouraged to favor altering the HTTPRoutes for additional route rules in all cases where the rules can be referred in the `rules` field of the RLP, whenever possible**.
How is this RLP implemented under the hood? @@ -329,10 +342,13 @@ In the case of existing other limit definitions targeting the `GET /toys*` match - paths: ["/toys*"] methods: ["GET"] hosts: ["*.toystore.com"] + - paths: ["/toys*"] + methods: ["POST"] + hosts: ["*.toystore.com"] configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-special-toys/read-special/2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05" + descriptor_value: "toystore/toystore-special-toys/read-special/c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}}] - request_headers: descriptor_key: "context.request.http.path" header_name: ":path" @@ -341,7 +357,7 @@ In the case of existing other limit definitions targeting the `GET /toys*` match ```yaml limits: - conditions: - - context.request.http.route_matcher == "toystore/toystore-special-toys/read-special/2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05" + - context.request.http.route_matcher == "toystore/toystore-special-toys/read-special/c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f" - context.request.http.path == "/toys/special" max_value: 150 seconds: 1 @@ -351,9 +367,9 @@ In the case of existing other limit definitions targeting the `GET /toys*` match #### Example 4. Route filtering by refining the HTTPRoute -To achieve the same goal as state in [Example 3](#example-3-route-filtering-using-when-conditions) yet ensuring proper merging of conflicting limits that target the same high-level route rule and simpler status report without additional condition associated with the limit, this example RLP is preceded by a change in the HTTPRoute. A new matching rule is added for the `GET /toys/special` case so it can be targeted by the policy using the `matches` field instead of the `when` conditions field. +To achieve the same goal as stated in [Example 3](#example-3-route-filtering-using-when-conditions), yet ensuring proper merging of conflicting limits that target the same high-level route rule and simpler status report without additional condition details associated with the limit, this example RLP is preceded by a change in the HTTPRoute. A new matching rule is added for the `GET /toys/special` case so it can be targeted by the policy using the `rules` field instead of the `when` conditions field. -New matching rule added to the HTTPRoute: +New rule added to the HTTPRoute: ```yaml apiVersion: gateway.networking.k8s.io/v1alpha2 @@ -368,10 +384,6 @@ spec: hostnames: ["*.toystore.com"] rules: - matches: - - path: # new matching rule added so it can be targeted by the RLP - type: Exact - value: "/toys/special" - method: GET - path: type: PathPrefix value: "/toys" @@ -383,6 +395,27 @@ spec: backendRefs: - name: toystore port: 80 + - matches: + - path: + type: PathPrefix + value: "/assets/" + backendRefs: + - name: toystore + port: 80 + filters: + - type: ResponseHeaderModifier + responseHeaderModifier: + set: + - name: Cache-Control + value: max-age=31536000, immutable + - matches: # new rule added so it can be targeted in the RLP + - path: + type: Exact + value: "/toys/special" + method: GET + backendRefs: + - name: toystore + port: 80 ``` RateLimitPolicy: @@ -399,11 +432,12 @@ spec: name: toystore limits: - name: read-special - matches: - - path: - type: Exact - value: "/toys/special" - method: GET + rules: + - matches: + - path: + type: Exact + value: "/toys/special" + method: GET rates: - limit: 150 unit: second @@ -421,43 +455,50 @@ spec: configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-special-toys/0/d6f67e4a75c1f4a9b8030b4939d2c1bdf13f2c86493a71de25e33c3748fc0d3c" + descriptor_value: "toystore/toystore-special-toys/0/8267df441e5cda729095a9ea78db3abb2420855f7152379b4e88c90b8a4f562e" # SHA256 hashing of [{"matches":{"path":{"type":"Exact","value":"/toys/special"},"method":"GET"}}] ``` ```yaml limits: - conditions: - - context.request.http.route_matcher == "toystore/toystore-special-toys/0/d6f67e4a75c1f4a9b8030b4939d2c1bdf13f2c86493a71de25e33c3748fc0d3c" + - context.request.http.route_matcher == "toystore/toystore-special-toys/0/8267df441e5cda729095a9ea78db3abb2420855f7152379b4e88c90b8a4f562e" max_value: 150 seconds: 1 namespace: "*.toystore.com" ```
-#### Example 5. One limit, two matches +#### Example 5. One limit, many rules + +In this example, both HTTPRoute rules, i.e. `GET|POST /toys*` and `/assets/*`, are targeted by the same limit of 50rpm per username. -In this example, both route matching rules, `GET /toys*` and `POST /toys*`, are targeted by the same limit. This will cause the limit to be bound to the two HTTPRouteMatches, effectively applying 50rpm per username, regardless of the HTTP method `GET` or `POST`, at requests with path prefix equal to `/toys`. I.e. the rules are OR'ed, just like in the HTTPRoute itself. The same unique hash identifier is associated to both route rules. +Because the HTTPRoute has no other rule, this is technically equivalent to targeting the entire HTTPRoute and therefore similar to [Example 1](#example-1-minimal-example---network-resource-targeted-entirely-without-filtering-unconditional-and-unqualified-rate-limiting). But if the HTTPRoute had other rules or got other rules added afterwards, this would ensure the limit applies only to the two original route rules. ```yaml apiVersion: kuadrant.io/v2beta1 kind: RateLimitPolicy metadata: - name: toystore-per-endpoint-per-user + name: toystore-per-user spec: targetRef: group: gateway.networking.k8s.io kind: HTTPRoute name: toystore limits: - - matches: - - path: - type: PathPrefix - value: "/toys" - method: GET - - path: - type: PathPrefix - value: "/toys" - method: POST + - rules: + - matches: + - path: + type: PathPrefix + value: "/toys" + method: GET + - path: + type: PathPrefix + value: "/toys" + method: POST + - matches: + - path: + type: PathPrefix + value: "/assets/" rates: - limit: 50 duration: 1 @@ -481,7 +522,24 @@ spec: configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-per-endpoint-per-user/0/e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07" + descriptor_value: "toystore/toystore-per-user/0/d9edf43707d9a99b4f499055aa59ef6848e2346a638021944bd8f1efce22a8b3" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}},{"matches":{"path":{"type":"PathPrefix","value":"/assets/"}}}] + - metadata: + descriptor_key: "auth.identity.username" + metadata_key: + key: "envoy.filters.http.ext_authz" + path: + - segment: + key: "identity" + - segment: + key: "username" + - rules: + - paths: ["/assets/*"] + methods: ["*"] + hosts: ["*.toystore.com"] + configurations: + - generic_key: + descriptor_key: "context.request.http.route_matcher" + descriptor_value: "toystore/toystore-per-user/0/d9edf43707d9a99b4f499055aa59ef6848e2346a638021944bd8f1efce22a8b3" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}},{"matches":{"path":{"type":"PathPrefix","value":"/assets/"}}}] - metadata: descriptor_key: "auth.identity.username" metadata_key: @@ -496,7 +554,7 @@ spec: ```yaml limits: - conditions: - - context.request.http.route_matcher == "toystore/toystore-per-endpoint-per-user/0/e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07" + - context.request.http.route_matcher == "toystore/toystore-per-user/0/d9edf43707d9a99b4f499055aa59ef6848e2346a638021944bd8f1efce22a8b3" variables: - auth.identity.username max_value: 50 @@ -507,9 +565,9 @@ spec: #### Example 6. Targeting the Gateway -Targeting a Gateway is a shortcut to targeting individually each HTTPRoute pointing to the gateway, without any filtering based on the `matches` field. +Targeting a Gateway is a shortcut to targeting all individual HTTPRoutes referencing to the gateway as parent, without any filtering based on the `rules` field. -> **Note:** it is hard to give any additional meaning and context to this without going into [defaults and overrides](https://gateway-api.sigs.k8s.io/references/policy-attachment/#hierarchy). +This differs from [Example 1](#example-1-minimal-example---network-resource-targeted-entirely-without-filtering-unconditional-and-unqualified-rate-limiting) because, by targeting the gateway rather than an individual HTTPRoute, the RLP applies automatically to all HTTPRoutes pointing to the gateway, including routes created before and after the creation of the RLP. Moreover, all those routes will share the same limit counters specified in the RLP. ```yaml apiVersion: kuadrant.io/v2beta1 @@ -530,31 +588,28 @@ spec:
How is this RLP implemented under the hood? + Because there is no matcher in the Gateway, this limit rather have to define a generic key descriptor that is valid for all routes – i.e. without the [artificial Limitador condition](#artificial-limitador-condition-for-rules) associated with the `rules` field, such as one that identifies the RLP only. + ```yaml gateway_actions: - - rules: - - paths: ["/toys*"] - methods: ["GET"] - hosts: ["*.toystore.com"] - - paths: ["/toys*"] - methods: ["POST"] - hosts: ["*.toystore.com"] - configurations: + - configurations: - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "toystore/toystore-simple-infra-rl/0/e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07" + descriptor_value: "toystore/toystore-simple-infra-rl" ``` ```yaml limits: - conditions: - - context.request.http.route_matcher == "toystore/toystore-simple-infra-rl/0/e6064bc3504d2fbc972b3432bec494b3af8fb760e480ba42bb72c1574e57be07" + - context.request.http.route_matcher == "toystore/toystore-simple-infra-rl" max_value: 5 seconds: 1 namespace: "*.toystore.com" ```
+> **Note:** Additional meaning and context may be given to this use case in the future, when discussing [defaults and overrides](https://gateway-api.sigs.k8s.io/references/policy-attachment/#hierarchy). + ### Comparison to current RateLimitPolicy @@ -588,7 +643,7 @@ spec: - + - + - + @@ -660,7 +715,7 @@ spec: ## Reference-level explanation [reference-level-explanation]: #reference-level-explanation -By completely dropping out the `configurations` field from the RLP, [composing the RL descriptor actions](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter#composing-actions) is now done based essentially on the selectors listed in the `when` conditions and the `counters`, plus artificial generic conditions used to match the specific route rule (when `matches` is specified). +By completely dropping out the `configurations` field from the RLP, [composing the RL descriptor actions](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter#composing-actions) is now done based essentially on the selectors listed in the `when` conditions and the `counters`, plus artificial generic conditions used to match the specific route rule (when the `rules` is non-empty or implicit). ### Well-known selectors @@ -668,7 +723,7 @@ Each selector is a direct reference to a path within a well-known data structure The well-known data structure for building RL descriptor actions resembles Authorino's ["Authorization JSON"](https://github.com/Kuadrant/authorino/blob/main/docs/architecture.md#the-authorization-json), whose `context` component consists of Envoy's [`AttributeContext`](https://pkg.go.dev/github.com/envoyproxy/go-control-plane/envoy/service/auth/v3?utm_source=gopls#AttributeContext) type of the external authorization API, marshalled as JSON. Compared to the more generic [`RateLimitRequest`](https://pkg.go.dev/github.com/envoyproxy/go-control-plane@v0.11.0/envoy/service/ratelimit/v3#RateLimitRequest) struct, the `AttributeContext` provides a more structured and arguibly more intuitive relation between the data sources for the RL descriptors actions and their corresponding key names through which the values are referred within the RLP, in a context of serving predominantly for HTTP-based APIs. -To keep compatibility with the [Envoy Rate Limit protocol](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter), the well-known data structure can optionally be extended with the `RateLimitRequest`, thus resulting in the following final structure. +To keep compatibility with the [Envoy Rate Limit API](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter), the well-known data structure can optionally be extended with the `RateLimitRequest`, thus resulting in the following final structure. ```yaml context: # Envoy's Ext-Authz `CheckRequest.AttributeContext` type @@ -687,12 +742,12 @@ context: # Envoy's Ext-Authz `CheckRequest.AttributeContext` type method: … headers: {…} - auth: # Dynamic metadata exported by the external authorization service +auth: # Dynamic metadata exported by the external authorization service - ratelimit: # Envoy's Rate Limit `RateLimitRequest` type - domain: … # generated by the Kuadrant controller - descriptors: {…} # descriptors configured by the user directly in the proxy (not generated by the Kuadrant controller, if allowed) - hitsAddend: … # only in case we want to allow users to refer to this value in a policy +ratelimit: # Envoy's Rate Limit `RateLimitRequest` type + domain: … # generated by the Kuadrant controller + descriptors: {…} # descriptors configured by the user directly in the proxy (not generated by the Kuadrant controller, if allowed) + hitsAddend: … # only in case we want to allow users to refer to this value in a policy ``` ### Mechanics of generating RL descriptor actions @@ -700,9 +755,9 @@ context: # Envoy's Ext-Authz `CheckRequest.AttributeContext` type From the perspective of a user who writes a RLP, the selectors used in then `when` and `counters` fields are paths to the well-known data structure ([_well-known selectors_](#well-known-selectors)). While desiging a policy, the user intuitively pictures the well-known data structure and designs each rule (each limit) thinking on the possible values assumed by each of those paths in the data plane. For example, The user story: -> _Whenever the context is an HTTP request sent to `dolls.toystore.com` (`context.request.http.host`), I want a rate limit of 50 rps per distinct user (`auth.identity.username`)._ +> _Whenever the context is an HTTP request sent to `dolls.toystore.com` hostname (`context.request.http.host`), I want a rate limit of 50 rps per distinct user (`auth.identity.username`)._ -Materializes as the following RLP: +...materializes as the following RLP: ```yaml apiVersion: kuadrant.io/v2beta1 @@ -728,11 +783,11 @@ spec: - auth.identity.username ``` -The following selectors are to be interpreted: +The following selectors are to be interpreted by the RLP controller: - `context.request.http.host` - `auth.identity.username` -The RLP controller uses a map to translate each selector into its corresponding descriptor action. Roughly described: +The RLP controller uses a map to translate each selector into its corresponding descriptor action. (Roughly described:) ``` context.source.address → source_cluster(...) # TBC @@ -764,13 +819,13 @@ rate_limits: key: "username" ``` -### Artificial Limitador condition for `matches` +### Artificial Limitador condition for `rules` -For each limit, the RLP controller will generate an artificial Limitador condition that ensures that the limit applies only when that one filterred matching rule is honoured to serve the request. This can be implemented with a 2-step procedure: -1. generate an unique identifier for the limit – e.g. by applying any sufficiently entropic hash function of choice to the `matches` block of the limit; -2. associate a `generic_key` type descriptor action to the each `HTTPRouteMatch` targeted by the limit. +For each limit with non-empty (or implicit) `rules` field, the RLP controller will generate an artificial Limitador condition that ensures that the limit applies only when the filterred rules are honoured when serving the request. This can be implemented with a 2-step procedure: +1. generate an unique identifier for the limit – e.g. by applying any sufficiently entropic hash function of choice to the `rules` block of the limit; +2. associate a `generic_key` type descriptor action to the each `HTTPRouteRule` targeted by the limit. -For example, given the following RLP ([Example 2](#example-2-specific-route-matching-rule-targeted-conditions-counter-qualifiers-and-multiple-rates)): +For example, given the following RLP: ```yaml apiVersion: kuadrant.io/v2beta1 @@ -783,12 +838,16 @@ spec: kind: HTTPRoute name: toystore limits: - - name: readers + - name: toys matches: - path: type: PathPrefix value: "/toys" method: GET + - path: + type: PathPrefix + value: "/toys" + method: POST rates: - limit: 50 duration: 1 @@ -798,12 +857,11 @@ spec: operator: neq value: admin - - name: writers + - name: assets matches: - path: type: PathPrefix - value: "/toys" - method: POST + value: "/assets/" rates: - limit: 5 duration: 1 @@ -814,7 +872,7 @@ spec: value: admin ``` -Apart from the descriptor action (associated with both routes): +Apart from the following descriptor action associated with both routes: ```yaml - metadata: @@ -837,15 +895,15 @@ auth.identity.group != "admin" The following additional artificial descriptor actions will be generated: ```yaml -# associated with route GET /toys* +# associated with route rule GET|POST /toys* - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05" # SHA256 hashing of [{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"}] + descriptor_value: "c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/toys"},"method":"GET"},{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}}] -# associated with route POST /toys* +# associated with route rule /assets/* - generic_key: descriptor_key: "context.request.http.route_matcher" - descriptor_value: "4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d" # SHA256 hashing of [{"path":{"type":"PathPrefix","value":"/toys"},"method":"POST"}] + descriptor_value: "643f8d429ff65b62bf9d69bf201461ce3bf5f47f0a5d54fd519d118fa91cce66" # SHA256 hashing of [{"matches":{"path":{"type":"PathPrefix","value":"/assets/"}}}] ``` ...and their corresponding Limitador conditions. @@ -854,27 +912,37 @@ In the end, the following Limitador configuration is yielded: ```yaml - conditions: - - context.request.http.route_matcher == "2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05" + - context.request.http.route_matcher == "c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f" - auth.identity.group != "admin" max_value: 50 seconds: 60 namespace: "*.toystore.com" - conditions: - - context.request.http.route_matcher == "4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d" + - context.request.http.route_matcher == "643f8d429ff65b62bf9d69bf201461ce3bf5f47f0a5d54fd519d118fa91cce66" - auth.identity.group != "admin" max_value: 5 seconds: 60 namespace: "*.toystore.com" ``` -The route matcher hash identifiers can be qualified with a plain identifier of the RLP itself (`namespace/name`) and limit where it is defined (`name` of the limit when available, index in the array of limits otherwise), thus making the identifier unique to scope of the entire cluster. E.g.: the two unique identifiers from above, prefixed with the unique limit qualifier, become respectively `toystore/toystore-non-admin-users/readers/2718701ec9bfd79132e58d92aed722489443094bf9c616e1b74361fe68360f05` and `toystore/toystore-non-admin-users/writers/4f70fc57ad52a2664e3920f373633a9b2b2f4f58f17b39a8d3a3a3485fd91c4d`. This has a consequence to the readability of the identifier, but also and more importantly it ensures uniqueness of the counters in Limitador. By qualifying (or salting) the identifiers, two limits or two RLPs that happen to target the same `HTTPRouteMatches` will not register the same counters in Limitador, but be treated as independent ones instead. +The limit-to-route rule matcher identifiers can be qualified with a plain identifier of the RLP itself (`namespace/name`) and the limit where it is defined (`name` of the limit when available, index in the array of limits otherwise), thus making the identifier unique to the scope of the entire cluster. + +E.g.: the two unique identifiers from above, prefixed with the unique limit qualifier, become respectively `toystore/toystore-non-admin-users/toys/c7a7782586bc506e89a88d69b2747e52997474bac19bdabe03be2a04fbd9dc0f` and `toystore/toystore-non-admin-users/assets/643f8d429ff65b62bf9d69bf201461ce3bf5f47f0a5d54fd519d118fa91cce66`. + +This has a consequence to the readability of the identifier, but also and more importantly it ensures uniqueness of the counters in Limitador. By qualifying (or salting) the identifiers, two limits or two RLPs that happen to target the same HTTPRouteRules will not register the same counters in Limitador, but be treated as independent ones instead. + +### Support in wasm shim and Envoy RL API + +This proposal tries to keep compatibility with the [Envoy API for rate limit](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter) and does not introduce any new requirement that otherwise would require the use of [wasm shim](https://github.com/Kuadrant/wasm-shim) to be implemented. + +In the case of implementation of this proposal in the wasm shim, all types of matchers supported by the [HTTPRouteMatch](https://gateway-api.sigs.k8s.io/references/spec/#gateway.networking.k8s.io/v1beta1.HTTPRouteMatch) type of Gateway API must be also supported in the `rate_limit_policies.gateway_actions.rules` field of the [wasm plugin configuration](https://github.com/Kuadrant/kuadrant-operator/blob/faf41ff6b08df27946a663c34a5736476578dea5/pkg/rlptools/wasm_utils.go#L109). These include matchers based on path (prefix, exact), headers, query string parameters and method. ## Drawbacks [drawbacks]: #drawbacks -**Two types of conditions – `matches` and `when` conditions**
-Although with different meanings (evaluates in the gateway vs. evaluated in Limitador) and capable of expressing different sets of rules (HTTPRouteMatch-related rules vs. HTTP request-related and non HTTP request-related rules), an overlap between these two types and ways of representing conditions does exist. +**Two types of conditions – `rules` and `when` conditions**
+Although with different meanings (evaluates in the gateway vs. evaluated in Limitador) and capable of expressing different sets of rules (HTTPRouteRule-targeting rules vs. HTTP request-related and non HTTP request-related "soft" conditions), an overlap between these two types and ways of representing conditions does exist. **Prone to consistency issues**
Typos and updates to the HTTPRoute can easily cause a mismatch and invalidate a RLP. @@ -959,5 +1027,5 @@ Most implementations currently orbiting around Gateway API (e.g. Istio, Envoy Ga ## Future possibilities [future-possibilities]: #future-possibilities -- Port `matches` and the semantics around it to the `AuthPolicy` API (aka "KAP v2"). +- Port `rules` and the semantics around it to the `AuthPolicy` API (aka "KAP v2"). - Defaults and overrides, either along the lines of [architecture#4](https://github.com/Kuadrant/architecture/pull/4) or [architecture#10](https://github.com/Kuadrant/architecture/pull/10).
spec.rateLimits.configurations as a list of "variables assignments" and direct exposure of Envoy's RL descriptor actions APIDescriptor actions composed implicitly from selectors used in the limit definitions (spec.limits.when.selector and spec.limits.counters) plus a fixed identifier of the route rule (spec.limits.matches)Descriptor actions composed implicitly from selectors used in the limit definitions (spec.limits.when.selector and spec.limits.counters) plus a fixed identifier of the route rules (spec.limits.rules)
  • Abstract the Envoy-specific concepts of "actions" and "descriptors"
    • @@ -609,19 +664,19 @@ spec:
Limitador conditions independent from the route rulesArtificial Limitador condition injected for each route matchArtificial Limitador condition injected for each route rule
    -
  • Ensure the limit is enforced only for corresponding filtered HTTPRouteMatch
    • +
  • Ensure the limit is enforced only for corresponding filtered HTTPRouteRule
spec.rateLimits.rules ⊆ httproute.spec.rulesspec.limits.matches ∊ httproute.spec.rules.matchesspec.limits.rules.matches == httproute.spec.rules.matches
    -
  • Perfect match to HTTPRoute matching rules
    • +
  • Perfect match to HTTPRoute rules
  • Simpler to solve for defaults and overrides