Enhancing Understanding of Gateway API Usage

[obervinov]

Hello everyone!

I’m exploring the possibility of migrating from Ingress (nginx) to Gateway API, and I have some questions about this process. I would like to better understand the recommended practices for operating and using Gateway API in Kubernetes clusters.

As I understand from the documentation, there are two permissible models for using Gateway API in Kubernetes, and they seem less flexible to me than the usual usage of ingress-nginx:

• сross namespaces - where the Gateway, its pod (e.g., istio-proxy), and TLS certificates for HTTPS listeners are deployed in an infrastructure namespace (e.g., istio-ingressgateway), and development teams publish HTTPRoutes in their namespaces using a single infrastructure Gateway.
• self hosted - where each team creates its own instance of the Gateway, its pod, and TLS certificate in their namespace.

I see several significant drawbacks to these two methods of using Gateway API:

• in the first case, we deprive the development teams of the ability to fully manage the published hostnames and their certificates without involving infrastructure teams, and we constrain the infrastructure Gateways to wildcard hostnames and wildcard certificates. If it’s not possible (for example, with a cert-manager issuer) to issue wildcard certificates, this scheme becomes unviable.
• in the second case, for each created Gateway in each namespace, we get +1 pod, +1 service, and +1 load balancer with the cloud provider, which significantly increases the cost of maintaining this solution.

Is there a more flexible strategy for operating Gateway API that is more similar to how it works with ingress-nginx?
Where an infrastructure pod with nginx, its service, and a cloud load balancer are deployed in the infrastructure namespace, while the publication of web resources with their hostnames and TLS certificates is delegated to each specific namespace.

I would appreciate any recommendations.
Thanks!

[youngnick]

Thanks for asking this @obervinov, you're basically correct.

There's a little more nuance to the usual process of running cross-namespace Gateways (in that you can also have the TLS certificates stored in a separate namespace as well using ReferenceGrant), and it's certainly not intended that there has to be only a single Gateway -although as you mention later, there's generally a 1:1 Gateway:External Load Balancer mapping, which carries cost implications, for sure.

And you're right about the disadvantages of having everything in a single namespace as well.

However, I think that it's easy to miss the flaw hiding in the flexibility available in the Ingress object - and that's around how it doesn't separate the various personas who are interested in that config very well.

For example, the published hostnames in Ingress are actually a global resource that's specified at a namespace level. We haven't entirely solved this with Gateways (in that hostnames are still global config specified in a namespaced resource), but we believe that being able to have an object boundary (Between Gateway and HTTPRoute) that aligns with the persona boundary (between Chihiro the Cluster Admin and Ana the Application Developer) is a safer and overall better design.

Your request (about being able to split the TLS config from the Gateway config) is actually being looked at in the PR in #3213 - the new experimental ListenerSet object in that PR is designed to allow infrastructure owners and implementations to break out Listener config into a separate object, to support cases where the Chihiro wants to be able to allow Ana to manage her own TLS Config.

[obervinov]

@youngnick, thank you very much for your detailed reply!
ListnerSet seems to be exactly what we need, we are following this PR with interest