Mesh gateway svc - support for additional annotations and loadBalancerIp

[wjrbetts]

Description

So that a user can have more control over how a cloud load balancer is created for a mesh gateway service, they should be able to specify a loadBalancerIp and additional annotations. These would then get included in the mesh gateway service that is created.

Perhaps something like:

apiVersion: kuma.io/v1alpha1
kind: MeshGatewayInstance
metadata:
  name: edge-gateway
spec:
  replicas: 1
  serviceType: LoadBalancer
  loadBalancerIp: 1.2.3.4
  additionalAnnotations:
    annotation1Name: annotation1Value
    annotation2Name: annotation2Value
  tags:
    kuma.io/service: edge-gateway

[lahabana]

Triage: We could just have a serviceTemplate in our crd this would enable users to do what they want and we'll complete/validate it with Gateway specific things (namely ports)

[github-actions]

This issue was inactive for 30 days it will be reviewed in the next triage meeting and might be closed.
If you think this issue is still relevant please comment on it promptly or attend the next triage meeting.

[ravish-kumar-maersk]

This would be a very handy feature for cloud-based LB.

Any ETA when this can be available?

[ravish-kumar-maersk]

jakubdyszkiewicz Please update on the estimated time for this feature !!!

[lahabana]

@ravish-kumar-maersk can you show an example of what you'd like to do? Curious to see other examples as we're not entirely sure which road to take on this feature

[ravish-kumar-maersk]

@lahabana
I am pretty much aligned with the use-case as posted in the original description.

With cloud-based LB we can assign a reserved IP address and add annotations directly to MeshGatewayInstance spec.

[johnharris85]

Not sure a ServiceTemplate (kinda like our new ContainerPatch makes sense here initially. There aren't very many fields in a Service (and even not all of those make sense to customize). In the near term I'd guess the 90% use case would be adding custom annotations (as that's how a lot of the cloud loadbalancer controllers configure the provisioned LB). Also we could add LB IP based on the OP request (edit: maybe we shouldn't, this is deprecated upstream: kubernetes/kubernetes#107235), and then monitor to see if there's more interest in customizing additional fields?

Others I can see that might be next: loadBalancerSourceRanges, externalTrafficPolicy?

[michaelbeaumont]

loadBalancerIP isn't supported

[johnharris85]

loadBalancerIP isn't supported

It's deprecated upstream, do we want to support it?

[michaelbeaumont]

loadBalancerIP isn't supported

It's deprecated upstream, do we want to support it?

Good point. It looks like it's still potentially useful to support, if a cloud provider depends on it. But I haven't looked into it.

I think this issue can serve as a "make sure it's possible to configure the Service from the MeshGatewayInstance as much as necessary" issue, as something to ensure before GA.

[wjrbetts]

loadBalancerIP isn't supported

It's deprecated upstream, do we want to support it?

Good point. It looks like it's still potentially useful to support, if a cloud provider depends on it. But I haven't looked into it.

I think this issue can serve as a "make sure it's possible to configure the Service from the MeshGatewayInstance as much as necessary" issue, as something to ensure before GA.

Our use case is to have an Azure Load Balancer listening on our static public IP. To do that today loadBalancerIp is required in the service (as well as annotations). Currently we are having to work around this by creating our own kubernetes service alongside the kuma-generated one and specifying loadBalancerIp there.

[michaelbeaumont]

I'll close this for now until we have more need for Service customization