Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EIP allocations and subnet ID order is not respected #3915

Open
dgadodia opened this issue Oct 28, 2024 · 0 comments
Open

EIP allocations and subnet ID order is not respected #3915

dgadodia opened this issue Oct 28, 2024 · 0 comments
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@dgadodia
Copy link

dgadodia commented Oct 28, 2024

Describe the bug
We are using service.beta.kubernetes.io/aws-load-balancer-eip-allocations and service.beta.kubernetes.io/aws-load-balancer-subnets annotations to create a mapping between EIP and subnets.
As per docs , EIP allocations has this side note Length/order must match subnets

However this order is not respected because we end up sorting the list of subnets.

func sortSubnetsByID(subnets []ec2types.Subnet) {

As a result, we end up mapping EIP to subnets which we don't intend (want) to

Steps to reproduce

Deployed k8s manifest example

apiVersion: v1
kind: Service
metadata:
  annotations:
    kubernetes.io/ingress.class: alb
    service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags: service=myapp-gateway,vertical=platform,cost_center=infrastructure
    service.beta.kubernetes.io/aws-load-balancer-alpn-policy: HTTP2Preferred
    service.beta.kubernetes.io/aws-load-balancer-attributes: deletion_protection.enabled=true,access_logs.s3.enabled=true,access_logs.s3.bucket=myapp-logs-internal-us-west-2,access_logs.s3.prefix=myapp-gateway-access-logs
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: 'false'
    service.beta.kubernetes.io/aws-load-balancer-eip-allocations: eip-w,eip-x,eip-y,eip-z
    service.beta.kubernetes.io/aws-load-balancer-ip-address-type: dualstack
    service.beta.kubernetes.io/aws-load-balancer-name: myapp-gateway
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
    service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: '*'
    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:us-west-2:1234567890:certificate/some-random-uuid
    service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: ELBSecurityPolicy-FS-1-2-Res-2020-10
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
    service.beta.kubernetes.io/aws-load-balancer-subnets: subnet-random-ewewe, subnet-random-wewe, subnet-random-qrqr, subnet-random-rqrq
    service.beta.kubernetes.io/aws-load-balancer-target-group-attributes: proxy_protocol_v2.enabled=true,deregistration_delay.timeout_seconds=240,deregistration_delay.connection_termination.enabled=true
    service.beta.kubernetes.io/aws-load-balancer-type: external
  labels:
    app: myapp-gateway
    install.operator.istio.io/owning-resource: unknown
    istio: ingressgateway
    istio.io/rev: 1-22
    operator.istio.io/component: IngressGateways
    release: istio
  name: myapp-gateway
  namespace: istio-system
spec:
  loadBalancerSourceRanges:
  - 0.0.0.0/0
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: 8080
  selector:
    app: myapp-gateway
    istio: ingressgateway
  type: LoadBalancer

Expected outcome
A concise description of what you expected to happen.

When we set these annotations

service.beta.kubernetes.io/aws-load-balancer-eip-allocations: eip-w, eip-x, eip-y, eip-z
service.beta.kubernetes.io/aws-load-balancer-subnets: subnet-random-ewewe, subnet-random-wewe, subnet-random-qrqr, subnet-random-rqrq

The mapping of the NLB created should be
eip-w -> subnet-random-ewewe
eip-x -> subnet-random-wewe
eip-y -> subnet-random-qrqr,
eip-z -> subnet-random-rqrq

Current outcome
The mapping which ends up being created is

>>> sorted(["subnet-random-ewewe", "subnet-random-wewe", "subnet-random-qrqr", "subnet-random-rqrq"])
['subnet-random-ewewe', 'subnet-random-qrqr', 'subnet-random-rqrq', 'subnet-random-wewe']

eip-w -> subnet-random-ewewe
eip-x -> subnet-random-qrqr
eip-y -> subnet-random-rqrq
eip-z -> subnet-random-wewe

Environment

  • AWS Load Balancer controller version: v2.8.3
  • Kubernetes version: 1.30
  • Using EKS (yes/no), if so version? yes

Additional Context:

@shraddhabang shraddhabang added the kind/bug Categorizes issue or PR as related to a bug. label Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

No branches or pull requests

2 participants