Support loadBalancerClass in service reconciliation

[pebrc]

Fixes #7691

From the k8s docs:

// loadBalancerClass is the class of the load balancer implementation this Service belongs to.
// If specified, the value of this field must be a label-style identifier, with an optional prefix,
// e.g. "internal-vip" or "example.com/internal-vip". Unprefixed names are reserved for end-users.
// This field can only be set when the Service type is 'LoadBalancer'. If not set, the default load
// balancer implementation is used, today this is typically done through the cloud provider integration,
// but should apply for any default implementation. If set, it is assumed that a load balancer
// implementation is watching for Services with a matching class. Any default load balancer
// implementation (e.g. cloud providers) should ignore Services that set this field.
// This field can only be set when creating or updating a Service to type 'LoadBalancer'.
// Once set, it can not be changed. This field will be wiped when a service is updated to a non 'LoadBalancer' type.

AWS EKS uses this field on EKS clusters > 1.24 that have upgraded to the new AWS LoadBalancer Controller

Behaviour this PR should implement:

• default the loadBalancerClass from the server side values if no value is set in the ECK service spec
• recreate the service if the value set in the ECK service spec differes from the server side value
• allow resetting the value only if the type of the service is no longer LoadBalancer

[barkbay]

I'm also wondering if we should not also update applyServerSideValues for ExternalTrafficPolicy and AllocateLoadBalancerNodePorts, I think needsUpdate(...) always returns true on my test cluster with the following diff:

  v1.ServiceSpec{
        ... // 8 identical fields
        LoadBalancerSourceRanges:      nil,
        ExternalName:                  "",
-       ExternalTrafficPolicy:         "",
+       ExternalTrafficPolicy:         "Cluster",
        HealthCheckNodePort:           0,
        PublishNotReadyAddresses:      false,
        SessionAffinityConfig:         nil,
        IPFamilies:                    {"IPv4"},
        IPFamilyPolicy:                &"SingleStack",
-       AllocateLoadBalancerNodePorts: nil,
+       AllocateLoadBalancerNodePorts: &true,
        LoadBalancerClass:             &"foo",
        InternalTrafficPolicy:         &"Cluster",
  }

[pebrc]

I'm also wondering if we should not also update applyServerSideValues for ExternalTrafficPolicy and AllocateLoadBalancerNodePorts

I think that's a good idea. I will add it to this PR

[barkbay]

LGTM!