Make affinity/antiaffinity topologyKey and required/preferred parameters configurable for podDistribution setting

[alexandv]

Hello,

We would like to have replicas spread across availability zones. Currently, the podDistribution settings only work with the hostname topologyKey as far as I can tell:
https://github.com/Altinity/clickhouse-operator/blob/master/pkg/model/affinity.go#L504
https://github.com/Altinity/clickhouse-operator/blob/master/pkg/model/affinity.go#L529
https://github.com/Altinity/clickhouse-operator/blob/master/pkg/model/affinity.go#L561

The only way we have found to have pods spread across zones now is to use one pod template per zone which is repetitive and error-prone.

I also noticed that pod affinity is currently enforced using PreferredDuringSchedulingIgnoredDuringExecution and pod anti affinity using RequiredDuringSchedulingIgnoredDuringExecution. Can we also make this configurable?

2 additional optional settings like this could work:

podDistribution:
- type: ReplicaAntiAffinity
  topologyKey: kubernetes.io/zone
  affType: preferredDuringSchedulingIgnoredDuringExecution

Thanks

[alex-zaitsev]

Make sense. We will plan it for the next release.

[alex-zaitsev]

topologyKey is added into 0.16.0 release.

[alexandv]

Thanks @alex-zaitsev . We will test it and let you know if there are any issues.

[emanuelioan]

Hey @alex-zaitsev , any update on supporting preferredDuringSchedulingIgnoredDuringExecution ? I'd like to spread replicas over multiple AZs and requiredDuringSchedulingIgnoredDuringExecution wouldn't work if I had more replicas than AZs.