Set initial node_labels for default workers and worker pools #550
Conversation
|
👍 I think this is useful, likewise, a similar feature to support taints (e.g. |
| @@ -61,6 +61,9 @@ systemd: | |||
| --lock-file=/var/run/lock/kubelet.lock \ | |||
| --network-plugin=cni \ | |||
| --node-labels=node.kubernetes.io/node \ | |||
| %{ for label in split(";", node_labels) } | |||
There was a problem hiding this comment.
Since var.node_labels is of type list, can't we get rid of the join/split here?
(Sorry if useless
There was a problem hiding this comment.
that was my initial take too, but tf only takes strings as template vars.
(hashicorp/terraform#9368 , etc..)
|
yup, also had taints in mind, similar pattern; that would work as a separate PR i guess (if acceptable) |
|
I think this is acceptable becauce it's already been reported and supported here: #429 I, too, second this feature. |
|
There's one more thing slightly related. While typhoon aims to keep things simple, my specific usecase requires cluster autoscaling for some one-time workload happening only on initialization. In order to fully support CA, i'd also need to be able to add tags to the workers autoscaling group (it uses those |
|
Yes, I'll take a No changes to tags. |
| @@ -86,6 +86,7 @@ data "template_file" "worker-config" { | |||
| cluster_dns_service_ip = cidrhost(var.service_cidr, 10) | |||
| cluster_domain_suffix = var.cluster_domain_suffix | |||
| cgroup_driver = local.flavor == "flatcar" && local.channel == "edge" ? "systemd" : "cgroupfs" | |||
| node_labels = join(";", var.node_labels) | |||
There was a problem hiding this comment.
Let's at least use a "," separator so this is less weird.
valer-cara
force-pushed
the
node-labels
branch
from
0dea9e4
to
77bac17
Compare
|
@dghubble - I've made the requested changes. Added node labels to aws (cl + fedora), gcp, azure. I've only tested the aws/cl setup so far. |
…` to workers - aws (cl, fedora) - google-cloud - azure
valer-cara
force-pushed
the
node-labels
branch
from
c72c16e
to
28306a4
Compare
|
Looks good. It'll need updates to docs and tutorials, but its probably simpler if I just add that. |
dghubble
changed the title
|
Thanks 👍 |
* Add `node_taints` variable to worker modules to set custom initial node taints on cloud platforms that support auto-scaling worker pools of heterogeneous nodes (i.e. AWS, Azure, GCP) * Worker pools could use custom `node_labels` to allowed workloads to select among differentiated nodes, while custom `node_taints` allows a worker pool's nodes to be tainted as special to prevent scheduling, except by workloads that explicitly tolerate the taint * Expose `daemonset_tolerations` in AWS, Azure, and GCP kubernetes cluster modules, to determine whether `kube-system` components should tolerate the custom taint (advanced use covered in docs) Rel: #550, #663 Closes #429
* Add `node_taints` variable to worker modules to set custom initial node taints on cloud platforms that support auto-scaling worker pools of heterogeneous nodes (i.e. AWS, Azure, GCP) * Worker pools could use custom `node_labels` to allowed workloads to select among differentiated nodes, while custom `node_taints` allows a worker pool's nodes to be tainted as special to prevent scheduling, except by workloads that explicitly tolerate the taint * Expose `daemonset_tolerations` in AWS, Azure, and GCP kubernetes cluster modules, to determine whether `kube-system` components should tolerate the custom taint (advanced use covered in docs) Rel: #550, #663 Closes #429
* Add `node_taints` variable to worker modules to set custom initial node taints on cloud platforms that support auto-scaling worker pools of heterogeneous nodes (i.e. AWS, Azure, GCP) * Worker pools could use custom `node_labels` to allowed workloads to select among differentiated nodes, while custom `node_taints` allows a worker pool's nodes to be tainted as special to prevent scheduling, except by workloads that explicitly tolerate the taint * Expose `daemonset_tolerations` in AWS, Azure, and GCP kubernetes cluster modules, to determine whether `kube-system` components should tolerate the custom taint (advanced use covered in docs) Rel: #550, #663 Closes #429
* Add `node_taints` variable to worker modules to set custom initial node taints on cloud platforms that support auto-scaling worker pools of heterogeneous nodes (i.e. AWS, Azure, GCP) * Worker pools could use custom `node_labels` to allowed workloads to select among differentiated nodes, while custom `node_taints` allows a worker pool's nodes to be tainted as special to prevent scheduling, except by workloads that explicitly tolerate the taint * Expose `daemonset_tolerations` in AWS, Azure, and GCP kubernetes cluster modules, to determine whether `kube-system` components should tolerate the custom taint (advanced use covered in docs) Rel: #550, #663 Closes #429
High level description of the change.
node_labelsvariable to kubernetes module (currently only for aws/container-linux)Testing
Launched a new cluster with extra labels that were successfully applied on the worker nodes.
This is a quick fix I needed for dedicated worker pools and nodeSelector. I guess that for this to be merged it'd require uniformity across all cloud/distro variants.
I'm submitting it just to see if this would be an acceptable feature. If so, I can add it to the rest of the platforms/distros.
Thanks!