Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[helm-chart] add values for setting annotations and labels for rollout-operator #6733

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
3 changes: 2 additions & 1 deletion operations/helm/charts/mimir-distributed/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ Entries should include a reference to the Pull Request that introduced the chang
* [ENHANCEMENT] Alerts: exclude `529` and `598` status codes from failure codes in `MimirRequestsError`. #7889
* [ENHANCEMENT] The new value `metaMonitoring.grafanaAgent.logs.clusterLabel` controls whether to add a `cluster` label and with what content to PodLogs logs. #7764
* [ENHANCEMENT] The new values `global.extraVolumes` and `global.extraVolumeMounts` adds volumes and volumeMounts to all pods directly managed by mimir-distributed. #7922
* [ENHANCEMENT] Add values for setting annotations and labels for rollout-operator. #6733
* [ENHANCEMENT] Smoke-test: Parameterized `backoffLimit` for smoke tests in Helm chart to accommodate slower startup environments like k3d. #8025
* [ENHANCEMENT] Add a volumeClaimTemplates section to the `chunks-cache`, `index-cache`, `metadata-cache`, and `results-cache` components. #8016
* [ENHANCEMENT] Add 'gateway.nginx.config.clientMaxBodySize' to the `gateway` to allow setting the maximum allowed size of the client request body. #7960 #8497
Expand Down Expand Up @@ -134,7 +135,7 @@ Entries should include a reference to the Pull Request that introduced the chang
* [BUGFIX] Let the unified gateway/nginx config listen on IPv6 as well. Followup to #5948. #6204
* [BUGFIX] Quote `checksum/config` when using external config. This allows setting `externalConfigVersion` to numeric values. #6407
* [BUGFIX] Update memcached-exporter to 0.14.1 due to CVE-2023-39325. #6861

## 5.1.4

* [BUGFIX] Update memcached-exporter to 0.14.1 due to CVE-2023-39325.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -573,6 +573,9 @@ which allows us to keep generating everything for the default zone.
"nodeSelector" ($rolloutZone.nodeSelector | default (dict) )
"replicas" $replicaPerZone
"storageClass" $rolloutZone.storageClass
"noDownscale" $rolloutZone.noDownscale
"downscaleLeader" $rolloutZone.downscaleLeader
"prepareDownscale" $rolloutZone.prepareDownscale
) -}}
{{- end -}}
{{- if $componentSection.zoneAwareReplication.migration.enabled -}}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,22 @@ metadata:
name: {{ include "mimir.resourceName" $args }}
labels:
{{- include "mimir.labels" $args | nindent 4 }}
{{- if (eq $rolloutZone.noDownscale true )}}
grafana.com/no-downscale: {{$rolloutZone.noDownscale}}
{{- else }}
{{- if (eq $rolloutZone.prepareDownscale true )}}
grafana.com/prepare-downscale: {{$rolloutZone.prepareDownscale}}
{{- end }}
{{- end }}
annotations:
{{- include "mimir.componentAnnotations" $args | nindent 4 }}
{{- if $rolloutZone.prepareDownscale }}
grafana.com/prepare-downscale-http-path: ingester/prepare-shutdown
grafana.com/prepare-downscale-http-port: {{ include "mimir.serverHttpListenPort" . }}
{{- end -}}
{{- if $rolloutZone.downscaleLeader }}
grafana.com/rollout-downscale-leader: $rolloutZone.downscaleLeader
{{- end }}
namespace: {{ .Release.Namespace | quote }}
spec:
replicas: {{ $rolloutZone.replicas }}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,23 @@ metadata:
name: {{ include "mimir.resourceName" $args }}
labels:
{{- include "mimir.labels" $args | nindent 4 }}
{{- if (eq $rolloutZone.noDownscale true )}}
grafana.com/no-downscale: {{$rolloutZone.noDownscale}}
{{- else }}
{{- if (eq $rolloutZone.prepareDownscale true )}}
grafana.com/prepare-downscale: {{$rolloutZone.prepareDownscale}}
grafana.com/min-time-between-zones-downscale: 12h
{{- end }}
{{- end }}
annotations:
{{- include "mimir.componentAnnotations" $args | nindent 4 }}
{{- if $rolloutZone.prepareDownscale }}
grafana.com/prepare-downscale-http-path: ingester/prepare-shutdown
grafana.com/prepare-downscale-http-port: {{ include "mimir.serverHttpListenPort" . }}
{{- end -}}
{{- if $rolloutZone.downscaleLeader }}
grafana.com/rollout-downscale-leader: $rolloutZone.downscaleLeader
{{- end }}
namespace: {{ .Release.Namespace | quote }}
spec:
podManagementPolicy: {{ .Values.ingester.podManagementPolicy }}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,23 @@ metadata:
name: {{ include "mimir.resourceName" $args }}
labels:
{{- include "mimir.labels" $args | nindent 4 }}
{{- if (eq $rolloutZone.noDownscale true )}}
grafana.com/no-downscale: {{$rolloutZone.noDownscale}}
{{- else }}
{{- if (eq $rolloutZone.prepareDownscale true )}}
grafana.com/prepare-downscale: {{$rolloutZone.prepareDownscale}}
grafana.com/min-time-between-zones-downscale: 30m
{{- end }}
{{- end }}
annotations:
{{- include "mimir.componentAnnotations" $args | nindent 4 }}
{{- if $rolloutZone.prepareDownscale }}
grafana.com/prepare-downscale-http-path: ingester/prepare-shutdown
grafana.com/prepare-downscale-http-port: {{ include "mimir.serverHttpListenPort" . }}
{{- end -}}
{{- if $rolloutZone.downscaleLeader }}
grafana.com/rollout-downscale-leader: $rolloutZone.downscaleLeader
{{- end }}
namespace: {{ .Release.Namespace | quote }}
spec:
podManagementPolicy: {{ .Values.store_gateway.podManagementPolicy }}
Expand Down
72 changes: 72 additions & 0 deletions operations/helm/charts/mimir-distributed/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -753,6 +753,14 @@ alertmanager:
# If set to "-", then use `storageClassName: ""`, which disables dynamic provisioning
# If undefined or set to null (the default), then fall back to the value of `alertmanager.persistentVolume.storageClass`.
storageClass: null
# -- noDownscale adds a label that can be used by the rollout-operator as documented in the rollout-operator repo: https://github.com/grafana/rollout-operator#webhooks
noDownscale: false
# -- downscaleLeader is an Annotation used by the rollout-operator
# If defined, downscaleLeader: <downscaleLeader>
# If undefined or set to null (the default), no Annotation is set
downscaleLeader: null
# -- prepareDownscale adds labels and annotations for the rollout-operator to prepare downscaling: https://github.com/grafana/rollout-operator#how-scaling-up-and-down-works
prepareDownscale: false
# -- Name of the zone, used in labels and selectors. Must follow Kubernetes naming restrictions: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
- name: zone-b
# -- nodeselector to restrict where pods of this zone can be placed. E.g.:
Expand All @@ -766,6 +774,14 @@ alertmanager:
# If set to "-", then use `storageClassName: ""`, which disables dynamic provisioning
# If undefined or set to null (the default), then fall back to the value of `alertmanager.persistentVolume.storageClass`.
storageClass: null
# -- noDownscale adds a label that can be used by the rollout-operator as documented in the rollout-operator repo: https://github.com/grafana/rollout-operator#webhooks
noDownscale: false
# -- downscaleLeader is an Annotation used by the rollout-operator
# If defined, downscaleLeader: <downscaleLeader>
# If undefined or set to null (the default), no Annotation is set
downscaleLeader: null
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be great if we can set the downscale leader automatically in the zonal templates. That way automatica downscaling can be just a single boolean toggle for the ingester/store-gateway which automatically sets up downscale leader and prepare downscale for the 3 zones.

But i think this PR has been lingering for long enough to try to slow it down further. happy to not include this change in the PR if you want.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

having said that - i think having noDownscale: false would be a safer change because it will mean folks can keep running the chart as they used to. If we get automatic downscaling automatically set up like I suggested above, then I think it's viable to have it on by default.

I think it makes sense to try to minimize the added manual effort for folks after they upgrade to a chart version which contains this PR

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think that changing noDownscale: false makes sense. and as you mentioned the PR has been lingering for a while and i just don't have the bandwidth to deal with it.

# -- prepareDownscale adds labels and annotations for the rollout-operator to prepare downscaling: https://github.com/grafana/rollout-operator#how-scaling-up-and-down-works
prepareDownscale: false
# -- Name of the zone, used in labels and selectors. Must follow Kubernetes naming restrictions: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
- name: zone-c
# -- nodeselector to restrict where pods of this zone can be placed. E.g.:
Expand All @@ -779,6 +795,14 @@ alertmanager:
# If set to "-", then use `storageClassName: ""`, which disables dynamic provisioning
# If undefined or set to null (the default), then fall back to the value of `alertmanager.persistentVolume.storageClass`.
storageClass: null
# -- noDownscale adds a label that can be used by the rollout-operator as documented in the rollout-operator repo: https://github.com/grafana/rollout-operator#webhooks
noDownscale: false
# -- downscaleLeader is an Annotation used by the rollout-operator
# If defined, downscaleLeader: <downscaleLeader>
# If undefined or set to null (the default), no Annotation is set
downscaleLeader: null
# -- prepareDownscale adds labels and annotations for the rollout-operator to prepare downscaling: https://github.com/grafana/rollout-operator#how-scaling-up-and-down-works
prepareDownscale: false

distributor:
# Setting it to null will produce a deployment without replicas set, allowing you to use autoscaling with the deployment
Expand Down Expand Up @@ -1080,6 +1104,14 @@ ingester:
# If set to "-", then use `storageClassName: ""`, which disables dynamic provisioning
# If undefined or set to null (the default), then fall back to the value of `ingester.persistentVolume.storageClass`.
storageClass: null
# -- noDownscale adds a label that can be used by the rollout-operator as documented in the rollout-operator repo: https://github.com/grafana/rollout-operator#webhooks
noDownscale: false
# -- downscaleLeader is an Annotation used by the rollout-operator
# If defined, downscaleLeader: <downscaleLeader>
# If undefined or set to null (the default), no Annotation is set
downscaleLeader: null
# -- prepareDownscale adds labels and annotations for the rollout-operator to prepare downscaling: https://github.com/grafana/rollout-operator#how-scaling-up-and-down-works
prepareDownscale: false
# -- Name of the zone, used in labels and selectors. Must follow Kubernetes naming restrictions: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
- name: zone-b
# -- nodeselector to restrict where pods of this zone can be placed. E.g.:
Expand All @@ -1093,6 +1125,14 @@ ingester:
# If set to "-", then use `storageClassName: ""`, which disables dynamic provisioning
# If undefined or set to null (the default), then fall back to the value of `ingester.persistentVolume.storageClass`.
storageClass: null
# -- noDownscale adds a label that can be used by the rollout-operator as documented in the rollout-operator repo: https://github.com/grafana/rollout-operator#webhooks
noDownscale: false
# -- downscaleLeader is an Annotation used by the rollout-operator
# If defined, downscaleLeader: <downscaleLeader>
# If undefined or set to null (the default), no Annotation is set
downscaleLeader: null
# -- prepareDownscale adds labels and annotations for the rollout-operator to prepare downscaling: https://github.com/grafana/rollout-operator#how-scaling-up-and-down-works
prepareDownscale: false
# -- Name of the zone, used in labels and selectors. Must follow Kubernetes naming restrictions: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
- name: zone-c
# -- nodeselector to restrict where pods of this zone can be placed. E.g.:
Expand All @@ -1106,6 +1146,14 @@ ingester:
# If set to "-", then use `storageClassName: ""`, which disables dynamic provisioning
# If undefined or set to null (the default), then fall back to the value of `ingester.persistentVolume.storageClass`.
storageClass: null
# -- noDownscale adds a label that can be used by the rollout-operator as documented in the rollout-operator repo: https://github.com/grafana/rollout-operator#webhooks
noDownscale: false
# -- downscaleLeader is an Annotation used by the rollout-operator
# If defined, downscaleLeader: <downscaleLeader>
# If undefined or set to null (the default), no Annotation is set
downscaleLeader: null
# -- prepareDownscale adds labels and annotations for the rollout-operator to prepare downscaling: https://github.com/grafana/rollout-operator#how-scaling-up-and-down-works
prepareDownscale: false

overrides_exporter:
enabled: true
Expand Down Expand Up @@ -2117,6 +2165,14 @@ store_gateway:
# If set to "-", then use `storageClassName: ""`, which disables dynamic provisioning
# If undefined or set to null (the default), then fall back to the value of `store_gateway.persistentVolume.storageClass`.
storageClass: null
# -- noDownscale adds a label that can be used by the rollout-operator as documented in the rollout-operator repo: https://github.com/grafana/rollout-operator#webhooks
noDownscale: false
# -- downscaleLeader is an Annotation used by the rollout-operator
# If defined, downscaleLeader: <downscaleLeader>
# If undefined or set to null (the default), no Annotation is set
downscaleLeader: null
# -- prepareDownscale adds labels and annotations for the rollout-operator to prepare downscaling: https://github.com/grafana/rollout-operator#how-scaling-up-and-down-works
prepareDownscale: false
# -- Name of the zone, used in labels and selectors. Must follow Kubernetes naming restrictions: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
- name: zone-b
# -- nodeselector to restrict where pods of this zone can be placed. E.g.:
Expand All @@ -2130,6 +2186,14 @@ store_gateway:
# If set to "-", then use `storageClassName: ""`, which disables dynamic provisioning
# If undefined or set to null (the default), then fall back to the value of `store_gateway.persistentVolume.storageClass`.
storageClass: null
# -- noDownscale adds a label that can be used by the rollout-operator as documented in the rollout-operator repo: https://github.com/grafana/rollout-operator#webhooks
noDownscale: false
# -- downscaleLeader is an Annotation used by the rollout-operator
# If defined, downscaleLeader: <downscaleLeader>
# If undefined or set to null (the default), no Annotation is set
downscaleLeader: null
# -- prepareDownscale adds labels and annotations for the rollout-operator to prepare downscaling: https://github.com/grafana/rollout-operator#how-scaling-up-and-down-works
prepareDownscale: false
# -- Name of the zone, used in labels and selectors. Must follow Kubernetes naming restrictions: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
- name: zone-c
# -- nodeselector to restrict where pods of this zone can be placed. E.g.:
Expand All @@ -2143,6 +2207,14 @@ store_gateway:
# If set to "-", then use `storageClassName: ""`, which disables dynamic provisioning
# If undefined or set to null (the default), then fall back to the value of `store_gateway.persistentVolume.storageClass`.
storageClass: null
# -- noDownscale adds a label that can be used by the rollout-operator as documented in the rollout-operator repo: https://github.com/grafana/rollout-operator#webhooks
noDownscale: false
# -- downscaleLeader is an Annotation used by the rollout-operator
# If defined, downscaleLeader: <downscaleLeader>
# If undefined or set to null (the default), no Annotation is set
downscaleLeader: null
# -- prepareDownscale adds labels and annotations for the rollout-operator to prepare downscaling: https://github.com/grafana/rollout-operator#how-scaling-up-and-down-works
prepareDownscale: false

compactor:
replicas: 1
Expand Down
Loading