A proposal for a helm chart that supports a monolithic

deployment. The same chart is used for monolithic,
as was existing for the microservices deployment mode.

To do this a new parameter 'deploymentMode' has been
added to the helm chart. All charts are adapted to
deploy services when applicable according to the
deployment model.

A new template is being added that installs the
monolithic mimir container. All nginx endpoints are
updated to direct to the monolithic service if this
deployment mode has been chosen.

The name of the chart has not been updated from
'mimir-distributed' to 'mimir' yet. This allows for
an easier maintenance as long as this is not merged.
Ideally the rename happens when merging to avoid
further confusion.

I attached a monolithic.yaml yaml file that I have
used to do a monolithic setup.

In monolithic mode, publish not ready addresses and
use these for the frontend.

The read-write mode could be added in the future.

The alert manager is not included in the monolithic
deployment. If you want to run it, you can run it
in a seperate pod by enabling it.

The zone aware reploication functionality is supported
int he monolithic deployment with a similar confiuction
as required for ingester or store gateway.

I have also added the functionality to enable compression
for the monolithic chart. Ideally this would be embedded
into the complete chart instead. A common configuration is
probably the easiest, if you want grpc message compression,
I guess you want it everywhere. The common configuration
setting can then also be used in the monolithic approach.

operations/helm/charts/mimir-distributed/CHANGELOG.md

@@ -31,12 +31,12 @@ Entries should include a reference to the Pull Request that introduced the chang
 * [CHANGE] Reduce `-server.grpc-max-concurrent-streams` from 1000 to 500 for ingester and to 100 for all components. #5666
 * [ENHANCEMENT] Update the `rollout-operator` subchart to `0.9.1`. #6022 #6110
 * [BUGFIX] Let the unified gatway/nginx config listen on IPv6 as well. Followup to #5948. #6204
+* [FEATURE] Support monolithic deployment with the helm chart. #4832
 
 ## 5.1.0
 
 * [ENHANCEMENT] Update Mimir image to 2.10.0 and GEM image to v2.10.1. #6077
 * [ENHANCEMENT] Make compactor podManagementPolicy configurable. #5902
-* [ENHANCEMENT] Distributor: dynamically set `GOMAXPROCS` based on the CPU request. This should reduce distributor CPU utilization, assuming the CPU request is set to a value close to the actual utilization. #5588
 * [ENHANCEMENT] Querier: dynamically set `GOMAXPROCS` based on the CPU request. This should reduce noisy neighbour issues created by the querier, whose CPU utilization could eventually saturate the Kubernetes node if unbounded. #5646
 * [ENHANCEMENT] Sets the `appProtocol` value to `tcp` for the `gossip-ring-svc` service template. This allows memberlist to work with istio protocol selection. #5673
 * [ENHANCEMENT] Update the `rollout-operator` subchart to `0.8.0`. #5718
@@ -663,4 +663,4 @@ Entries should include a reference to the Pull Request that introduced the chang
 
 ## 0.1.0
 
-* [FEATURE] Initial commit, Mimir only, derived from `enterprise-metrics` chart. #1141
+* [FEATURE] Initial commit, Mimir only, derived from `enterprise-metrics` chart. #1141

operations/helm/charts/mimir-distributed/monolithic.yaml

@@ -0,0 +1,81 @@
+# This is an example setup for monolithic mimir deployment
+
+deploymentMode: monolithic
+
+fullnameOverride: mimir
+
+serviceAccount:
+  annotations:
+    "eks.amazonaws.com/role-arn": "arn:aws:iam::829850591703:role/multi-az-eks-multi-az-mimir"
+
+mimir:
+  structuredConfig:
+    limits:
+      # Delete from storage metrics data older than 62 days.
+      compactor_blocks_retention_period: 62d
+    common:
+      storage:
+        backend: s3
+        s3:
+          endpoint: s3.eu-west-1.amazonaws.com
+          region: eu-west-1
+    blocks_storage:
+      storage_prefix: blocks
+      s3:
+        bucket_name: multi-az-eks-multi-az-mimir
+    alertmanager_storage:
+      storage_prefix: alertmanager
+      s3:
+        bucket_name: multi-az-eks-multi-az-mimir
+    ruler_storage:
+      storage_prefix: ruler
+      s3:
+        bucket_name: multi-az-eks-multi-az-mimir
+
+monolithic:
+  replicas: 3
+  persistentVolume:
+    enabled: true
+    size: "8Gi"
+    storageClass: gp3-3000-125
+  priorityClassName: "optional-rescheduling-priority-multi-az"
+    # The zone aware replication doesn't yet make a lot of difference, since it is not used at query side yet. This is
+    # however planned in an upcoming release, and therefore it's best to have this configuration already enabled.
+  zoneAwareReplication:
+    enabled: true
+    topologyKey: "topology.kubernetes.io/zone"  # This generates default anti-affinity rules
+    zones:  # Zone list has to be fully redefined for modification. Update with you actual zones or skip to use logical zones only.
+    - name: zone-a
+      nodeSelector:
+        topology.kubernetes.io/zone: eu-west-1a
+    - name: zone-b
+      nodeSelector:
+        topology.kubernetes.io/zone: eu-west-1b
+    - name: zone-c
+      nodeSelector:
+        topology.kubernetes.io/zone: eu-west-1c
+  compression:
+    enabled: true
+
+nginx:
+  replicas: 1
+  priorityClassName: "mandatory-rescheduling-priority-multi-az"
+  ingress:
+    enabled: true
+    hosts:
+      - host: "mimir.multi-az.test.ngdata.com"
+        paths:
+          - path: /
+            pathType: Prefix
+    tls:
+    # empty, disabled.
+
+# Disable Minio, we use S3 storage instead
+minio:
+  enabled: false
+  replicas: 0
+
+# The Alert manager is currently not used. When enabling, it has to be configured for zone aware replication similar to
+# monolithic setup.
+alertmanager:
+  enabled: false

operations/helm/charts/mimir-distributed/templates/_helpers.tpl

@@ -397,6 +397,7 @@ Examples:
 */}}
 {{- define "mimir.componentSectionFromName" -}}
 {{- $componentsMap := dict
+  "monolithic" "monolithic"
   "admin-api" "admin_api"
   "admin-cache" "admin-cache"
   "alertmanager" "alertmanager"
@@ -442,6 +443,7 @@ mimir.vaultAgent.annotations takes 2 arguments
 */}}
 {{- define "mimir.vaultAgent.annotations" -}}
 {{- $vaultEnabledComponents := dict
+  "monolithic" true
   "admin-api" true
   "alertmanager" true
   "compactor" true
@@ -528,7 +530,7 @@ which allows us to keep generating everything for the default zone.
 {{- end -}}
 
 {{- $requestedReplicas := $componentSection.replicas -}}
-{{- if and (has .component (list "ingester" "alertmanager")) $componentSection.zoneAwareReplication.migration.enabled (not $componentSection.zoneAwareReplication.migration.writePath) -}}
+{{- if and (has .component (list "monolithic" "ingester" "alertmanager")) $componentSection.zoneAwareReplication.migration.enabled (not $componentSection.zoneAwareReplication.migration.writePath) -}}
 {{- $requestedReplicas = $componentSection.zoneAwareReplication.migration.replicas }}
 {{- end -}}
 {{- $replicaPerZone := div (add $requestedReplicas $numberOfZones -1) $numberOfZones -}}
@@ -640,4 +642,4 @@ mimir.parseCPU takes 1 argument
     {{- else -}}
         {{- $value_string }}
     {{- end -}}
-{{- end -}}
+{{- end -}}

operations/helm/charts/mimir-distributed/templates/alertmanager/alertmanager-config.yaml

@@ -1,3 +1,4 @@
+{{- if has .Values.deploymentMode (list "microservices" "monolithic")}}
 {{- if .Values.alertmanager.enabled -}}
 {{- if .Values.alertmanager.fallbackConfig -}}
 apiVersion: v1
@@ -14,3 +15,4 @@ data:
     {{- .Values.alertmanager.fallbackConfig | nindent 4 }}
 {{- end -}}
 {{- end -}}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/alertmanager/alertmanager-dep.yaml

@@ -1,3 +1,4 @@
+{{- if has .Values.deploymentMode (list "microservices" "monolithic")}}
 {{- if .Values.alertmanager.enabled -}}
 {{- if not .Values.alertmanager.statefulSet.enabled -}}
 apiVersion: apps/v1
@@ -145,3 +146,4 @@ spec:
         {{- end }}
 {{- end -}}
 {{- end -}}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/alertmanager/alertmanager-pdb.yaml

@@ -1,3 +1,5 @@
+{{- if has .Values.deploymentMode (list "microservices" "monolithic")}}
 {{- if .Values.alertmanager.enabled -}}
 {{- include "mimir.lib.podDisruptionBudget" (dict "ctx" $ "component" "alertmanager" "memberlist" true) }}
 {{- end -}}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/alertmanager/alertmanager-servmon.yaml

@@ -1,3 +1,5 @@
+{{- if has .Values.deploymentMode (list "microservices" "monolithic")}}
 {{- if .Values.alertmanager.enabled -}}
 {{- include "mimir.lib.serviceMonitor" (dict "ctx" $ "component" "alertmanager" "memberlist" true) }}
 {{- end -}}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/alertmanager/alertmanager-statefulset.yaml

@@ -1,3 +1,4 @@
+{{- if has .Values.deploymentMode (list "microservices" "monolithic")}}
 {{- if .Values.alertmanager.enabled -}}
 {{- if .Values.alertmanager.statefulSet.enabled -}}
 {{- $args := dict "ctx" . "component" "alertmanager" "memberlist" true -}}
@@ -197,3 +198,4 @@ spec:
 {{ end }}
 {{ end }}
 {{ end }}
+{{ end }}

operations/helm/charts/mimir-distributed/templates/alertmanager/alertmanager-svc-headless.yaml

@@ -1,3 +1,4 @@
+{{- if has .Values.deploymentMode (list "microservices" "monolithic")}}
 {{- if .Values.alertmanager.enabled -}}
 {{- $clusterPort := regexReplaceAll ".+[:]" (default "0.0.0.0:9094" (include "mimir.alertmanagerClusterBindAddress" .) ) "" -}}
 apiVersion: v1
@@ -32,3 +33,4 @@ spec:
   selector:
     {{- include "mimir.selectorLabels" (dict "ctx" . "component" "alertmanager" "memberlist" true) | nindent 4 }}
 {{- end -}}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/alertmanager/alertmanager-svc.yaml

@@ -1,3 +1,4 @@
+{{- if has .Values.deploymentMode (list "microservices" "monolithic")}}
 {{- if .Values.alertmanager.enabled -}}
 {{- $args := dict "ctx" . "component" "alertmanager" "memberlist" true -}}
 {{- $zonesMap := include "mimir.zoneAwareReplicationMap" $args | fromYaml -}}
@@ -37,4 +38,5 @@ spec:
 ---
 {{ end }}
 {{ end }}
+{{ end }}
 {{ end }}

operations/helm/charts/mimir-distributed/templates/chunks-cache/chunks-cache-pdb.yaml

@@ -1,3 +1,5 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 {{- if index .Values "chunks-cache" "enabled" }}
 {{- include "mimir.lib.podDisruptionBudget" (dict "ctx" $ "component" "chunks-cache") }}
 {{- end -}}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/chunks-cache/chunks-cache-servmon.yaml

@@ -1,3 +1,5 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 {{- if index .Values "chunks-cache" "enabled" }}
 {{- include "mimir.lib.serviceMonitor" (dict "ctx" $ "component" "chunks-cache") }}
 {{- end -}}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/chunks-cache/chunks-cache-statefulset.yaml

@@ -1 +1,3 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 {{- include "mimir.memcached.statefulSet" (dict "ctx" $ "component" "chunks-cache" ) }}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/chunks-cache/chunks-cache-svc-headless.yaml

@@ -1 +1,3 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 {{- include "mimir.memcached.service" (dict "ctx" $ "component" "chunks-cache" ) }}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/compactor/compactor-pdb.yaml

@@ -1 +1,3 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 {{- include "mimir.lib.podDisruptionBudget" (dict "ctx" $ "component" "compactor" "memberlist" true) }}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/compactor/compactor-servmon.yaml

@@ -1 +1,3 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 {{- include "mimir.lib.serviceMonitor" (dict "ctx" $ "component" "compactor" "memberlist" true) }}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/compactor/compactor-statefulset.yaml

@@ -1,3 +1,4 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 apiVersion: apps/v1
 kind: StatefulSet
 metadata:
@@ -156,3 +157,4 @@ spec:
             {{- with .Values.compactor.extraEnvFrom }}
               {{- toYaml . | nindent 12 }}
             {{- end }}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/compactor/compactor-svc.yaml

@@ -1,3 +1,4 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 apiVersion: v1
 kind: Service
 metadata:
@@ -23,3 +24,4 @@ spec:
       targetPort: grpc
   selector:
     {{- include "mimir.selectorLabels" (dict "ctx" . "component" "compactor" "memberlist" true) | nindent 4 }}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/distributor/distributor-dep.yaml

@@ -1,3 +1,4 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 apiVersion: apps/v1
 kind: Deployment
 metadata:
@@ -139,4 +140,4 @@ spec:
           emptyDir: {}
         - name: active-queries
           emptyDir: {}
-
+{{- end }}

operations/helm/charts/mimir-distributed/templates/distributor/distributor-pdb.yaml

@@ -1 +1,3 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 {{- include "mimir.lib.podDisruptionBudget" (dict "ctx" $ "component" "distributor" "memberlist" true) }}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/distributor/distributor-servmon.yaml

@@ -1 +1,3 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 {{- include "mimir.lib.serviceMonitor" (dict "ctx" $ "component" "distributor" "memberlist" true) }}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/distributor/distributor-svc-headless.yaml

@@ -1,3 +1,4 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 apiVersion: v1
 kind: Service
 metadata:
@@ -25,3 +26,4 @@ spec:
       targetPort: grpc
   selector:
     {{- include "mimir.selectorLabels" (dict "ctx" . "component" "distributor" "memberlist" true) | nindent 4 }}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/distributor/distributor-svc.yaml

@@ -1,3 +1,4 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 apiVersion: v1
 kind: Service
 metadata:
@@ -23,3 +24,4 @@ spec:
       targetPort: grpc
   selector:
     {{- include "mimir.selectorLabels" (dict "ctx" . "component" "distributor" "memberlist" true) | nindent 4 }}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/gateway/gateway-dep.yaml

@@ -1,3 +1,4 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 {{- if (eq (include "mimir.gateway.isEnabled" .) "true") }}
 {{- with .Values.gateway }}
 {{- $isGEMGateway := $.Values.enterprise.enabled -}}
@@ -169,3 +170,4 @@ spec:
         {{- end }}
 {{- end -}}
 {{- end -}}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/gateway/gateway-ingress.yaml

@@ -1,3 +1,4 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 {{- if and (eq (include "mimir.gateway.isEnabled" .) "true") .Values.gateway.ingress.enabled -}}
 apiVersion: networking.k8s.io/v1
 kind: Ingress
@@ -42,3 +43,4 @@ spec:
           {{- end }}
     {{- end }}
 {{- end -}}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/gateway/gateway-pdb.yaml

@@ -1,3 +1,5 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 {{- if eq (include "mimir.gateway.isEnabled" .) "true" -}}
 {{- include "mimir.lib.podDisruptionBudget" (dict "ctx" $ "component" "gateway") }}
 {{- end -}}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/gateway/gateway-route.yaml

@@ -1,3 +1,4 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 {{- if .Values.gateway.route.enabled }}
 kind: Route
 apiVersion: route.openshift.io/v1
@@ -23,3 +24,4 @@ spec:
     {{- toYaml . | nindent 4 }}
 {{- end }}
 {{- end }}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/gateway/gateway-servmon.yaml

@@ -1,3 +1,5 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 {{- if and (eq (include "mimir.gateway.isEnabled" .) "true") (not .Values.gateway.enabledNonEnterprise) -}}
 {{- include "mimir.lib.serviceMonitor" (dict "ctx" $ "component" "gateway") }}
 {{- end -}}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/gateway/gateway-svc.yaml

@@ -1,3 +1,4 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 {{- if eq (include "mimir.gateway.isEnabled" .) "true" -}}
 {{- with .Values.gateway -}}
 apiVersion: v1
@@ -35,3 +36,4 @@ spec:
     {{- include "mimir.selectorLabels" (dict "ctx" $ "component" "gateway") | nindent 4 }}
 {{- end -}}
 {{- end -}}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/gateway/gateway-v2-hpa.yaml

@@ -1,3 +1,4 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 {{- if .Values.gateway.autoscaling.enabled }}
 {{- if eq (include "mimir.hpa.version" .) "autoscaling/v2" }}
 apiVersion: autoscaling/v2
@@ -33,3 +34,4 @@ spec:
   {{- end }}
 {{- end -}}
 {{- end -}}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/gateway/gateway-v2beta1-hpa.yaml

@@ -1,3 +1,4 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 {{- if .Values.gateway.autoscaling.enabled }}
 {{- if eq (include "mimir.hpa.version" .) "autoscaling/v2beta1" }}
 apiVersion: autoscaling/v2beta1
@@ -29,3 +30,4 @@ spec:
   {{- end }}
 {{- end -}}
 {{- end -}}
+{{- end }}

operations/helm/charts/mimir-distributed/templates/gateway/nginx-configmap.yaml

@@ -1,3 +1,4 @@
+{{- if eq .Values.deploymentMode "microservices" }}
 {{- if and (not .Values.enterprise.enabled) .Values.gateway.enabledNonEnterprise -}}
 apiVersion: v1
 kind: ConfigMap
@@ -10,3 +11,4 @@ data:
   nginx.conf: |
     {{- tpl .Values.gateway.nginx.config.file . | nindent 4 }}
 {{- end -}}
+{{- end }}