Feat/sentry ingest consumer

README.md

@@ -14,6 +14,18 @@ Big thanks to the maintainers of the [deprecated chart](https://github.com/helm/
 
 For now the full list of values is not documented but you can get inspired by the values.yaml specific to each directory.
 
+
+## Upgrading from 4.x.x version of this Chart to 5.0.0
+
+As Relay is now part of this chart your need to make sure you enable either Nginx or the Ingress. Please read the next paragraph for more informations.
+
+If you are using an ingress gateway (like istio), you have to change your inbound path from sentry-web to nginx.
+
+## NGINX and/or Ingress
+
+By default, NGINX is enabled to allow sending the incoming requests to [Sentry Relay](https://getsentry.github.io/relay/) or the Django backend depending on the path. When Sentry is meant to be exposed outside of the Kubernetes cluster, it is recommended to disable NGINX and let the Ingress do the same. It's recommended to go with the go to Ingress Controller, [NGINX Ingress](https://kubernetes.github.io/ingress-nginx/) but others should work as well.
+
+
 ## Upgrading from 3.1.0 version of this Chart to 4.0.0
 
 Following Helm Chart best practices the new version introduces some breaking changes, all configuration for external

sentry/Chart.lock

@@ -14,5 +14,8 @@ dependencies:
 - name: postgresql
   repository: https://kubernetes-charts.storage.googleapis.com/
   version: 8.2.1
-digest: sha256:b0c03af78854426fbb32803b55427b4c6aadb98a753f1163f8ed906ade6e69d9
-generated: "2020-07-23T08:54:02.125539557+02:00"
+- name: nginx
+  repository: https://charts.bitnami.com/bitnami
+  version: 6.0.5
+digest: sha256:95a04a0fd6722f2f1b2d1750355060369a7d8e1cee6fd0e26974a4c402aff998
+generated: "2020-08-10T10:58:35.648936347+02:00"

sentry/Chart.yaml

@@ -2,8 +2,8 @@ apiVersion: v2
 name: sentry
 description: A Helm chart for Kubernetes
 type: application
-version: 4.8.1
-appVersion: 20.7.0
+version: 5.0.0
+appVersion: 20.7.2
 dependencies:
   - name: redis
     repository: https://charts.bitnami.com/bitnami
@@ -26,3 +26,7 @@ dependencies:
     repository: https://kubernetes-charts.storage.googleapis.com/
     version: 8.2.1
     condition: postgresql.enabled
+  - name: nginx
+    repository: https://charts.bitnami.com/bitnami
+    version: 6.0.5
+    condition: nginx.enabled

sentry/README.md

@@ -7,6 +7,7 @@ From the previous upgrade, make sure to get the following from your previous ins
 Both should be in the `secrets` of your original 9.0 release. Make a note of both of these values.
 
 #### Upgrade Steps
+
 Due to an issue where transferring from Helm 2 to 3. Statefulsets that use the following: `heritage: {{ .Release.Service }}` in the metadata field will error out with a `Forbidden` error during the upgrade. The only workaround is to delete the existing statefulsets (Don't worry, PVC will be retained):
 
 > kubectl delete --all sts -n <Sentry Namespace>
@@ -33,6 +34,8 @@ Parameter                          | Description
 `user.create` | if `true`, creates a default admin user defined from `email` and `password` | `true`
 `user.email` | Admin user email | `admin@sentry.local`
 `user.password` | Admin user password| `aaaa`
+`ingress.enabled` | Enabling Ingress | `false`
+`nginx.enabled` | Enabling NGINX | `true`
 `metrics.enabled`| if `true`, enable Prometheus metrics | `false`
 `metrics.image.repository`         | Metrics exporter image repository                                                                          | `prom/statsd-exporter`
 `metrics.image.tag`                | Metrics exporter image tag                                                                                 | `v0.10.5`
@@ -57,6 +60,10 @@ Parameter                          | Description
 `metrics.serviceMonitor.scrapeInterval` | interval between Prometheus scraping | `30s`
 `system.secretKey` | secret key for the session cookie ([documentation](https://develop.sentry.dev/config/#general)) | `nil`
 
+## NGINX and/or Ingress
+
+By default, NGINX is enabled to allow sending the incoming requests to [Sentry Relay](https://getsentry.github.io/relay/) or the Django backend depending on the path. When Sentry is meant to be exposed outside of the Kubernetes cluster, it is recommended to disable NGINX and let the Ingress do the same. It's recommended to go with the go to Ingress Controller, [NGINX Ingress](https://kubernetes.github.io/ingress-nginx/) but others should work as well.
+
 ## Sentry secret key
 
 For your security, the [`system.secret-key`](https://develop.sentry.dev/config/#general) is generated for you on the first installation. Another one will be regenerated on each upgrade invalidating all the current sessions unless it's been provided. The value is stored in the `sentry-sentry` configmap.

sentry/templates/_helper.tpl

@@ -1,14 +1,17 @@
+{{/* vim: set filetype=mustache: */}}
+
 {{- define "sentry.prefix" -}}
     {{- if .Values.prefix -}}
         {{.Values.prefix}}-
     {{- else -}}
     {{- end -}}
 {{- end -}}
 
+{{- define "nginx.port" -}}{{ default "8080" .Values.nginx.containerPort }}{{- end -}}
+{{- define "relay.port" -}}3000{{- end -}}
 {{- define "sentry.port" -}}9000{{- end -}}
 {{- define "snuba.port" -}}1218{{- end -}}
 
-{{/* vim: set filetype=mustache: */}}
 {{/*
 Expand the name of the chart.
 */}}

sentry/templates/configmap-nginx.yaml

@@ -0,0 +1,31 @@
+{{- if .Values.nginx.enabled }}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: {{ template "sentry.fullname" . }}-nginx
+data:
+  server-block.conf: |
+    upstream relay {
+      server {{ template "sentry.fullname" . }}-relay:{{ template "relay.port" }};
+    }
+
+    upstream sentry {
+      server {{ template "sentry.fullname" . }}-web:{{ template "sentry.port" }};
+    }
+
+    server {
+      listen {{ template "nginx.port" }};
+
+      location /api/store/ {
+        proxy_pass http://relay;
+      }
+
+      location ~ ^/api/[1-9]\d*/ {
+        proxy_pass http://relay;
+      }
+
+      location / {
+        proxy_pass http://sentry;
+      }
+    }
+{{- end }}

sentry/templates/configmap-relay.yaml

@@ -0,0 +1,36 @@
+{{- $redisHost := include "sentry.redis.host" . -}}
+{{- $redisPort := include "sentry.redis.port" . -}}
+{{- $redisPass := include "sentry.redis.password" . -}}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: {{ template "sentry.fullname" . }}-relay
+  labels:
+    app: sentry
+    chart: "{{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}"
+    release: "{{ .Release.Name }}"
+    heritage: "{{ .Release.Service }}"
+data:
+  config.yml: |-
+    relay:
+      mode: managed
+      upstream: "http://{{ template "sentry.fullname" . }}-web:{{ template "sentry.port" }}/"
+      host: 0.0.0.0
+      port: {{ template "relay.port" }}
+
+    processing:
+      enabled: true
+
+      kafka_config:
+        - name: "bootstrap.servers"
+          value: {{ printf "%s:%s" (include "sentry.kafka.host" .) (include "sentry.kafka.port" .) }}
+        - name: "message.max.bytes"
+          value: 50000000  # 50MB or bust
+
+      {{- if $redisPass }}
+      redis: "redis://:{{ $redisPass }}@{{ $redisHost }}:{{ $redisPort }}"
+      {{- else }}
+      redis: "redis://{{ $redisHost }}:{{ $redisPort }}"
+      {{- end }}
+
+{{ .Values.config.relay | indent 4 }}

sentry/templates/configmap-sentry.yaml

@@ -295,6 +295,7 @@ data:
                 "organizations:performance-view",
                 "organizations:repos",
                 "organizations:require-2fa",
+                "organizations:relay",
                 "organizations:sentry10",
                 "organizations:sso-basic",
                 "organizations:sso-rippling",
@@ -335,6 +336,12 @@ data:
     # BITBUCKET_CONSUMER_KEY = 'YOUR_BITBUCKET_CONSUMER_KEY'
     # BITBUCKET_CONSUMER_SECRET = 'YOUR_BITBUCKET_CONSUMER_SECRET'
 
+    #########
+    # Relay #
+    #########
+    SENTRY_RELAY_WHITELIST_PK = []
+    SENTRY_RELAY_OPEN_REGISTRATION = True
+
 {{- if .Values.metrics.enabled }}
     SENTRY_METRICS_BACKEND = 'sentry.metrics.statsd.StatsdMetricsBackend'
     SENTRY_METRICS_OPTIONS = {

sentry/templates/deployment-relay.yaml

@@ -0,0 +1,117 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: {{ template "sentry.fullname" . }}-relay
+  labels:
+    app: {{ template "sentry.fullname" . }}
+    chart: "{{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}"
+    release: "{{ .Release.Name }}"
+    heritage: "{{ .Release.Service }}"
+spec:
+  selector:
+    matchLabels:
+      app: {{ template "sentry.fullname" . }}
+      release: "{{ .Release.Name }}"
+      role: relay
+{{- if not .Values.relay.autoscaling.enabled }}
+  replicas: {{ .Values.relay.replicas }}
+{{- end }}
+  template:
+    metadata:
+      annotations:
+        checksum/relay: {{ .Values.config.relay | sha256sum }}
+        checksum/config.yaml: {{ include (print $.Template.BasePath "/configmap-relay.yaml") . | sha256sum }}
+        {{- if .Values.relay.annotations }}
+{{ toYaml .Values.relay.annotations | indent 8 }}
+        {{- end }}
+      labels:
+        app: {{ template "sentry.fullname" . }}
+        release: "{{ .Release.Name }}"
+        role: relay
+        {{- if .Values.relay.podLabels }}
+{{ toYaml .Values.relay.podLabels | indent 8 }}
+        {{- end }}
+    spec:
+      affinity:
+      {{- if .Values.relay.affinity }}
+{{ toYaml .Values.relay.affinity | indent 8 }}
+      {{- end }}
+      {{- if .Values.relay.nodeSelector }}
+      nodeSelector:
+{{ toYaml .Values.relay.nodeSelector | indent 8 }}
+      {{- end }}
+      {{- if .Values.relay.tolerations }}
+      tolerations:
+{{ toYaml .Values.relay.tolerations | indent 8 }}
+      {{- end }}
+      {{- if .Values.images.relay.imagePullSecrets }}
+      imagePullSecrets:
+{{ toYaml .Values.images.relay.imagePullSecrets | indent 8 }}
+      {{- end }}
+      initContainers:
+        - name: {{ .Chart.Name }}-relay-init
+          image: {{ .Values.images.relay.repository }}:{{ .Values.images.relay.tag }}
+          args:
+            - "credentials"
+            - "generate"
+          env:
+            - name: RELAY_PORT
+              value: '{{ template "relay.port" }}'
+          volumeMounts:
+            - name: credentials
+              mountPath: /work/.relay
+            - name: config
+              mountPath: /work/.relay/config.yml
+              subPath: config.yml
+              readOnly: true
+      containers:
+      - name: {{ .Chart.Name }}-relay
+        image: "{{ .Values.images.relay.repository }}:{{ .Values.images.relay.tag }}"
+        imagePullPolicy: {{ .Values.images.relay.pullPolicy }}
+        ports:
+        - containerPort: {{ template "relay.port" }}
+        env:
+        - name: RELAY_PORT
+          value: '{{ template "relay.port" }}'
+{{- if .Values.relay.env }}
+{{ toYaml .Values.relay.env | indent 8 }}
+{{- end }}
+        volumeMounts:
+          - name: credentials
+            mountPath: /work/.relay
+          - name: config
+            mountPath: /work/.relay/config.yml
+            subPath: config.yml
+            readOnly: true
+        livenessProbe:
+          failureThreshold: 5
+          httpGet:
+            path: /api/relay/healthcheck/ready/
+            port: {{ template "relay.port" }}
+            scheme: HTTP
+          initialDelaySeconds: {{ .Values.relay.probeInitialDelaySeconds }}
+          periodSeconds: 10
+          successThreshold: 1
+          timeoutSeconds: 2
+        readinessProbe:
+          failureThreshold: 10
+          httpGet:
+            path: /api/relay/healthcheck/ready/
+            port: {{ template "relay.port" }}
+            scheme: HTTP
+          initialDelaySeconds: {{ .Values.relay.probeInitialDelaySeconds }}
+          periodSeconds: 10
+          successThreshold: 1
+          timeoutSeconds: 2
+        resources:
+{{ toYaml .Values.relay.resources | indent 12 }}
+      volumes:
+      - name: config
+        configMap:
+          name: {{ template "sentry.fullname" . }}-relay
+          defaultMode: 0644
+      - name: credentials
+        emptyDir: {}
+      {{- if .Values.relay.priorityClassName }}
+      priorityClassName: "{{ .Values.relay.priorityClassName }}"
+      {{- end }}