Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: pgbouncer for postgresql #1607

Merged
merged 5 commits into from
Nov 28, 2024
Merged

Conversation

shcherbak
Copy link
Contributor

No description provided.

@shcherbak
Copy link
Contributor Author

shcherbak commented Nov 22, 2024

an example for feature request: #1604

I will cleanup this and make proper tests if accepted.

@patsevanton
Copy link
Contributor

@shcherbak i tested

helm install -n test --wait sentry . --values values.yaml --timeout=1000s

get pods

k get pod -n test
NAME                                                              READY   STATUS    RESTARTS      AGE
sentry-billing-metrics-consumer-7fdf59945f-rcccz                  1/1     Running   0             26m
sentry-clickhouse-0                                               1/1     Running   0             38m
sentry-cron-7c76c944bd-gsmsk                                      1/1     Running   2 (37m ago)   38m
sentry-generic-metrics-consumer-7d54df5775-btvgs                  1/1     Running   0             26m
sentry-ingest-consumer-attachments-8d948fc4c-vwrhx                1/1     Running   0             26m
sentry-ingest-consumer-events-68d56779f6-r2bz2                    1/1     Running   0             26m
sentry-ingest-consumer-transactions-6649b4ccdd-h6xjs              1/1     Running   0             26m
sentry-ingest-monitors-78c4f7965f-lxctv                           1/1     Running   0             26m
sentry-ingest-occurrences-5969495499-qtzq2                        1/1     Running   0             26m
sentry-ingest-replay-recordings-6c84bcf-p775z                     1/1     Running   0             26m
sentry-issue-occurrence-consumer-5db94d5658-866hq                 1/1     Running   0             25m
sentry-kafka-controller-0                                         1/1     Running   0             38m
sentry-kafka-controller-1                                         1/1     Running   0             38m
sentry-kafka-controller-2                                         1/1     Running   0             38m
sentry-metrics-consumer-5dccb9ff99-g5m6p                          1/1     Running   0             26m
sentry-nginx-78665d4559-hf5wx                                     1/1     Running   0             38m
sentry-post-process-forward-errors-549f7f79b-ls2sx                1/1     Running   0             26m
sentry-post-process-forward-issue-platform-cb64b9d79-7xc69        1/1     Running   0             26m
sentry-post-process-forward-transactions-84598f8955-8lj89         1/1     Running   0             26m
sentry-rabbitmq-0                                                 1/1     Running   0             38m
sentry-relay-67b8c8fdd7-r9lzr                                     1/1     Running   0             25m
sentry-sentry-postgresql-0                                        1/1     Running   0             38m
sentry-sentry-redis-master-0                                      1/1     Running   0             38m
sentry-sentry-redis-replicas-0                                    1/1     Running   0             38m
sentry-snuba-api-7c4477f94f-ghtmj                                 1/1     Running   0             38m
sentry-snuba-consumer-5f58b45676-9x552                            1/1     Running   0             26m
sentry-snuba-generic-metrics-counters-consumer-5958fdbdfb-bx9l9   1/1     Running   0             26m
sentry-snuba-generic-metrics-distributions-consumer-6b9f5bd6zfd   1/1     Running   0             26m
sentry-snuba-generic-metrics-sets-consumer-64dcbcf556-rfhg2       1/1     Running   0             26m
sentry-snuba-group-attributes-consumer-5499c88f78-k4ftf           1/1     Running   0             26m
sentry-snuba-metrics-consumer-85b94f67b5-sbxm9                    1/1     Running   0             26m
sentry-snuba-outcomes-billing-consumer-648bb8db8c-fxrkz           1/1     Running   0             26m
sentry-snuba-outcomes-consumer-6f87949797-sz2jq                   1/1     Running   0             25m
sentry-snuba-replacer-6d4c8d987-9qlsn                             1/1     Running   0             25m
sentry-snuba-replays-consumer-766d8f46f9-xn4t5                    1/1     Running   0             26m
sentry-snuba-spans-consumer-7d8c4c88bb-kdf9n                      1/1     Running   0             26m
sentry-snuba-subscription-consumer-events-8657877bf4-swcn6        1/1     Running   0             25m
sentry-snuba-subscription-consumer-metrics-5bb7875c8f-4tvtm       1/1     Running   0             25m
sentry-snuba-subscription-consumer-transactions-5c96cbf458b77p4   1/1     Running   0             25m
sentry-snuba-transactions-consumer-5bc869f55b-h7d9s               1/1     Running   0             26m
sentry-subscription-consumer-events-758dd79459-nzj6p              1/1     Running   0             26m
sentry-subscription-consumer-generic-metrics-b78474bb4-dtx2t      1/1     Running   0             26m
sentry-subscription-consumer-metrics-7b7f7b7dcb-2857d             1/1     Running   0             26m
sentry-subscription-consumer-transactions-7c99cf6c67-drslm        1/1     Running   0             26m
sentry-web-5d8cb8c964-t44nt                                       1/1     Running   1 (37m ago)   38m
sentry-worker-84fbbb4bf-5chmb                                     1/1     Running   2 (37m ago)   38m
sentry-zookeeper-clickhouse-0                                     1/1     Running   0             38m

other checks

k get pod -A | grep pgbouncer
empty
k get deploy -A | grep pgbouncer
empty

@shcherbak
Copy link
Contributor Author

shcherbak commented Nov 23, 2024 via email

@patsevanton
Copy link
Contributor

@shcherbak I have enabled the installation of pgbouncer

@shcherbak
Copy link
Contributor Author

there was a typo in enabler value name

@shcherbak
Copy link
Contributor Author

shcherbak commented Nov 23, 2024

there is work to be done with pgbouncer chart: additional envs, proper port configuration. maybe pgbouncer should be enabled by default. after enabling pgbouncer my sentry setup with HPA started using for 30% less resources. and i want to use this feature from main stream chart

@patsevanton
Copy link
Contributor

patsevanton commented Nov 24, 2024

values.yaml

pgbouncer:
  enabled: true

log pgbouncer

ERROR ==> POSTGRESQL_PASSWORD must be set

k get pod -n test

NAME                                READY   STATUS             RESTARTS       AGE
sentry-clickhouse-0                 1/1     Running            0              62m
sentry-cron-5bdc5f8dc9-k8ltp        1/1     Running            2 (61m ago)    62m
sentry-kafka-controller-0           1/1     Running            0              62m
sentry-kafka-controller-1           1/1     Running            0              62m
sentry-kafka-controller-2           1/1     Running            0              62m
sentry-nginx-597f7c6d6f-dtrmg       1/1     Running            0              62m
sentry-pgbouncer-664cf46864-g6z6q   0/1     CrashLoopBackOff   17 (39s ago)   62m
sentry-pgbouncer-664cf46864-jlbvf   0/1     CrashLoopBackOff   17 (28s ago)   62m
sentry-rabbitmq-0                   1/1     Running            0              62m
sentry-sentry-postgresql-0          1/1     Running            0              62m
sentry-sentry-redis-master-0        1/1     Running            0              62m
sentry-sentry-redis-replicas-0      1/1     Running            0              62m
sentry-snuba-api-cb6f47f66-jdpjt    1/1     Running            0              62m
sentry-web-68478c8d7f-kt78g         1/1     Running            1 (61m ago)    62m
sentry-worker-58788dd48-2dff8       1/1     Running            2 (61m ago)    62m
sentry-zookeeper-clickhouse-0       1/1     Running            0              62m

@shcherbak
Copy link
Contributor Author

fixed typo: POSTGRES_PASSWORD -> POSTGRESQL_PASSWORD

@shcherbak
Copy link
Contributor Author

this version of chart is deployed on my cluster with pgbouncer enabled

@patsevanton
Copy link
Contributor

patsevanton commented Nov 25, 2024

work

pgbouncer:
  enabled: true

k get pod -n test

NAME                                                              READY   STATUS    RESTARTS      AGE
sentry-billing-metrics-consumer-56dbcff4c5-7crqg                  1/1     Running   0             4m56s
sentry-clickhouse-0                                               1/1     Running   0             16m
sentry-cron-55ddb47768-r7zk9                                      1/1     Running   0             16m
sentry-generic-metrics-consumer-7795688cf5-r66gf                  1/1     Running   0             4m55s
sentry-ingest-consumer-attachments-784996987f-97cgz               1/1     Running   0             4m54s
sentry-ingest-consumer-events-6c844f8cbb-l7px5                    1/1     Running   0             4m53s
sentry-ingest-consumer-transactions-655d9c9cf8-7bc68              1/1     Running   0             4m52s
sentry-ingest-monitors-f4796dc87-vbjl9                            1/1     Running   0             4m52s
sentry-ingest-occurrences-768bd7c455-kq5qh                        1/1     Running   0             4m51s
sentry-ingest-replay-recordings-7697dbd89f-xsttq                  1/1     Running   0             4m50s
sentry-issue-occurrence-consumer-85b545496c-qchtq                 1/1     Running   0             4m36s
sentry-kafka-controller-0                                         1/1     Running   0             16m
sentry-kafka-controller-1                                         1/1     Running   0             16m
sentry-kafka-controller-2                                         1/1     Running   0             16m
sentry-metrics-consumer-695f55884c-2mg66                          1/1     Running   0             4m49s
sentry-nginx-597f7c6d6f-ncmxs                                     1/1     Running   0             16m
sentry-pgbouncer-68df79c88-6hxjw                                  1/1     Running   0             16m
sentry-pgbouncer-68df79c88-crcxq                                  1/1     Running   0             16m
sentry-post-process-forward-errors-76b7cc694-kcdtt                1/1     Running   0             4m48s
sentry-post-process-forward-issue-platform-7867cdb555-r6k9z       1/1     Running   0             4m48s
sentry-post-process-forward-transactions-77f49957cf-rrfbh         1/1     Running   0             4m47s
sentry-rabbitmq-0                                                 1/1     Running   0             16m
sentry-relay-7c66c955c4-jqh9s                                     1/1     Running   0             4m30s
sentry-sentry-postgresql-0                                        1/1     Running   0             16m
sentry-sentry-redis-master-0                                      1/1     Running   0             16m
sentry-sentry-redis-replicas-0                                    1/1     Running   0             16m
sentry-snuba-api-cb6f47f66-qpvhg                                  1/1     Running   0             16m
sentry-snuba-consumer-85c5f6b9c-jrv27                             1/1     Running   0             4m47s
sentry-snuba-generic-metrics-counters-consumer-64688445c-msb86    1/1     Running   0             4m43s
sentry-snuba-generic-metrics-distributions-consumer-77ccf5mtwkg   1/1     Running   0             4m42s
sentry-snuba-generic-metrics-sets-consumer-6754bf8b77-wlg55       1/1     Running   0             4m41s
sentry-snuba-group-attributes-consumer-7bf8f89c47-mhw8f           1/1     Running   0             4m41s
sentry-snuba-metrics-consumer-5bf57b5776-2p44w                    1/1     Running   0             4m40s
sentry-snuba-outcomes-billing-consumer-f96ddfcb6-2jhvf            1/1     Running   0             4m39s
sentry-snuba-outcomes-consumer-7fd9b7bfd4-qmjfm                   1/1     Running   0             4m35s
sentry-snuba-replacer-5d7d547476-d2qgc                            1/1     Running   0             4m34s
sentry-snuba-replays-consumer-7dbb575bfd-5vxdf                    1/1     Running   0             4m38s
sentry-snuba-spans-consumer-77cf4c4d87-fklp9                      1/1     Running   0             4m38s
sentry-snuba-subscription-consumer-events-796bfffc75-rqk8r        1/1     Running   0             4m33s
sentry-snuba-subscription-consumer-metrics-7d77dfdbd4-q7vp6       1/1     Running   0             4m32s
sentry-snuba-subscription-consumer-transactions-556496dc6c5fb7h   1/1     Running   0             4m31s
sentry-snuba-transactions-consumer-798bcdd8bf-2dcq7               1/1     Running   0             4m37s
sentry-subscription-consumer-events-78fb6c74bc-kf57v              1/1     Running   0             4m46s
sentry-subscription-consumer-generic-metrics-6df54d67cd-2vxvr     1/1     Running   0             4m45s
sentry-subscription-consumer-metrics-7cb966b598-snbm6             1/1     Running   0             4m44s
sentry-subscription-consumer-transactions-7455f59b99-gbd8p        1/1     Running   0             4m44s
sentry-web-7997689d8d-m8bcb                                       1/1     Running   2 (15m ago)   16m
sentry-worker-5595d79b77-rbzfw                                    1/1     Running   0             16m
sentry-zookeeper-clickhouse-0                                     1/1     Running   0             16m

@shcherbak
Copy link
Contributor Author

@Mokto @patsevanton Is it ready to merge? Or there more to do?

@adonskoy
Copy link
Contributor

You can deploy postgres and pgbouncer yourself and specify the connection via pgbouncer in externalPostgres values section. Why add this to the chart?

@shcherbak
Copy link
Contributor Author

@adonskoy because pgpouncer is the single correct option to use postgres with workloads under autoscaling control

Copy link
Contributor

@Mokto Mokto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@Mokto Mokto merged commit 07157bd into sentry-kubernetes:develop Nov 28, 2024
2 of 3 checks passed
@Mokto Mokto mentioned this pull request Nov 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants