docker-compose: fix the issue where celery broker was unable to push cfg #283
Conversation
|
|
Codecov Report
@@ Coverage Diff @@
## master #283 +/- ##
=======================================
Coverage 46.76% 46.76%
=======================================
Files 58 58
Lines 2677 2677
=======================================
Hits 1252 1252
Misses 1425 1425 Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Interesting about links being depricated. I do not actually use docker-compose directly (this is meant more as a demo) so I have not followed the specific details of this request.
I made comments on a few things that I'm curious about, but it may be Monday before I'm able to test this for myself (It is currently Friday afternoon in Japan right now)
| @@ -14,30 +14,28 @@ services: | |||
|
|
|||
| web: | |||
| extends: base | |||
| command: web -b 0.0.0.0:8000 | |||
| command: web -b 0.0.0.0:8000 --reload | |||
There was a problem hiding this comment.
In general --reload is really more for a development server so this should probably be removed.
There was a problem hiding this comment.
You're right, this will be removed.
| - prometheus | ||
| - mysql | ||
| - redis | ||
| command: worker -l info --queues localhost,celery,prometheus,alertmanager |
There was a problem hiding this comment.
Typically the worker only needs to listen to 2 queues at max.
celery is the default queue for almost everything (notifications, etc) and there should be a single named queue for each Prometheus worker that matches is host name. In the case of the docker container it might just be prometheus so perhaps this command should be --queues prometheus,celery
There was a problem hiding this comment.
Alright, it seems to work properly with:
command: worker -l info --queus prometheus,celery
| links: | ||
| - alertmanager | ||
| - blackbox | ||
| user: "root" |
There was a problem hiding this comment.
In general, I think it's probably best to run as a non-root user
There was a problem hiding this comment.
I agree with you that it should be runned as a non-root user.
The original issue is that prometheus couldn't create the required files&folders.
prometheus_1 | level=error ts=2020-06-05T13:30:24.569Z caller=query_logger.go:87 component=activeQueryTracker msg="Error opening query log file" file=/prometheus/queries.active err="open /prometheus/queries.active: permission denied"
prometheus_1 | panic: Unable to create mmap-ed active query log
This is due to the fact that by default the folders were created with the following user:groups
drwxr-xr-x 6 systemd-coredump root 4096 Jun 5 09:30 mysql_data
drwxr-xr-x 2 root root 4096 Jun 5 09:30 prom_dataIt looks like prometheus uses the nobody user https://github.com/prometheus/prometheus/blob/master/Dockerfile#L19
What do you think about using the user nobody as prometheus does?
The folder creation could then be done by an entrypoint script.
There was a problem hiding this comment.
Using the same prometheus user (nobody) should be fine.
| command: | ||
| - --config.file=/etc/prometheus/prometheus.yml | ||
| - --storage.tsdb.path=/prometheus | ||
| - --web.console.libraries=/usr/share/prometheus/console_libraries | ||
| - --web.console.templates=/usr/share/prometheus/consoles | ||
| - --web.enable-lifecycle |
There was a problem hiding this comment.
I think all the defaults should be ok, so I'm unsure the reason to add them here.
There was a problem hiding this comment.
The command option is overridden to add the --web.enable-lifecycle which is required to allow prometheus to reload its configuration.
Prometheus can reload its configuration at runtime. If the new configuration is not well-formed, the changes will not be applied. A configuration reload is triggered by sending a SIGHUP to the Prometheus process or sending a HTTP POST request to the /-/reload endpoint (when the --web.enable-lifecycle flag is enabled). This will also reload any configured rule files
There was a problem hiding this comment.
That's a good catch then. I suppose I need to test more carefully then because I never noticed this before (in production I do not deploy with docker so I've only tested the web part in docker)
| @@ -47,14 +45,14 @@ services: | |||
| blackbox: | |||
| image: prom/blackbox-exporter | |||
| network_mode: bridge | |||
| ports: | |||
| - "9115:9115" | |||
There was a problem hiding this comment.
blackbox_exporter is typically not accessed outside of the cluster, so normally the ports wouldn't need to be exported.
There was a problem hiding this comment.
I don't have a strong opinion on this point. I found it useful to have access to the page, mainly during testing, this can be removed.
|
Closing in favor of #402 |
Those changes should allow one to get started.
With those change, everything should work out of the box at the exception of the
admin/site/sitesthing that still needs to be configured.The
linkshave been removed because it's deprecated and is not required.https://docs.docker.com/compose/compose-file/compose-file-v2/#links