From c1450be5857350047417b578d4e7447bec07245b Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Mon, 4 Feb 2019 14:12:05 +0000 Subject: [PATCH 1/8] Fix wrapping in docker readme --- docker/README.md | 57 ++++++++++++++++++++++++++++++------------------ 1 file changed, 36 insertions(+), 21 deletions(-) diff --git a/docker/README.md b/docker/README.md index 3c00d1e9480d..08453b219f05 100644 --- a/docker/README.md +++ b/docker/README.md @@ -1,22 +1,22 @@ # Synapse Docker -This Docker image will run Synapse as a single process. It does not provide a database -server or a TURN server, you should run these separately. +This Docker image will run Synapse as a single process. It does not provide a +database server or a TURN server, you should run these separately. ## Run -We do not currently offer a `latest` image, as this has somewhat undefined semantics. -We instead release only tagged versions so upgrading between releases is entirely -within your control. +We do not currently offer a `latest` image, as this has somewhat undefined +semantics. We instead release only tagged versions so upgrading between +releases is entirely within your control. ### Using docker-compose (easier) -This image is designed to run either with an automatically generated configuration -file or with a custom configuration that requires manual editing. +This image is designed to run either with an automatically generated +configuration file or with a custom configuration that requires manual editing. An easy way to make use of this image is via docker-compose. See the -[contrib/docker](../contrib/docker) -section of the synapse project for examples. +[contrib/docker](../contrib/docker) section of the synapse project for +examples. ### Without Compose (harder) @@ -88,7 +88,8 @@ variables are available for configuration: * ``SYNAPSE_TURN_URIS``, set this variable to the coma-separated list of TURN uris to enable TURN for this homeserver. * ``SYNAPSE_TURN_SECRET``, set this to the TURN shared secret if required. -* ``SYNAPSE_MAX_UPLOAD_SIZE``, set this variable to change the max upload size [default `10M`]. +* ``SYNAPSE_MAX_UPLOAD_SIZE``, set this variable to change the max upload size + [default `10M`]. Shared secrets, that will be initialized to random values if not set: @@ -99,27 +100,41 @@ Shared secrets, that will be initialized to random values if not set: Database specific values (will use SQLite if not set): -* `POSTGRES_DB` - The database name for the synapse postgres database. [default: `synapse`] -* `POSTGRES_HOST` - The host of the postgres database if you wish to use postgresql instead of sqlite3. [default: `db` which is useful when using a container on the same docker network in a compose file where the postgres service is called `db`] -* `POSTGRES_PASSWORD` - The password for the synapse postgres database. **If this is set then postgres will be used instead of sqlite3.** [default: none] **NOTE**: You are highly encouraged to use postgresql! Please use the compose file to make it easier to deploy. -* `POSTGRES_USER` - The user for the synapse postgres database. [default: `matrix`] +* `POSTGRES_DB` - The database name for the synapse postgres + database. [default: `synapse`] +* `POSTGRES_HOST` - The host of the postgres database if you wish to use + postgresql instead of sqlite3. [default: `db` which is useful when using a + container on the same docker network in a compose file where the postgres + service is called `db`] +* `POSTGRES_PASSWORD` - The password for the synapse postgres database. **If + this is set then postgres will be used instead of sqlite3.** [default: none] + **NOTE**: You are highly encouraged to use postgresql! Please use the compose + file to make it easier to deploy. +* `POSTGRES_USER` - The user for the synapse postgres database. [default: + `matrix`] Mail server specific values (will not send emails if not set): * ``SYNAPSE_SMTP_HOST``, hostname to the mail server. -* ``SYNAPSE_SMTP_PORT``, TCP port for accessing the mail server [default ``25``]. -* ``SYNAPSE_SMTP_USER``, username for authenticating against the mail server if any. -* ``SYNAPSE_SMTP_PASSWORD``, password for authenticating against the mail server if any. +* ``SYNAPSE_SMTP_PORT``, TCP port for accessing the mail server [default + ``25``]. +* ``SYNAPSE_SMTP_USER``, username for authenticating against the mail server if + any. +* ``SYNAPSE_SMTP_PASSWORD``, password for authenticating against the mail + server if any. ## Build -Build the docker image with the `docker build` command from the root of the synapse repository. +Build the docker image with the `docker build` command from the root of the +synapse repository. ``` docker build -t docker.io/matrixdotorg/synapse . -f docker/Dockerfile ``` -The `-t` option sets the image tag. Official images are tagged `matrixdotorg/synapse:` where `` is the same as the release tag in the synapse git repository. +The `-t` option sets the image tag. Official images are tagged +`matrixdotorg/synapse:` where `` is the same as the release +tag in the synapse git repository. -You may have a local Python wheel cache available, in which case copy the relevant -packages in the ``cache/`` directory at the root of the project. +You may have a local Python wheel cache available, in which case copy the +relevant packages in the ``cache/`` directory at the root of the project. From 5c0dc58b22713161741683ac6a7d4fcd15215ca9 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Mon, 4 Feb 2019 14:28:52 +0000 Subject: [PATCH 2/8] Clean up some docs on the docker image --- docker/Dockerfile | 13 +++++++++++++ docker/README.md | 32 +++++++------------------------- 2 files changed, 20 insertions(+), 25 deletions(-) diff --git a/docker/Dockerfile b/docker/Dockerfile index 4b739e7d02a7..8a27b1355c15 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -1,3 +1,16 @@ +# Dockerfile to build the matrixdotorg/synapse docker images. +# +# To build the image, run `docker build` command from the root of the +# synapse repository: +# +# docker build -f docker/Dockerfile . +# +# There is an optional PYTHON_VERSION build argument which sets the +# version of python to build against: for example: +# +# docker build -f docker/Dockerfile --build-arg PYTHON_VERSION=3.6 . +# + ARG PYTHON_VERSION=2 ### diff --git a/docker/README.md b/docker/README.md index 08453b219f05..24381ef132fd 100644 --- a/docker/README.md +++ b/docker/README.md @@ -1,13 +1,12 @@ # Synapse Docker -This Docker image will run Synapse as a single process. It does not provide a -database server or a TURN server, you should run these separately. +This Docker image will run Synapse as a single process. By default it uses a +sqlite database; for production use you should connect it to a separate +postgres database. -## Run +The image also does *not* provide a TURN server. -We do not currently offer a `latest` image, as this has somewhat undefined -semantics. We instead release only tagged versions so upgrading between -releases is entirely within your control. +## Run ### Using docker-compose (easier) @@ -32,7 +31,7 @@ docker run \ -v ${DATA_PATH}:/data \ -e SYNAPSE_SERVER_NAME=my.matrix.host \ -e SYNAPSE_REPORT_STATS=yes \ - docker.io/matrixdotorg/synapse:latest + matrixdotorg/synapse:latest ``` ## Volumes @@ -71,7 +70,7 @@ then customize it manually. No other environment variable is required. Otherwise, a dynamic configuration file will be used. The following environment variables are available for configuration: -* ``SYNAPSE_SERVER_NAME`` (mandatory), the current server public hostname. +* ``SYNAPSE_SERVER_NAME`` (mandatory), the server public hostname. * ``SYNAPSE_REPORT_STATS``, (mandatory, ``yes`` or ``no``), enable anonymous statistics reporting back to the Matrix project which helps us to get funding. * ``SYNAPSE_NO_TLS``, set this variable to disable TLS in Synapse (use this if @@ -80,7 +79,6 @@ variables are available for configuration: the Synapse instance. * ``SYNAPSE_ALLOW_GUEST``, set this variable to allow guest joining this server. * ``SYNAPSE_EVENT_CACHE_SIZE``, the event cache size [default `10K`]. -* ``SYNAPSE_CACHE_FACTOR``, the cache factor [default `0.5`]. * ``SYNAPSE_RECAPTCHA_PUBLIC_KEY``, set this variable to the recaptcha public key in order to enable recaptcha upon registration. * ``SYNAPSE_RECAPTCHA_PRIVATE_KEY``, set this variable to the recaptcha private @@ -122,19 +120,3 @@ Mail server specific values (will not send emails if not set): any. * ``SYNAPSE_SMTP_PASSWORD``, password for authenticating against the mail server if any. - -## Build - -Build the docker image with the `docker build` command from the root of the -synapse repository. - -``` -docker build -t docker.io/matrixdotorg/synapse . -f docker/Dockerfile -``` - -The `-t` option sets the image tag. Official images are tagged -`matrixdotorg/synapse:` where `` is the same as the release -tag in the synapse git repository. - -You may have a local Python wheel cache available, in which case copy the -relevant packages in the ``cache/`` directory at the root of the project. From ab878a8d016bce709ad8a5a6caf0c8c6f455711e Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 5 Feb 2019 10:40:00 +0000 Subject: [PATCH 3/8] Enable ACME support in the docker image Also, a workaround for #4554. --- docker/Dockerfile | 2 +- docker/README.md | 22 ++++++++++++++++++++++ docker/conf/dummy.tls.crt | 17 +++++++++++++++++ docker/conf/homeserver.yaml | 18 ++++++++++++++++-- docker/start.py | 18 ++++++++++++++---- 5 files changed, 70 insertions(+), 7 deletions(-) create mode 100644 docker/conf/dummy.tls.crt diff --git a/docker/Dockerfile b/docker/Dockerfile index 8a27b1355c15..d94fedee4e3f 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -69,6 +69,6 @@ COPY ./docker/conf /conf VOLUME ["/data"] -EXPOSE 8008/tcp 8448/tcp +EXPOSE 8008/tcp 8009/tcp 8448/tcp ENTRYPOINT ["/start.py"] diff --git a/docker/README.md b/docker/README.md index 24381ef132fd..3b7b917fc40f 100644 --- a/docker/README.md +++ b/docker/README.md @@ -52,6 +52,27 @@ In order to setup an application service, simply create an ``appservices`` directory in the data volume and write the application service Yaml configuration file there. Multiple application services are supported. +## TLS certificates + +Synapse requires a valid TLS certificate. You can do one of: + + * Provide your own certificate and key (as + `${DATA_PATH}/${SYNAPSE_SERVER_NAME}.crt|key`, or elsewhere by providing + an entire config as `${SYNAPSE_CONFIG_PATH`). + + * Use a reverse proxy to terminate incoming TLS, and forward the plain http + traffic to port 8008 in the container. In this case you should set `-e + SYNAPSE_NO_TLS=1`. + + * Use the ACME (Let's Encrypt) support built into Synapse. This requires + `${SYNAPSE_SERVER_NAME}` port 80 to be forwarded to port 8009 in the + container, for example with `-p 80:8009`. To enable it in the docker + container, set `-e SYNAPSE_ACME=1`. + +If you don't do any of these, Synapse will fail to start with an error like: + + synapse.config._base.ConfigError: Error accessing file '/data/.tls.crt' (config for tls_certificate): No such file or directory + ## Environment Unless you specify a custom path for the configuration file, a very generic @@ -88,6 +109,7 @@ variables are available for configuration: * ``SYNAPSE_TURN_SECRET``, set this to the TURN shared secret if required. * ``SYNAPSE_MAX_UPLOAD_SIZE``, set this variable to change the max upload size [default `10M`]. +* ``SYNAPSE_ACME``: set this to enable the ACME certificate renewal support. Shared secrets, that will be initialized to random values if not set: diff --git a/docker/conf/dummy.tls.crt b/docker/conf/dummy.tls.crt new file mode 100644 index 000000000000..8e3b1a9aaa2f --- /dev/null +++ b/docker/conf/dummy.tls.crt @@ -0,0 +1,17 @@ +-----BEGIN CERTIFICATE----- +MIICnTCCAYUCAgPoMA0GCSqGSIb3DQEBCwUAMBQxEjAQBgNVBAMMCWxvY2FsaG9z +dDAeFw0xOTAxMTUwMDQxNTBaFw0yOTAxMTIwMDQxNTBaMBQxEjAQBgNVBAMMCWxv +Y2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMKqm81/8j5d +R1s7VZ8ueg12gJrPVCCAOkp0UnuC/ZlXhN0HTvnhQ+B0IlSgB4CcQZyf4jnA6o4M +rwSc7VX0MPE9x/idoA0g/0WoC6tsxugOrvbzCw8Tv+fnXglm6uVc7aFPfx69wU3q +lUHGD/8jtEoHxmCG177Pt2lHAfiVLBAyMQGtETzxt/yAfkloaybe316qoljgK5WK +cokdAt9G84EEqxNeEnx5FG3Vc100bAqJS4GvQlFgtF9KFEqZKEyB1yKBpPMDfPIS +V9hIV0gswSmYI8dpyBlGf5lPElY68ZGABmOQgr0RI5qHK/h28OpFPE0q3v4AMHgZ +I36wii4NrAUCAwEAATANBgkqhkiG9w0BAQsFAAOCAQEAfD8kcpZ+dn08xh1qtKtp +X+/YNZaOBIeVdlCzfoZKNblSFAFD/jCfObNJYvZMUQ8NX2UtEJp1lTA6m7ltSsdY +gpC2k1VD8iN+ooXklJmL0kxc7UUqho8I0l9vn35h+lhLF0ihT6XfZVi/lDHWl+4G +rG+v9oxvCSCWrNWLearSlFPtQQ8xPtOE0nLwfXtOI/H/2kOuC38ihaIWM4jjbWXK +E/ksgUfuDv0mFiwf1YdBF5/M3/qOowqzU8HgMJ3WoT/9Po5Ya1pWc+3BcxxytUDf +XdMu0tWHKX84tZxLcR1nZHzluyvFFM8xNtLi9xV0Z7WbfT76V0C/ulEOybGInYsv +nQ== +-----END CERTIFICATE----- diff --git a/docker/conf/homeserver.yaml b/docker/conf/homeserver.yaml index 529118d184d6..f07d5c1001f1 100644 --- a/docker/conf/homeserver.yaml +++ b/docker/conf/homeserver.yaml @@ -2,10 +2,24 @@ ## TLS ## +{% if SYNAPSE_NO_TLS %} +no_tls: True + +# workaround for https://github.com/matrix-org/synapse/issues/4554 +tls_certificate_path: "/conf/dummy.tls.crt" + +{% else %} + tls_certificate_path: "/data/{{ SYNAPSE_SERVER_NAME }}.tls.crt" tls_private_key_path: "/data/{{ SYNAPSE_SERVER_NAME }}.tls.key" -no_tls: {{ "True" if SYNAPSE_NO_TLS else "False" }} -tls_fingerprints: [] + +{% if SYNAPSE_ACME %} +acme: + enabled: true + port: 8009 +{% endif %} + +{% endif %} ## Server ## diff --git a/docker/start.py b/docker/start.py index 346df8c87f06..941d9996a826 100755 --- a/docker/start.py +++ b/docker/start.py @@ -47,9 +47,8 @@ def generate_secrets(environ, secrets): # In normal mode, generate missing keys if any, then run synapse else: - # Parse the configuration file if "SYNAPSE_CONFIG_PATH" in environ: - args += ["--config-path", environ["SYNAPSE_CONFIG_PATH"]] + config_path = environ["SYNAPSE_CONFIG_PATH"] else: check_arguments(environ, ("SYNAPSE_SERVER_NAME", "SYNAPSE_REPORT_STATS")) generate_secrets(environ, { @@ -58,10 +57,21 @@ def generate_secrets(environ, secrets): }) environ["SYNAPSE_APPSERVICES"] = glob.glob("/data/appservices/*.yaml") if not os.path.exists("/compiled"): os.mkdir("/compiled") - convert("/conf/homeserver.yaml", "/compiled/homeserver.yaml", environ) + + config_path = "/compiled/homeserver.yaml" + + convert("/conf/homeserver.yaml", config_path, environ) convert("/conf/log.config", "/compiled/log.config", environ) subprocess.check_output(["chown", "-R", ownership, "/data"]) - args += ["--config-path", "/compiled/homeserver.yaml"] + + + args += [ + "--config-path", config_path, + + # tell synapse to put any generated keys in /data rather than /compiled + "--keys-directory", "/data", + ] + # Generate missing keys and start synapse subprocess.check_output(args + ["--generate-keys"]) os.execv("/sbin/su-exec", ["su-exec", ownership] + args) From c56fcea0fab8319951a93cdd6dd10fa6f4055154 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 5 Feb 2019 11:19:13 +0000 Subject: [PATCH 4/8] changelog --- changelog.d/4566.feature | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelog.d/4566.feature diff --git a/changelog.d/4566.feature b/changelog.d/4566.feature new file mode 100644 index 000000000000..11fc07476e5a --- /dev/null +++ b/changelog.d/4566.feature @@ -0,0 +1 @@ +enable ACME support in the docker image From de34f582def9abe02237577b0b3b4da80a1993df Mon Sep 17 00:00:00 2001 From: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Date: Tue, 5 Feb 2019 11:45:46 +0000 Subject: [PATCH 5/8] docker/README.md: review suggestions Co-Authored-By: richvdh <1389908+richvdh@users.noreply.github.com> --- docker/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/README.md b/docker/README.md index 3b7b917fc40f..824242fe3d36 100644 --- a/docker/README.md +++ b/docker/README.md @@ -54,7 +54,7 @@ configuration file there. Multiple application services are supported. ## TLS certificates -Synapse requires a valid TLS certificate. You can do one of: +Synapse requires a valid TLS certificate. You can do one of the following: * Provide your own certificate and key (as `${DATA_PATH}/${SYNAPSE_SERVER_NAME}.crt|key`, or elsewhere by providing From 26bbb0ec8d3f43d54c897dcb36228f159510ccee Mon Sep 17 00:00:00 2001 From: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Date: Tue, 5 Feb 2019 11:46:01 +0000 Subject: [PATCH 6/8] docker/README.md: review suggestions Co-Authored-By: richvdh <1389908+richvdh@users.noreply.github.com> --- docker/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/README.md b/docker/README.md index 824242fe3d36..8dbdb642fe16 100644 --- a/docker/README.md +++ b/docker/README.md @@ -58,7 +58,7 @@ Synapse requires a valid TLS certificate. You can do one of the following: * Provide your own certificate and key (as `${DATA_PATH}/${SYNAPSE_SERVER_NAME}.crt|key`, or elsewhere by providing - an entire config as `${SYNAPSE_CONFIG_PATH`). + an entire config as `${SYNAPSE_CONFIG_PATH}`). * Use a reverse proxy to terminate incoming TLS, and forward the plain http traffic to port 8008 in the container. In this case you should set `-e From 6e89c15e0444f45e8715b2326599c71cb4a61a4e Mon Sep 17 00:00:00 2001 From: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Date: Tue, 5 Feb 2019 11:46:09 +0000 Subject: [PATCH 7/8] docker/README.md: review suggestions Co-Authored-By: richvdh <1389908+richvdh@users.noreply.github.com> --- docker/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/README.md b/docker/README.md index 8dbdb642fe16..723ba0110c61 100644 --- a/docker/README.md +++ b/docker/README.md @@ -69,7 +69,7 @@ Synapse requires a valid TLS certificate. You can do one of the following: container, for example with `-p 80:8009`. To enable it in the docker container, set `-e SYNAPSE_ACME=1`. -If you don't do any of these, Synapse will fail to start with an error like: +If you don't do any of these, Synapse will fail to start with an error similar to: synapse.config._base.ConfigError: Error accessing file '/data/.tls.crt' (config for tls_certificate): No such file or directory From f2e897c78a50e7b58a807776bf2b27a415b73ec0 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 5 Feb 2019 11:47:31 +0000 Subject: [PATCH 8/8] clarification --- docker/README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docker/README.md b/docker/README.md index 723ba0110c61..3faedf629f98 100644 --- a/docker/README.md +++ b/docker/README.md @@ -57,8 +57,9 @@ configuration file there. Multiple application services are supported. Synapse requires a valid TLS certificate. You can do one of the following: * Provide your own certificate and key (as - `${DATA_PATH}/${SYNAPSE_SERVER_NAME}.crt|key`, or elsewhere by providing - an entire config as `${SYNAPSE_CONFIG_PATH}`). + `${DATA_PATH}/${SYNAPSE_SERVER_NAME}.crt` and + `${DATA_PATH}/${SYNAPSE_SERVER_NAME}.key`, or elsewhere by providing an + entire config as `${SYNAPSE_CONFIG_PATH}`). * Use a reverse proxy to terminate incoming TLS, and forward the plain http traffic to port 8008 in the container. In this case you should set `-e