Enable certificate provisioning via letsencrypt

[jayaddison]

It'd be great to allow users to automatically provision a hostname and letsencrypt certificate by using environment variables and/or build arguments supplied to docker-compose build.

There are a few projects which provide similar or related functionality, and it may be possible to integrate with one of them, or select some functionality from them:

• https://github.com/wmnnd/nginx-certbot (requires an init script)
• https://github.com/nginx-proxy/docker-letsencrypt-nginx-proxy-companion (creates an nginx proxy)
• https://github.com/evertramos/docker-compose-letsencrypt-nginx-proxy-companion (creates an nginx proxy)
• https://github.com/linuxserver/docker-letsencrypt (creates an nginx proxy, has good documentation)

Edit: condense research-related comments into the pull request description

[tet3]

I think that the best way to do this is to run the container behind an nginx reverse proxy that handles the certs, rather than including the Lets Encrypt functionality in these containers. I'm currently using the 2nd solution in your list to run a few self-hosted web apps in Docker containers, behind an nginx reverse proxy. I started adjusting this repo's docker-compose.yml to work with it, but then ran into some other issues which I haven't been able to sort (I think having to do with the IPs of the docker network being different follwing a reboot of the VPS). Anyway, haven't gotten grocy working there yet. Then saw mention of https://github.com/jc21/nginx-proxy-manager yesterday in r/grocy, which looks way slicker than what I currently have, so may try to switch.

[jayaddison]

@tet3 Awesome. Let me know how you get on. I'm relatively new to writing docker/compose files, and to the grocy project, but glad to help debug and diagnose anything if you get stuck.

[abhinavrau]

@tet3 @jayaddison I am the author of cloud-grocy and it uses the first approach mentioned in the issue description. It also uses this project (great project!) as its base docker-compose file. I would be happy to assist in any way to get it letsencrypt integrated.

The crux to get lestencrypt working is DNS. I have some scripts in cloud-grocy project that may prove useful.

Let me know your thoughts.

-Abhi

[jayaddison]

@abhinavrau Thanks for getting in touch - cloud-grocy looks excellent. Out of interest, is the exposure and regular reload of the nginx conf.d volume in place to handle a specific use case?

As far as possible I think it's good to keep the use of bash scripts to a minimum; I'm reading up on the use of EFF's certbot/certbot image in your docker-compose.yml at the moment (and the associated shell scripts) and keen to see whether we can minimize scripting and prefer environmental configuration and parameterization.

[abhinavrau]

@jayaddison The regular reload of nginx is for handling the reload of SSL certificates in case of letsencrypt renewal. The certbot container is also checking for renewal every 12 hours.

The init-letsencrypt.sh script is used to generate the initial letsencrypt certs. It is a modified version of the same file from the nginx-certbot project.

I would be interested to see how not to use scripting but I don't see any ways around for the way certbot does cert generation. In order for the letsencrypt domain validation to happen, certbot command has to be run is some way. The goal of the script is to minimize user intervention as much as possible and get certificates installed. You could minimize scripting by using the certbot nginx plugin but that may involve user interaction and may not allow flexibility with the current grocy-docker nginx conf files.

Let me know your thoughts on this. I would be happy to work on any improvements to the code.

[jayaddison]

@abhinavrau That all sounds reasonable; and thanks for the link to the nginx-certbot project - I've had a read through some of the content there.

When you have time I'd encourage opening a pull request to bring your changes into grocy-docker, I'll be glad to review those.

I'd caution that I may be a little conservative and slow to accept changes because I'd like to understand the best practices and minimize risk to container users before introducing them.

In preparation (and so that I can review swiftly once you have changes ready) I'll do a bit more research into Docker / container image configuration in the meantime.

[abhinavrau]

@jayaddison @tet3 After taking a another look at docker-compose-letsencrypt-nginx-proxy-companion project you mentioned earlier, I played with and got it working with cloud-grocy. I think it is a much cleaner solution and it avoids a lot of scripting. Since it is a reverse proxy allowing virtual hosts, it allows for integrating project like barcode buddy.

I think it would make a good candidate to use for integrating to grocy-docker but integrating Letencrypt comes with a lot of caveats that needs to be considered:

• There are 3 additional docker containers that need to be started using docker-compose prior to starting grocy-docker containers. This will pose resource requirements for folks running on Raspberry Pi's etc.
• Given the resource requirements and some users not using it publicly, do we want to make LetsEncrypt integration an optional feature?
• Public IP and DNS registration is a requirement for enabling LetsEncrypt. To provide a better user experience, do we need to provide some scripts for waiting for DNS resolution before kicking off the LetsEncrypt registration?

Let me know your thoughts on what would make sense on moving this issue forward.

[jayaddison]

@abhinavrau That's great to hear! Out of interest, are the three additional containers required at application run-time, or are some/all of them only required during certificate provisioning (and renewal)?

Making LetsEncrypt optional is the way to go; let's make a (hopefully reasonable) assumption that a good number of folks may want to run grocy entirely within their own home network.

Combining that and your points regarding setup scripts - an approach I'd suggest would be that we identify the minimum number of configuration settings to get LetsEncrypt to work when the user already has a public IP and DNS record pointing to it.

From a brief look at docker-compose-letsencrypt-nginx-proxy-companion, LETSENCRYPT_HOST seems to be that single setting as an environment variable.

Is it somehow possible to allow the user to optionally provide that variable to docker-compose, and to perform the LetsEncrypt provisioning only when it is present?

I also wonder whether we could use the depends_on directive in docker-compose to bring the containers up in the correct order -- even when no certificate provisioning is performed (AFAIK, 'conditional' service/container logic is discouraged in compose files).

It looks like you've been committing code to your fork (edit to re-clarify: cloud-grocy repository, not fork!) so I'll take a look there soon. cc @tet3 just in case you missed the initial update here (and apologies for any duplicate notifications if you already did).

[jayaddison]

Linking some recent thoughts about automatic certificate provisioning at #183 (comment) (these are in the context of potentially using RFC 6762 private domain names by default)

[jayaddison]

There is a draft migration of the frontend webserver to caddy in pull request #192 -- a key promise of that is the potential for automatic certificate management for the container.

However: I haven't attempted to configure Let's Encrypt / ZeroSSL / any other ACME providers using it (I've used the built-in local ACME CA for testing, so far).

Ideally, regardless of whether the deployment is private/internal or using a public DNS name, it should be possible to (re)generate certificates on-demand as and when required (including when they expire).