Experimental: migrate frontend webserver to Caddy

[jayaddison]

The Caddy webserver features an in-built certificate authority, allowing it to generate certificates on-demand without us having to implement any other moving parts / logic.

Those certificates won't be trusted by most web browsers -- but, for users with public DNS domain names, the changes required to enable dynamic certificate provisioning via public ACME-enabled certificate services like Let's Encrypt and ZeroSSL should be minimal.

Some implementation notes:

• Writable storage is required for Caddy's certificate authority -- in this case, we're providing that as a volume
  • This could be made completely ephemeral - for public DNS situations that'd probably be absolutely fine -- but for local-only usage that'd be annoying (clients would have to re-trust the cert store every time the container was shut down and restarted)
• caddy autosaves its' configuration, which is redundant in this case (the configuration is read-only and static)
  • There doesn't appear to be a way to disable this at the moment. The relevant allowPersist variable appears hard-coded to true in v2.6.2 of Caddy (feature request filed as Configurable autosave behaviour from Caddyfile caddyserver/caddy#5209)
• caddy attempts to add its' local certificate authority into the trust root for the machine that it is hosted on. That's likely useful in cases where long-lived browser-to-local-server trust is desired (fewer in-browser warnings), but since this is a containerized service, all clients should be external -- so this behaviour is disabled here using the global skipInstallTrust option
• caddy is a bit fussy when it starts up to check whether the config file has been formatted correctly - that's worth checking before any potential future merge of this changeset

Overall: I'm not sure whether this is worth introducing - lots of factors to weigh up. Seems worth opening as a pull request for potential discussion though.

Follows on from discussion about TLS configuration in #183.

Resolves #188.

[jayaddison]

Todo: to make this configuration suitable for use with arbitarily-chosen domain names, this server address should be updated to expand from an environment variable.

That environment variable should have a default value of localhost specified in the Containerfile.

[theAkito]

Very good point.

[theAkito]

Overall: I'm not sure whether this is worth introducing - lots of factors to weigh up. Seems worth opening as a pull request for potential discussion though.

The mentioned features seem helpful for local setups.

Would the certificate generation be still optional, i.e. easy to disable, though?

It would become redundant, if there were already an external certificate issuer in place.

If it would be optional, the whole server setup would be more generic and applicable to more server environments.
This certificate feature set provided by Caddy in this case is pretty specific.

[jayaddison]

Todo: this certificate issuer configuration will require updates if/when this configuration is used in combination with public ACME CA servers.

[jayaddison]

Would the certificate generation be still optional, i.e. easy to disable, though?
If it would be optional, the whole server setup would be more generic and applicable to more server environments.

Yep, agreed. I'll spend some more time to think about how this could be made optional.

This certificate feature set provided by Caddy in this case is pretty specific.

Somewhat true, although ACME certification provisioning has a reasonable amount of standardization and deployment history (~2012 or so?) behind it.

Even so: one webserver implementation where it's conveniently baked-in isn't necessarily a reason to migrate -- especially for a non-core feature and if it's possible to implement with other tooling.

[theAkito]

Somewhat true, although ACME certification provisioning has a reasonable amount of standardization and deployment history (~2012 or so?) behind it.

Correct. I was mostly referring to the bigger picture of letting this server handle certificates in the first place. If there is an external issuer already set up, this Caddy based setup would not only be redundant, but probably also wouldn't work well, I would guess.

Even so: one webserver implementation where it's conveniently baked-in isn't necessarily a reason to migrate -- especially for a non-core feature and if it's possible to implement with other tooling.

Indeed.

Yep, agreed. I'll spend some more time to think about how this could be made optional.

As for the perspective of a user, it would be very Docker Compose compatible, if it were a single or two environment variables, which would be able to configure how it works. One for disable/enable and another one for the type of certificate or source of it.

As for the implementation, I would need to look into how exactly it works, to be able to give a proper suggestion.

[jayaddison]

Although I haven't determined the feasibility of this yet, one idea for how to provide optional support for certificates would be to use the import directive.

What I don't understand yet is whether it's possible to combine that with dynamic/conditional behaviour. Based on my understanding of use cases so far, I think the possible configurations could be:

• certificates.disabled: no certificate management (default)
• certificates.local: use the built-in caddy certificate authority
• certificates.public: use one of the public ACME certificate authorities to issue certificates
  • requires a fully-qualified domain name as a parameter to the build
  • requires a selection of an ACME issuer

In any of these cases, it should be possible for the deployment site to choose whether to mount an (empty, tmpfs, persistent) volume at /var/lib/caddy/. That said: I still think it would be good in the default use-case for that directory to be empty and not require any filesystem writes.

Aside

A weird hypothetical scenario I've been thinking about is public, ephemeral containers. So a container spins up and claims to be able to serve traffic from a public domain name -- and the ACME server will verify that. If there could be strong-enough checks (in all of the certificate, browser, and ACME issuer) that the content served is static and has integrity -- and perhaps if the issued certificate includes a hyperlink to and/or integrity hash of the source code -- then perhaps those ephemeral (and stateless) containers could all be trusted to serve that static content for that domain.