Automatic TLS with LetsEncrypt

[emersion]

Just like Caddy.

[emersion]

https://github.com/mholt/certmagic

[foxcpp]

I think we can get it, but with some important caveats:

• Users must be able to change the CertMagic storage directory to avoid clashes with other certificate configurations (caddy, manual management).
• We need HTTP endpoint module to respond to challenges (this also leads to a slightly different design, see autoconfig support #67 issue for details)
• Don't agree for EULA automatically, require users to explicitly add "agreed" flag.

tls auto {
  filestorage ...
  agreed
}

[emersion]

Don't agree for EULA automatically, require users to explicitly add "agreed" flag.

It would be interesting to know what Caddy's stance about this is.

[foxcpp]

It would be interesting to know what Caddy's stance about this is.

It does it automatically.

[emersion]

Yeah, I meant: have people discussed whether it's okay/legal/desirable/etc?

[foxcpp]

Let's Encrypt Subscriber Agreement

Caddy may prompt to agree to Let's Encrypt Subscriber Agreement. This is configurable with ACME_AGREE environment variable. Set it to true to agree. ACME_AGREE=true.

Or I'm wrong...

[foxcpp]

So far the fundamental problem with automatic TLS is that ACME is built mostly for HTTP(S) servers and that is visible as 2 out of 3 challenges using HTTP(S) ports. maddy is not a HTTP(S) server and making it one (or in reverse, somebody proposed to make maddy a plugin for caddy) is going to push project complexity beyond what I believe is acceptable. In other words, the only challenge I see feasible to be implemented is a DNS-based one. This will also require per-provider plugins for a magnitude of proprietary interfaces.

I welcome work in this direction, somebody needs to wire certmagic library into maddy's TLS certificate loading code. That is how I can see it being used:

smtp tcp://0.0.0.0:25 {
  tls auto mx.example.org
  ...
}

This needs some refactoring in TLS certificate loading to only "load" it if any endpoints are initialized. This is important as maddyctl reads server configuration and that has a side effect of triggering TLS cert. path.

One might also consider implementing On-Demand TLS, but I have concerns about performance of DNS challenge and how that might affect user experience.

[emersion]

In other words, the only challenge I see feasible to be implemented is a DNS-based one

The DNS challenge is pretty lame, due to the lack of standardized DNS provider API.

Another alternative would be to grab certificates from a local certificate daemon, but this doesn't exist, yet.

[foxcpp]

The DNS challenge is pretty lame, due to the lack of standardized DNS provider API.

It kind of exists, RFC 2136. Unfortunately it is not widely supported :(

[foxcpp]

It might be possible to also support HTTP but it will not be magically working in configurations where there is also an HTTP server running. If it is running - there will be a need to configure maddy to use a separate port to serve challenge responses and have running HTTP server proxy requests. That might involve proxying based on Host header field if certbot/etc is also used to issue certificates for other domains (all bets are off if MX is using the same domain). But I think in this setup it might be also possible to just reuse the same certificate management solution but hard-link or copy certificates/keys to make them accessible by maddy. In fact, I do this for @hexanet.dev mail. And we expect people to have HTTP server running, especially if we ever get JMAP in (massive amount of work required, yeah...).

I see an JMAP-enabled maddy setup with webmail from a far future as follows:

• There is a HTTPS server running for mail.example.org
• Certificates are acquired using ACME (HTTP challenge) by certbot or functionality built into server itself (e.g. Caddy)
• JMAP API endpoint is reverse proxied to maddy
• Certificates are copied into maddy config directory so it can use them for TLS on IMAP/SMTP. [*] [**]
• The same HTTPS server is serving JMAP-based webmail frontend

[*] If reverse proxy supports it and we have XCLIENT extension for SMTP in maddy, it might be possible to have proxy do TLS termination and leave certificate management completely out of maddy.
[**] Copying of certs/keys can be replaced by proxying ACME challenge endpoint and having maddy get certificates for itself but I do not feel like it justifies the additional complexity.

And of course alternative to this is DNS-based challenge for a separate domain (e.g. mx.example.org) and maddy being completely independent of HTTP business (if there is JMAP involved - it just works as a usual webapp backend, accepting proxied requests). IMO this is the most clean solution even if it involves having dozens of per-provider components. Fortunately Caddy developers already wrote a large collection of such components.

[foxcpp]

Experimental implementation using CertMagic library and dns-01 challenge is in acme branch.
Getting into dev tree is blocked by caddyserver/certmagic#19.

[mholt]

Just a heads-up... I've completely rewritten CertMagic's underlying ACME library from scratch -- it's really quite good though, based on years of experience, and definitely the best one written in Go -- an improvement over what we were using before. It might change a feeeew little things with the exported CertMagic API but nothing major. One benefit is the logging is now (meaning, the commit I'm going to push later this week) configurable using zap.

PS. I do think a mail server would make a fine Caddy 2 module (could be an app module just like the HTTP server or TLS app which manages certificates). Once you get nestled into the JSON config it's pretty comfy: https://caddyserver.com/docs/extending-caddy

[foxcpp]

First case - ACME, http-01 challenge

tls.loader.acme local_tls {
    hostname $(hostname)
    challenge http-01
}

tls &local_tls # set default TLS config for all endpoints

# Configure reverse-proxy on port 80 to forward ACME challenges here
http tcp://127.0.0.1:8080 {
    serve acme_http_01 &local_tls # or just 'serve &local_tls'
}

Second case - ACME, dns-01 challenge

tls.loader.acme local_tls {
    hostname $(hostname)
    challenge dns-01
    # provider configuration is done via env-vars
}

tls &local_tls # set default TLS config for all endpoints

Third case - Manual certs with OCSP stapling

# Handled automatically if we revamp tls.loader.file to use Certmagic
tls file /etc/maddy/cert.pem /etc/maddy/cert.key

[emersion]

I'd advise against supporting the http-01 challenge, and instead for supporting the tls-alpn-01 challenge. Both are equivalent from a high-level point of view, but the latter doesn't rely on HTTP.

[foxcpp]

tls-alpn-01 can be definitely supported but it has an unfortunate limitation of requiring exclusive control over 443 port which maddy might not have.

[foxcpp]

Therefore I also propose supporting http-01 - if there is an web server running already and using both 80 and 443 ports then it can just proxy ACME endpoint to maddy so it can issue certificates for itself.

[emersion]

Hm. That is unfortunate.

FWIW, forwarding tls-alpn-01 to backend servers is something I'm planning to implement in tlstunnel. Since SMTP pretty much requires STARTTLS, so requires the mail server to handle TLS certificates, I'm wondering if it would make sense to have a very thin SMTP parser in tlstunnel that waits for the first SMTP command, upgrades to TLS if it's STARTTLS, then forwards to the email server.

[foxcpp]

Handling TLS termination in a SMTP-aware way is definitely making it possible to keep all TLS management in tlstunnel but it would require more than you described:

1. ESMTP capabilities exposed by backend server should be forwarded back to the client in response to EHLO
2. Client might proceed to issue MAIL FROM - in this case tlstunnel would have to act as a plain text proxy
3. Mail server needs knowledge of the client IP and whether TLS was negotiated (see 2), while former can be implemented using HAProxy's PROXY protocol (Support HAProxy's PROXY protocol for SMTP and IMAP #369), I am not sure what to do for the latter.

[emersion]

Yep, these are good points.

Mail server needs knowledge of the client IP and whether TLS was negotiated (see 2), while former can be implemented using HAProxy's PROXY protocol (#369), I am not sure what to do for the latter.

The PROXY protocol allows indicating whether TLS is in use by the frontend client via PP2_TYPE_SSL.

[foxcpp]

The PROXY protocol allows indicating whether TLS is in use by the frontend client via PP2_TYPE_SSL.

Oh, that's nice. Well, other than being cumbersome there is no problem.

To be honest, I really like (possibly opinionated) approach of replacing all certificate management solutions with tlstunnel and would gladly work on SMTP-aware mode support in tlstunnel if you believe this is worth it.

[emersion]

Yeah, it's quite an opinionated approach, hopefully not completely incompatible with other approaches so in the end users can choose. There are downsides to the tlstunnel approach too, for instance domain names need to be configured in both tlstunnel and the backend server (see this issue for brainstorming for fixing this).

[foxcpp]

At least for SMTP this is easily solvable: SMTP server identifies itself in the initial greeting.

[foxcpp]

I am working on implementing tls.loader.acme with support for dns-01 challenge using the most recent CertMagic library version. Since CertMagic moved from using go-acme/lego to acmez, support for dns providers will be more limited (18 providers) and does not include RFC 2136. :(

[foxcpp]

tls.loader.acme implementation is available on the dev branch for testing. See maddy-tls(5) for details.