x/crypto/acme: implement ACME RFC 8555

[x1ddos]

RFC 8555: https://tools.ietf.org/html/rfc8555

---

Let's Encrypt is planning to roll out the next version of ACME which they call API v2 endpoint:
https://letsencrypt.org/2017/06/14/acme-v2-api.html

It isn't clear yet what v2 will actually look like because the latest ACME draft v07 has not been standardized yet:

We refer to "V2" as whatever will be standardized by the IETF as an RFC.
The ACME specification hasn't become an RFC yet - draft-07 is the current document
being worked on. It's likely very close to what will be the RFC and the "V2" that we implement.

(from https://community.letsencrypt.org/t/request-whats-new-in-acme-v2-post/37042/6)

Here's the diff between the currently implemented protocol and the latest draft version:
https://tools.ietf.org/rfcdiff?url1=https://tools.ietf.org/id/draft-ietf-acme-acme-01.txt&url2=https://tools.ietf.org/id/draft-ietf-acme-acme-07.txt

The issue is to track that work. Just want to get a head start.
@bradfitz please, assign this to me.

[x1ddos]

A set of testing-focused packages implementing the upcoming Let's Encrypt endpoint: https://github.com/letsencrypt/pebble.
Could be useful.

[mdlayher]

Sorry for the noise, trying to edit the issue on mobile.

[mavimo]

@x1ddos @mdlayher a new draft is available https://datatracker.ietf.org/doc/draft-ietf-acme-acme/
I'm start looking on it and seems there are infos needed to start work on it, do you already implement some part of specs? I'm considering to start work on it...

[x1ddos]

@mavimo I haven't read it yet. Do you know if that's the closest to what v2 will be based on or will there be more iterations?

[mavimo]

@x1ddos tech spec seems stable (endpoint naming, response payload, ...) but spec are not closed. I suppose should possible starting a WIP implementation (using pebble) to test client implementation.

https://github.com/ietf-wg-acme/acme/
https://tools.ietf.org/wg/acme/agenda
https://datatracker.ietf.org/meeting/100/materials/slides-100-acme-acme-acme/

[kf6nux]

From Let's Encrypt,
"We are planning to have a public test API endpoint up by January 4, and we’ve set a date for the full launch: Tuesday, February 27."
https://letsencrypt.org/2017/12/07/looking-forward-to-2018.html

[x1ddos]

Would've been nice if they actually referred to some documentation, unless it's going to be exactly the latest IETF draft but I have some doubts.

[cpu]

@x1ddos any observed divergences from draft-09 would be welcome as issues on the Boulder repo.

[x1ddos]

@cpu yeah I saw the announcement not long ago. This is great! Will make sure to file issues if I notice anything.

[titanous]

From #23342:

I think there might actually be other ACME-compatible servers but even if there aren't any, I know for a fact there exist non-public implementations. I'd like to keep v1 working.

I'm now through the bulk of the implementation of v2, and there is no reasonable way to keep support for both the previous version and draft-09 in the same package due to substantial divergence in the ordering flow, object fields, and requests. A few helpers could be re-used, but that's about it. How intent are you on keeping support for "v1"? (which is really a Let's Encrypt specific version that doesn't match any previous draft).

[titanous]

@bradfitz Do you have any thoughts on if there's a good approach to pulling in pebble for integration tests? Worst case I can make a separate integration testing package and throw it on GitHub, but I'm wondering if there's a good way to get it running with every CL. I haven't come up with a good way to do it without it getting vendored in the x/crypto repo, which seems like a no-go.

[x1ddos]

I'm now through the bulk of the implementation of v2
@titanous where is it. Can I take a look?

[x1ddos]

@titanous also, it's a shame you're implementing something without at least consulting here. why duplicate work?

[titanous]

also, it's a shame you're implementing something without at least consulting here. why duplicate work?

I did consult, in the inadvertently duplicated #23342, but GitHub appears to have never sent me your reply. Anyway, this was just a weekend project, it's really not a problem for me if we have to change it or even throw the whole thing away. I'll push a CL in a moment.

[gopherbot]

Change https://golang.org/cl/86635 mentions this issue: acme: Implement IETF draft-09

[x1ddos]

Aside from too many unnecessary renamings, especially of exported types, golang.org/cl/86635 is a good start. But, I'd like to keep v1 working as is, without breaking. Besides, autocert.Manager would need to be changed too, in order to work with v2.

So, how about an acme.ClientV2 or acme/v2 package, or change the type of acme.Client to an interface and have multiple implementations, v1 and v2. It is also easy to automatically infer which version a URL points to, for instance by looking at the Directory response during discovery.

[titanous]

But, I'd like to keep v1 working as is, without breaking.

Given that v1 is never going to change (it’s a Let’s Encrypt-specific implementation and they are not going to develop the API after deprecation), wouldn’t it make more sense just to snapshot the current v1 implementation into a GitHub repo somewhere? Alternatively we can put v2 into a different package, but do you want to have autocert also support v1?

Besides, autocert.Manager would need to be changed too, in order to work with v2.

Yes, I didn’t want to do that until we’d finalized the v2 client API.

I really don’t think that an interface would properly encapsulate both versions under the same methods and types.

draft-09 is very close to being the final RFC, so I think it’s worth strongly considering making a clean API break to match the new spec (hence the various renames) instead of saddling ourselves with vestiges of what will be a forgotten draft API design (v1).

[x1ddos]

do you want to have autocert also support v1

Yes, because I'd like autocert to always work regardless, as in "go get -u .../autocert" and it works at any point in time.

The v2 implementation isn't stable yet anyway. In my experience, being close to final doesn't make it final and may never will. So, how about we create acme/v2 package and place everything v2-related in there, until it's polished or at least until works well with Let's Encrypt and then see how we can integrate it into the acme and acme/autocert packages.

[titanous]

Yes, because I'd like autocert to always work regardless, as in "go get -u .../autocert" and it works at any point in time.

To be clear, I was proposing that the cutover to v2 happen after LE turns on their production endpoint, so autocert will always just work.

So, how about we create acme/v2 package and place everything v2-related in there

Sounds good, I'll ping the CL when I have added the remaining tests.

[x1ddos]

or may even put it in acme/internal/v2. this is what we did with acme initially and it worked quite well. Am 08.01.2018 15:54 schrieb "Jonathan Rudenberg" <notifications@github.com>:

…

Yes, because I'd like autocert to always work regardless, as in "go get -u .../autocert" and it works at any point in time. To be clear, I was proposing that the cutover to v2 happen after LE turns on their production endpoint, so autocert will always just work. So, how about we create acme/v2 package and place everything v2-related in there Sounds good, I'll ping the CL when I have added the remaining tests. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#21081 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABjPfKzBeMlA5vS2pBYm86L4NogwA6Xks5tIiw8gaJpZM4Ocd5Y> .

[kf6nux]

I don't want to bikeshed on names, but last I heard, the IETF has refused calling this v2 as it's the first version they're ratifying.

I'm concerned that if the IETF ever releases v2 and Go already has a v2, it'll lead to more confusion.

Perhaps naming it ietfv1-acme is the right thing? @x1ddos @titanous

[kf6nux]

Let's Encrypt has launched their v2 service: https://community.letsencrypt.org/t/acme-v2-production-environment-wildcards/55578

[jmhodges]

It would be really nice to update that URL constant and finish off this work. I just got hit with the brownout that Let's Encrypt is putting in place and noticed a straight-forward update of crypto/acme didn't fix my problem.

(Boy, am I glad they did that brownout)

[cjellick]

@jmhodges did you identify a workaround? We're hit with the same problem

[jmhodges]

Oh, the solution is straightforward.

When creating your acme.Client, set its DirectoryURL field to "https://acme-v02.api.letsencrypt.org/directory" (for production use) or "https://acme-staging-v02.api.letsencrypt.org/directory" (for live testing).

I think you can't reuse accounts from the v01 API in the v02 one (because of the account's URI being a permanent identifier that includes the API URLs). Because of that, you'll need to make new accounts for the new API.

You can reuse accounts! No changes there needed.

[jmhodges]

(This, of course, means that folks have to write code to make acme.Client, and can't rely on autocert.Manager to do the right thing.)

[bradfitz]

https://golang.org/cl/199520 (mentioned above) is in review. Feel free to help test that out.

[gopherbot]

Change https://golang.org/cl/200638 mentions this issue: acme: update Directory URL to Let's Encrypt v2

[cohix]

Hello, we have customers using an application in production with autocert and letsencrypt, is there a stable workaround that we can use today to enable v2 support for them (I.e. is manually creating the acme.Client with the directory URL referenced above sufficient?), or are more changes needed before we can support this?

[x1ddos]

@cohix you can apply https://golang.org/cl/199520 manually or wait until it's merged.

[cohix]

@x1ddos is is reasonable to expect it will be merged in time for the v1 EOL next month? I'd prefer not trying to deploy a non-merged patch to my production application :P

[x1ddos]

@cohix it is now merged. see commit message for details: golang/crypto@0e8c3a9

[cohix]

@x1ddos great news. Tested it just now and all is well. Does it address https://community.letsencrypt.org/t/acme-v2-scheduled-deprecation-of-unauthenticated-resource-gets/74380 as well?

[bradfitz]

@cohix, https://github.com/golang/crypto/blob/4663e18/acme/http.go#L158

[cohix]

👏 hat tip to you all, thank you for averting a small crisis :)

[gopherbot]

Change https://golang.org/cl/203919 mentions this issue: acme/autocert: always pass AuthzURLs from AuthorizeOrder to deactivatePendingAuthz