Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/crypto/acme: implement ACME RFC 8555 #21081

Closed
x1ddos opened this issue Jul 19, 2017 · 83 comments
Closed

x/crypto/acme: implement ACME RFC 8555 #21081

x1ddos opened this issue Jul 19, 2017 · 83 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@x1ddos
Copy link

x1ddos commented Jul 19, 2017

RFC 8555: https://tools.ietf.org/html/rfc8555


Let's Encrypt is planning to roll out the next version of ACME which they call API v2 endpoint:
https://letsencrypt.org/2017/06/14/acme-v2-api.html

It isn't clear yet what v2 will actually look like because the latest ACME draft v07 has not been standardized yet:

We refer to "V2" as whatever will be standardized by the IETF as an RFC.
The ACME specification hasn't become an RFC yet - draft-07 is the current document
being worked on. It's likely very close to what will be the RFC and the "V2" that we implement.

(from https://community.letsencrypt.org/t/request-whats-new-in-acme-v2-post/37042/6)

Here's the diff between the currently implemented protocol and the latest draft version:
https://tools.ietf.org/rfcdiff?url1=https://tools.ietf.org/id/draft-ietf-acme-acme-01.txt&url2=https://tools.ietf.org/id/draft-ietf-acme-acme-07.txt

The issue is to track that work. Just want to get a head start.
@bradfitz please, assign this to me.

@gopherbot gopherbot added this to the Unreleased milestone Jul 19, 2017
@x1ddos
Copy link
Author

x1ddos commented Jul 19, 2017

A set of testing-focused packages implementing the upcoming Let's Encrypt endpoint: https://github.com/letsencrypt/pebble.
Could be useful.

@mdlayher mdlayher assigned mdlayher and x1ddos and unassigned mdlayher Jul 19, 2017
@mdlayher
Copy link
Member

Sorry for the noise, trying to edit the issue on mobile.

@mavimo
Copy link

mavimo commented Nov 18, 2017

@x1ddos @mdlayher a new draft is available https://datatracker.ietf.org/doc/draft-ietf-acme-acme/
I'm start looking on it and seems there are infos needed to start work on it, do you already implement some part of specs? I'm considering to start work on it...

@x1ddos
Copy link
Author

x1ddos commented Nov 23, 2017

@mavimo I haven't read it yet. Do you know if that's the closest to what v2 will be based on or will there be more iterations?

@mavimo
Copy link

mavimo commented Nov 26, 2017

@x1ddos tech spec seems stable (endpoint naming, response payload, ...) but spec are not closed. I suppose should possible starting a WIP implementation (using pebble) to test client implementation.

https://github.com/ietf-wg-acme/acme/
https://tools.ietf.org/wg/acme/agenda
https://datatracker.ietf.org/meeting/100/materials/slides-100-acme-acme-acme/

@kf6nux
Copy link

kf6nux commented Dec 21, 2017

From Let's Encrypt,
"We are planning to have a public test API endpoint up by January 4, and we’ve set a date for the full launch: Tuesday, February 27."
https://letsencrypt.org/2017/12/07/looking-forward-to-2018.html

@x1ddos
Copy link
Author

x1ddos commented Dec 22, 2017

Would've been nice if they actually referred to some documentation, unless it's going to be exactly the latest IETF draft but I have some doubts.

@cpu
Copy link

cpu commented Jan 5, 2018

@x1ddos any observed divergences from draft-09 would be welcome as issues on the Boulder repo.

@x1ddos
Copy link
Author

x1ddos commented Jan 5, 2018

@cpu yeah I saw the announcement not long ago. This is great! Will make sure to file issues if I notice anything.

@titanous
Copy link
Member

titanous commented Jan 7, 2018

From #23342:

I think there might actually be other ACME-compatible servers but even if there aren't any, I know for a fact there exist non-public implementations. I'd like to keep v1 working.

I'm now through the bulk of the implementation of v2, and there is no reasonable way to keep support for both the previous version and draft-09 in the same package due to substantial divergence in the ordering flow, object fields, and requests. A few helpers could be re-used, but that's about it. How intent are you on keeping support for "v1"? (which is really a Let's Encrypt specific version that doesn't match any previous draft).

@titanous
Copy link
Member

titanous commented Jan 7, 2018

@bradfitz Do you have any thoughts on if there's a good approach to pulling in pebble for integration tests? Worst case I can make a separate integration testing package and throw it on GitHub, but I'm wondering if there's a good way to get it running with every CL. I haven't come up with a good way to do it without it getting vendored in the x/crypto repo, which seems like a no-go.

@x1ddos
Copy link
Author

x1ddos commented Jan 8, 2018

I'm now through the bulk of the implementation of v2
@titanous where is it. Can I take a look?

@x1ddos
Copy link
Author

x1ddos commented Jan 8, 2018

@titanous also, it's a shame you're implementing something without at least consulting here. why duplicate work?

@titanous
Copy link
Member

titanous commented Jan 8, 2018

also, it's a shame you're implementing something without at least consulting here. why duplicate work?

I did consult, in the inadvertently duplicated #23342, but GitHub appears to have never sent me your reply. Anyway, this was just a weekend project, it's really not a problem for me if we have to change it or even throw the whole thing away. I'll push a CL in a moment.

@gopherbot
Copy link

Change https://golang.org/cl/86635 mentions this issue: acme: Implement IETF draft-09

@x1ddos
Copy link
Author

x1ddos commented Jan 8, 2018

Aside from too many unnecessary renamings, especially of exported types, golang.org/cl/86635 is a good start. But, I'd like to keep v1 working as is, without breaking. Besides, autocert.Manager would need to be changed too, in order to work with v2.

So, how about an acme.ClientV2 or acme/v2 package, or change the type of acme.Client to an interface and have multiple implementations, v1 and v2. It is also easy to automatically infer which version a URL points to, for instance by looking at the Directory response during discovery.

@titanous
Copy link
Member

titanous commented Jan 8, 2018

But, I'd like to keep v1 working as is, without breaking.

Given that v1 is never going to change (it’s a Let’s Encrypt-specific implementation and they are not going to develop the API after deprecation), wouldn’t it make more sense just to snapshot the current v1 implementation into a GitHub repo somewhere? Alternatively we can put v2 into a different package, but do you want to have autocert also support v1?

Besides, autocert.Manager would need to be changed too, in order to work with v2.

Yes, I didn’t want to do that until we’d finalized the v2 client API.

I really don’t think that an interface would properly encapsulate both versions under the same methods and types.

draft-09 is very close to being the final RFC, so I think it’s worth strongly considering making a clean API break to match the new spec (hence the various renames) instead of saddling ourselves with vestiges of what will be a forgotten draft API design (v1).

@x1ddos
Copy link
Author

x1ddos commented Jan 8, 2018

do you want to have autocert also support v1

Yes, because I'd like autocert to always work regardless, as in "go get -u .../autocert" and it works at any point in time.

The v2 implementation isn't stable yet anyway. In my experience, being close to final doesn't make it final and may never will. So, how about we create acme/v2 package and place everything v2-related in there, until it's polished or at least until works well with Let's Encrypt and then see how we can integrate it into the acme and acme/autocert packages.

@titanous
Copy link
Member

titanous commented Jan 8, 2018

Yes, because I'd like autocert to always work regardless, as in "go get -u .../autocert" and it works at any point in time.

To be clear, I was proposing that the cutover to v2 happen after LE turns on their production endpoint, so autocert will always just work.

So, how about we create acme/v2 package and place everything v2-related in there

Sounds good, I'll ping the CL when I have added the remaining tests.

@x1ddos
Copy link
Author

x1ddos commented Jan 8, 2018 via email

@kf6nux
Copy link

kf6nux commented Feb 6, 2018

I don't want to bikeshed on names, but last I heard, the IETF has refused calling this v2 as it's the first version they're ratifying.

I'm concerned that if the IETF ever releases v2 and Go already has a v2, it'll lead to more confusion.

Perhaps naming it ietfv1-acme is the right thing? @x1ddos @titanous

@kf6nux
Copy link

kf6nux commented Mar 13, 2018

Let's Encrypt has launched their v2 service: https://community.letsencrypt.org/t/acme-v2-production-environment-wildcards/55578

@jmhodges
Copy link
Contributor

jmhodges commented Oct 10, 2019

It would be really nice to update that URL constant and finish off this work. I just got hit with the brownout that Let's Encrypt is putting in place and noticed a straight-forward update of crypto/acme didn't fix my problem.

(Boy, am I glad they did that brownout)

@cjellick
Copy link

@jmhodges did you identify a workaround? We're hit with the same problem

@jmhodges
Copy link
Contributor

jmhodges commented Oct 11, 2019

Oh, the solution is straightforward.

When creating your acme.Client, set its DirectoryURL field to "https://acme-v02.api.letsencrypt.org/directory" (for production use) or "https://acme-staging-v02.api.letsencrypt.org/directory" (for live testing).

I think you can't reuse accounts from the v01 API in the v02 one (because of the account's URI being a permanent identifier that includes the API URLs). Because of that, you'll need to make new accounts for the new API.

You can reuse accounts! No changes there needed.

@jmhodges
Copy link
Contributor

(This, of course, means that folks have to write code to make acme.Client, and can't rely on autocert.Manager to do the right thing.)

@bradfitz
Copy link
Contributor

bradfitz commented Oct 11, 2019

https://golang.org/cl/199520 (mentioned above) is in review. Feel free to help test that out.

@gopherbot
Copy link

Change https://golang.org/cl/200638 mentions this issue: acme: update Directory URL to Let's Encrypt v2

gopherbot pushed a commit to golang/crypto that referenced this issue Oct 11, 2019
This updates the default Directory URL to Let's Encrypt v2.
Their v1 has been deprecated and new account registrations
will be disabled in Nov 2019. See the following for details:
https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430

There's no good reason to keep the default URL pointing to v1
any longer.

Updates golang/go#21081

Change-Id: I34520e4dfb04a30d82a50559369bac819076d49c
Reviewed-on: https://go-review.googlesource.com/c/crypto/+/200638
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@cohix
Copy link

cohix commented Oct 11, 2019

Hello, we have customers using an application in production with autocert and letsencrypt, is there a stable workaround that we can use today to enable v2 support for them (I.e. is manually creating the acme.Client with the directory URL referenced above sufficient?), or are more changes needed before we can support this?

@x1ddos
Copy link
Author

x1ddos commented Oct 11, 2019

@cohix you can apply https://golang.org/cl/199520 manually or wait until it's merged.

@cohix
Copy link

cohix commented Oct 11, 2019

@x1ddos is is reasonable to expect it will be merged in time for the v1 EOL next month? I'd prefer not trying to deploy a non-merged patch to my production application :P

@x1ddos
Copy link
Author

x1ddos commented Oct 11, 2019

@cohix it is now merged. see commit message for details: golang/crypto@0e8c3a9

@cohix
Copy link

cohix commented Oct 11, 2019

@x1ddos great news. Tested it just now and all is well. Does it address https://community.letsencrypt.org/t/acme-v2-scheduled-deprecation-of-unauthenticated-resource-gets/74380 as well?

@bradfitz
Copy link
Contributor

@cohix
Copy link

cohix commented Oct 11, 2019

👏 hat tip to you all, thank you for averting a small crisis :)

@gopherbot
Copy link

Change https://golang.org/cl/203919 mentions this issue: acme/autocert: always pass AuthzURLs from AuthorizeOrder to deactivatePendingAuthz

gopherbot pushed a commit to golang/crypto that referenced this issue Oct 29, 2019
…ePendingAuthz

Previously, the o.AuthzURLs slice was sometimes used from the call to
client.WaitOrder at the bottom of the for loop.

By that point, o may be nil if client.WaitOrder returned an error,
which would cause a nil pointer dereference panic inside the deferred
function call. If client.WaitOrder did not return an error, then the
call to deactivatePendingAuthz would use its AuthzURLs slice instead
of the one from client.AuthorizeOrder.

Fixes golang/go#35225
Updates golang/go#21081

Change-Id: I7db055ee1149871b6e5d34a8618526899c68f827
Reviewed-on: https://go-review.googlesource.com/c/crypto/+/203919
Reviewed-by: Alex Vaghin <ddos@google.com>
codebien pushed a commit to codebien/build that referenced this issue Nov 13, 2019
Filippo suggested I add myself (x1ddos) in
golang/go#21081 (comment)

Brad for crypto/acme/autocert.

Change-Id: Ia14a3da50d20e15ff086c05b2a81ee81b149603f
Reviewed-on: https://go-review.googlesource.com/c/build/+/189398
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@golang golang locked and limited conversation to collaborators Oct 28, 2020
c-expert-zigbee pushed a commit to c-expert-zigbee/crypto_go that referenced this issue Mar 28, 2022
The Manager now uses RFC 8555 implementation of Let's Encrypt by default.
Existing users need not do any manual upgrades. If you vendor
acme/autocert, it is enough to just rebuild your binaries at this CL.

If there's an account key stored in Manager's cache which has been used
with an earlier Let's Encrypt implementation (aka v1 or draft-02),
it will be automatically re-registered with the new endpoint.

One notable change is the CAServer from internal/acmetest was amended
to simulate a CA implementing RFC 8555, replacing draft-02.
Support for both RFC and draft-02 seemed too complicated and not worth
the benefits: the old pre-RFC bits will be removed from both acme and
acme/autocert packages at some point.

Fixes golang/go#21081

Change-Id: Id530758ac612b1c20f9df51c4d10f770e5f41ecf
Reviewed-on: https://go-review.googlesource.com/c/crypto/+/199520
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
c-expert-zigbee pushed a commit to c-expert-zigbee/crypto_go that referenced this issue Mar 29, 2022
The Manager now uses RFC 8555 implementation of Let's Encrypt by default.
Existing users need not do any manual upgrades. If you vendor
acme/autocert, it is enough to just rebuild your binaries at this CL.

If there's an account key stored in Manager's cache which has been used
with an earlier Let's Encrypt implementation (aka v1 or draft-02),
it will be automatically re-registered with the new endpoint.

One notable change is the CAServer from internal/acmetest was amended
to simulate a CA implementing RFC 8555, replacing draft-02.
Support for both RFC and draft-02 seemed too complicated and not worth
the benefits: the old pre-RFC bits will be removed from both acme and
acme/autocert packages at some point.

Fixes golang/go#21081

Change-Id: Id530758ac612b1c20f9df51c4d10f770e5f41ecf
Reviewed-on: https://go-review.googlesource.com/c/crypto/+/199520
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
c-expert-zigbee pushed a commit to c-expert-zigbee/crypto_go that referenced this issue Mar 29, 2022
The Manager now uses RFC 8555 implementation of Let's Encrypt by default.
Existing users need not do any manual upgrades. If you vendor
acme/autocert, it is enough to just rebuild your binaries at this CL.

If there's an account key stored in Manager's cache which has been used
with an earlier Let's Encrypt implementation (aka v1 or draft-02),
it will be automatically re-registered with the new endpoint.

One notable change is the CAServer from internal/acmetest was amended
to simulate a CA implementing RFC 8555, replacing draft-02.
Support for both RFC and draft-02 seemed too complicated and not worth
the benefits: the old pre-RFC bits will be removed from both acme and
acme/autocert packages at some point.

Fixes golang/go#21081

Change-Id: Id530758ac612b1c20f9df51c4d10f770e5f41ecf
Reviewed-on: https://go-review.googlesource.com/c/crypto/+/199520
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@rsc rsc unassigned x1ddos Jun 23, 2022
LewiGoddard pushed a commit to LewiGoddard/crypto that referenced this issue Feb 16, 2023
The Manager now uses RFC 8555 implementation of Let's Encrypt by default.
Existing users need not do any manual upgrades. If you vendor
acme/autocert, it is enough to just rebuild your binaries at this CL.

If there's an account key stored in Manager's cache which has been used
with an earlier Let's Encrypt implementation (aka v1 or draft-02),
it will be automatically re-registered with the new endpoint.

One notable change is the CAServer from internal/acmetest was amended
to simulate a CA implementing RFC 8555, replacing draft-02.
Support for both RFC and draft-02 seemed too complicated and not worth
the benefits: the old pre-RFC bits will be removed from both acme and
acme/autocert packages at some point.

Fixes golang/go#21081

Change-Id: Id530758ac612b1c20f9df51c4d10f770e5f41ecf
Reviewed-on: https://go-review.googlesource.com/c/crypto/+/199520
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
BiiChris pushed a commit to BiiChris/crypto that referenced this issue Sep 15, 2023
The Manager now uses RFC 8555 implementation of Let's Encrypt by default.
Existing users need not do any manual upgrades. If you vendor
acme/autocert, it is enough to just rebuild your binaries at this CL.

If there's an account key stored in Manager's cache which has been used
with an earlier Let's Encrypt implementation (aka v1 or draft-02),
it will be automatically re-registered with the new endpoint.

One notable change is the CAServer from internal/acmetest was amended
to simulate a CA implementing RFC 8555, replacing draft-02.
Support for both RFC and draft-02 seemed too complicated and not worth
the benefits: the old pre-RFC bits will be removed from both acme and
acme/autocert packages at some point.

Fixes golang/go#21081

Change-Id: Id530758ac612b1c20f9df51c4d10f770e5f41ecf
Reviewed-on: https://go-review.googlesource.com/c/crypto/+/199520
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests