Why not both? In the go mod world, it's been formalized that if you make an incompatible revision of a library, it has to live at v2/ forever. So, we could commit new code that supports RFC 8555 under crypto/acme/v2, and keep the v0 library under crypto/acme. That would allow current code to keep working, as well as providing an upgrade path ASAP.

This related to #21324 - with go mod support becoming mainstream in 1.13, I think it's valuable for the x/all libraries to show confidence in the versioning plan that Go recommends.

Another trouble with /v1 and /v2 I forgot to mention is autocert: it would need to import both and use one or the other based on the ACME version. It complicates autocert code too much imho.

Alternatively, you'd need to make separate /v2/autocert. Even more maintenance hell.

Regarding this con in the first plan:

makes acme and autocert packages incompatible with other ACME CAs which haven't
implemented RFC8555 yet; at the time of writing:

has there been positive confirmation that any of these CA's implementation of "ACME v1" function with the existing codebase?

I don't think it can be taken on faith or based on limited public documentation. There's no specification for ACME v1 and no single draft that can be identified as a closest match. I don't think losing compatibility with these CAs should be considered as a con unless it can be confirmed they were compatible to begin with.

I just tried now with the first in the list using a small cmd line tool which calls acme.Client directly.

$ acme reg -gen -accept -d https://api.buypass.com/acme/directory mailto:XXX
$ acme whoami
URI:		 https://api.buypass.com/acme/acct/1TYYjD4k1-6bNw
Key:		 ZZZ/.config/acme/account.key
Contact:	 mailto:XXX
Terms:		 https://api.buypass.com/acme/terms/750
Accepted:	 yes

$ acme cert -d https://api.buypass.com/acme/directory -expiry 24h -dns one.acmetest.bots.run
Add a TXT record for _acme-challenge.one.acmetest.bots.run with the value "aya3lx3gaVWF31c61-i_lwX_tkmnjqrpQyNuf3vFn74" and press enter after it has propagated.
<ENTER>
cert url: https://api.buypass.com/acme/cert/K1xHK-DunMk

$ openssl x509 -noout -serial -issuer -dates -subject -in ~/.config/acme/one.acmetest.bots.run.crt
serial=10619A17D62C509CF49B
issuer=C = NO, O = Buypass AS-983163327, CN = Buypass Class 2 CA 5
notBefore=Jul 24 18:38:25 2019 GMT
notAfter=Jul 25 21:59:00 2019 GMT
subject=CN = one.acmetest.bots.run

Can try with some others but it's missing the point a bit. The list I composed is just a sample. Plus, some private CAs like I mentioned.

@x1ddos: Why is it missing the point? Your con says:

"makes acme and autocert packages incompatible with other ACME CAs which haven't implemented RFC8555 yet".

I think establishing whether it was compatible or not to begin with is important in understanding the cost of breaking the compatibility. Since there is no specification its very difficult to ascertain compatibility without experimentation. Entrust's documentation specifically assumes Certbot, and I suspect other pre-RFC 8555 CAs used the "it works with Certbot so it's ACME v1 compatible" approach to interop.

I'm convinced that Buypass' pre-RFC 8555 ACME endpoint is compatible but are the others? Breaking compatibility with 1 CA is a different level of compromise than breaking compatibility with 5+.

Why is it missing the point?

• realistically, you'll never get a confirmation from all of them and they won't upgrade to be RFC compliant in 2 months anyway
• there are always hacks. Many have tried to reach a compliance in HTTP servers with browser vendors or have a clean OAuth2 client implementation. Never worked. I doubt it'll be that much different with RFC8555.
• so, to extend the previous point, if the package will indeed contain hacks, I'd rather it worked with 5+ CAs than 1 or 2. Dropping everything just to be RFC compliant and nothing else doesn't really help it and would certainly make it much harder to cover potential special cases in 5+ CAs in the future which haven't reached RFC.

realistically, you'll never get a confirmation from all of them

Indeed. So the compatibility being lost is primarily hypothetical 👍

Many have tried to reach a compliance in HTTP servers with browser vendors or have a clean OAuth2 client implementation. Never worked. I doubt it'll be that much different with RFC8555.

Perhaps, but like with HTTP for RFC 8555 there is standards text that can be used to mediate compliance discussions as a point of common reference. There is no such thing for ACME v1. There is no way to determine if a change requested by a server operator for compliance is a one-off request for a specific implementation, or a deviation from a standard that would affect multiple implementations without experimentation.

so, to extend the previous point, if the package will indeed contain hacks, I'd rather it worked with 5+ CAs than 1 or 2

It seems like we're in agreement then: to know how to evaluate the hacks required means needing to understand the CAs that are compatible with this package now, and how many will be compatible based on proposed changes to the package.

@cpu this is a nice theory. Unfortunately, practice tells me a different story which is why I expressed my preference towards option 2.

@cpu I think the point here is not to say that the library shouldn't upgrade to ACMEv2, it is to say, should it do so in a way that breaks people with no notice. I would go so far to say if a package broke me with no notice on a major thing like this I would have trust issues with the package.

With that in mind, it does not seem unreasonable to me to try to maintain backward compatibility with the current behavior for some period of time.

To be clear personally, I do not like the idea of a merged implementation as is represented in option #2. With that said, it does sound like it doesn't break anyone (which I think is important) and gets Let's Encrypt ACMEv2 in the library in time for their migration from their "past ACME implementation" to the 8555 compatible one.

I would like to recommend that if that approach is taken it comes with a statement that the prior implementation will be dropped at some specific and notified point of time so a cleaner and more maintainable library is in place long term.

In Go, there is always an implicit compatibility target, which is what currently works.

I agree that there is no such a thing as ACMEv1 to target, so now that there is a specified ACME protocol we shouldn't spend any more effort chasing compatibility with pre-RFC CAs. But that's different from changing behavior in such a way that breaks what used to work before the change.

As reflected by both the the Go 1 Compatibility Promise and by modules' semantic versioning (although this package is not bound by either), the ecosystem really values this kind of stability.

That doesn't mean we need to keep supporting "ACMEv1", but precisely the servers that already work, because that's what users will have tested and developed an expectation for.

Considering this, option 2 here is generally preferable, if the API complexity cost is moderate. It looks like there wouldn't be more than a couple of v1-only and v2-only methods, which looks acceptable. I like the Discover based autodetection.

As an implementation strategy, I'd like to see v2 code in separate files as much as possible, even if it means more code duplication. That should make it easier to rip out v1 eventually, and will ensure the v2 implementation is good enough to stand on its own.

Ah, about using modules versioning for this: there are two ways to do it, either bumping all of x/crypto to v2, causing an import path change that is not necessary for most packages, or splitting x/crypto/acme into its own module, and then bumping that.

Nested modules are a lot of complexity, they carve a hole in the parent module, and interact weirdly with it. To maintain compatibility we'd probably have to keep the current acme package in x/crypto in its current state, but we'd probably not want to have two different autocert versions, and I am not sure we can redirect from one to the other. I also worry that v2 would be less discoverable than v0 because the latter would be what projects already have in their go.mod.

It's a judgement call, but I think this complexity is more than that of supporting both protocols in the same package.

Just chiming in that we are starting to see failures from this now:
Velocidex/velociraptor#120

This message https://community.letsencrypt.org/t/important-notice-to-acme-client-developers-regarding-acme-v1-deprecation/100795 points me to a list of platforms that support this new protocol here https://github.com/letsencrypt/website/blob/master/data/clients.json and autocert seems to be the only golang option and it does not support ACMEv2.

If we are starting to see rolling outages in prod it would be nice to have this fixed ASAP (or at least have some supported alternative for the time being).

Ah just saw golang/crypto@0e8c3a9 thanks for that! We will rebuild our binaries at that commit.

Thanks for the prompt fixes. Maybe this issue can be closed?

This got implemented. Closing. 🙌