Like what I'm seeing so far!

I wonder if the ACME configuration should be in a separate struct value -- do we want to tether the http.Server type to ACME concretely?

One of the requests we've had in Caddy is to abstract the way certificates are Obtain()ed and Renew()ed -- in other words, an interface with approximately these two methods. An ACME client would be one implementation, a hashicorp/vault implementation might be another, etc. So I wonder if, instead, there should be some sort of interface type (TLSManager? Not sure what you'd call it), which some type tls.ACMEClient implements (or something like that).

In fact, thinking on these lines, it might be beneficial to drop down to the tls package somehow when working with ACME. In other words, you create a TLS listener that has some notion of ACME, but the HTTP server doesn't care. It's just one application of TLS and uses that listener and goes about its usual business.

FWIW, In Caddy I found myself writing essentially a wrapper type over tls.Config that specifies all the ACME stuff and makes the GetCertificate callback, etc. It all happens within TLS (not just HTTPS), so how about an easy way to make an ACME-capable TLS listener that you pass into http.Serve()?

I've been moving all of my Go servers to use ACME (with letsencrypt) and I've been using Russ's letsencrypt package for now because it is so simple and easy. However, I've also contributed to the acme package that @crhym3 wrote because its design was inspired by Russ's package and because I wanted ACME to be in the stdlib. I think it is a very common use case that we should cater towards.

However, I don't like the use of fields in http.Server to configure ACME. It's very easy to just create a tls.Listener using acme.Manager's GetCertificate callback. However for convenience, it just makes more sense for this to be part of crypto/tls rather than purely net/http so that even generic TLS servers can use ACME more easily.

(as I was writing this, I noticed @mholt's comment which I 👍)

This is super cool!

With my Let's Encrypt hat on: One important thing to consider though is that currently neither the ACME spec nor the API implementation provided by Let's Encrypt should be considered completely stable. The Let's Encrypt API currently implements a amalgam of the four ACME RFC drafts which we colloquially refer to as v01. Once the RFC is finalized we intend to implement a new API version, v02, which will be a strict implementation of the final specification language which will not be completely backwards compatible with v01.

Once v02 is made public (we are currently aiming for somewhere around the start of 2017) we will provide a timetable for depreciating the v01 implementation . Given this might it make sense to wait until the finalized stable API is available before adding this support to the stdlib in order to prevent churn and breakage in users who don't update?

My goal is for HTTPS to be dead simple out of the box

👍🏻

Once v02 is made public (we are currently aiming for somewhere around the start of 2017) we will provide a timetable for depreciating the v01 implementation . Given this might it make sense to wait until the finalized stable API is available before adding this support to the stdlib in order to prevent churn and breakage in users who don't update?

I totally agree. But Go 1.8 isn't due until 2017 anyway. The timing might work out? But even Go 1.9 is acceptable, if that works out best timing-wse.

Another Let's Encrypt developer here. Wrote out a post saying what @rolandshoemaker said about upcoming ACME changes, but he beat me to it!

Also, ideally the design should encourage disk-based caching as much as possible. If Go programs are designed to issue on each startup, people will very quickly run into Let's Encrypt rate limits. I see that autocert already has a Cache, so it's probably enough to include the Cache mechanism in the library and make sure the examples use it.

Lastly: This is great, and I'm super happy to see it. This is exactly the type of thing Let's Encrypt exists to enable. Thanks!

@jsha, as I said in my first post, caching would automatic. There wouldn't even be an option to disable it. Only an option to change the directory it uses.

Got it! I read too quickly. :-)

As awesome as this sounds, I think this shouldn't be included in the standard library.

I understand that people will say: it's about security, or other arguments, which, yes, are valid.

But please consider this:

• the ACME standards are in in Draft right now
• the functionality exists outside of the standard library just fine

Considering proposals to include different useful things in the language or standard library have been shut down of over the course of years, I would like to know if this means a change in vision on how the standard library / language should evolve from here on out. Does it mean that I can raise a proposal for having ? : syntax in the language? It is super useful in so many cases but it has been deemed not necessary as we can do if / else.

@mholt, @nhooyr, as I wrote:

The ACMEFoo names are placeholders. Maybe they'd deserve their own struct.

I agree they're probably too stuttery. Being a listener is interesting, though. Maybe:

package main

import (
    "crypto/tls"
    "log"
    "net/http"
)

func main() {
    http.HandleFunc("/", websiteHandler)
    var srv http.Server
    log.Fatal(srv.Serve(&tls.ACMEListener{
        Domain:   "foo.com",
        Provider: tls.LetsEncrypt,
        AgreeTOS: tls.YesIAgreeTOS,
    }))
}

@bradfitz I like that. But now, I'm not sure if crypto/tls is the best place for such a listener. Perhaps we should add acme.Listener?

@dlsniper, the timing with regards to its draft status is discussed above. See comments from @rolandshoemaker and @jsha.

the functionality exists outside of the standard library just fine

Yes. And maybe all we do here is add documentation examples on how to use autocert. Maybe that's enough. I'd like it to be even easier, though.

Does it mean that I can raise a proposal for having ? : syntax in the language?

You can propose anything you like, but that discussion happened a long time ago and was already decided. If you insist on raising it again, please do it elsewhere and not in this issue.

@nhooyr, I don't really want to add a new package, though.

We already have https://golang.org/pkg/net/http/httputil/ though. Maybe that's a suitable location.

@bradfitz That's better. I imagine the tls.ACMEListener will listen only on port 443?

And maybe the Provider value could use a sensible default. Let's Encrypt's endpoint is a good one -- the question is, should it be their staging endpoint or production one? If people are testing their Go program against the "default" endpoint which happens to be the production endpoint, especially if they're wiping their file system (containers?) or cleaning a test directory or something, it could hit CA rate limits pretty fast. So maybe, on second thought, they should be required to specify an endpoint either way...

And why not just true for agree TOS?

@bradfitz actually, I meant the high level package https://godoc.org/golang.org/x/crypto/acme/autocert.

So

package main

import (
    "crypto/acme"
    "crypto/acme/autocert"
    "crypto/tls"
    "log"
    "net/http"
)

func main() {
    http.HandleFunc("/", websiteHandler)
    var srv http.Server
    log.Fatal(srv.Serve(&autocert.Listener{
        Domain:   "foo.com",
        Provider: acme.LetsEncrypt,
        AgreeTOS: autocert.AcceptTOS,
    }))
}

I think it would fit in with autocert being the high level package for using acme.

@mholt, I addressed my opinion on the default in my initial proposal text at top:

ACMEServer would be required as the opt-in, and we wouldn't make LetsEncrypt be automatic or preferred, but we would add a constant const LetsEncrypt = "https://acme-v01.api.letsencrypt.org/directory" like the acme package has.

We could provide two LetsEncrypt constants for prod vs staging but I don't want to make LetsEncrypt be automatic. Depending on any specific organization (for-profit or otherwise) rubs me the wrong way. I want users to explicitly opt-in to it and know what's happening. (Or hopefully know what's happening by being curious what the line Provider: acme.LetsEncrypt means)

@mholt the acme package uses a callback to determine whether or not you agree to the TOS. the autocert package provides a convenient AcceptTOS function that always returns true. Maybe we could just remove it? I think pretty much everyone just agrees, even if they don't read it. For people who actually care, they can use autocert.

@nhooyr

I think it would fit in with autocert being the high level package for using acme.

The only issue there is that you then have to import an external package to use it.

@nhooyr, you can basically already do that. This proposal is about whether Go should come out of the box with ACME support, now that ACME is a thing. Go has always been very pro-easy-HTTPS, but ACME wasn't an option at the time when Go first came out.

The question is whether we continue the tradition of promoting easy HTTPS.

@nhooyr,

Maybe we could just remove it? I think pretty much everyone just agrees, even if they don't read it.

I already addressed that. See my original text at the top of this issue.

@bradfitz

I addressed my opinion on the default in my initial proposal text at top

Oops, you're right.

Depending on any specific organization (for-profit or otherwise) rubs me the wrong way. ... I want users to explicitly opt-in to it and know what's happening.

This makes sense. And I agree with this philosophy. But if you're going to provide a const for Let's Encrypt, would you have to do that for all public ACME CAs? Maybe the user should just provide their own CA URL string.

@bradfitz
I understand now. In that case, the current approach with adding it to crypto/tls seems fine to me (though I'd definitely prefer a separate package but I can see why you don't want add more packages to the stdlib).
I was under the impression that the acme package would be added to the standard library which would make it's inclusion in crypto/tls or net/http redundant. I think I remember @crhym3 mentioning it somewhere. I'll try and find it. (edit: never mind I misinterpreted, sorry).

@mholt
I thought Lets Encrypt was the de facto ACME CA? I think it's used enough to warrant it's inclusion as a package level const because I am not even aware of any other public ACME CA.

But if you're going to provide a const for Let's Encrypt, would you have to do that for all public ACME CAs?

Sure. Or for all public, non-profit (or at least free) ones. I'd rather the user not have to provide a big ugly string. I'd like to keep the boilerplate as tidy as possible.

@nhooyr,

I thought Lets Encrypt was the de facto ACME CA? I think it's used enough to warrant it's inclusion as a package level const because I am not even aware of any other public ACME CA.

Let's Encrypt created the ACME spec, but anybody can implement it. StartCom/StartSSL has announced they plan to use ACME: https://www.ietf.org/mail-archive/web/acme/current/msg01290.html So maybe there will be two. We'll see.

@bradfitz
Why would we need a second public free CA for ACME? StartCom/StartSSL wants to use it for customers which according to your earlier comment means it wouldn't be included and I agree. For anyone starting another public free ACME CA this seems relevant.

The proposal would dramatically increase the amount of code net/http depends on. For that reason alone, I'd prefer to leave ACME support where it is and settle on clean, standard way to use it. I do not believe it is necessary for it to be in the standard repo for it to be easy to use.

To put it another way, you have stated your goal of easy security, which I applaud, but there has not been sufficient study of the various ways to do that.

@robpike, it's not much code, if that's the root of your argument.

Given that it's not easy today, why not make it easy with an x/ import first and then reevaluate whether it needs the extra step to go into the standard library? I could see an argument for being in the standard library if it were also going to be on by default, but that doesn't seem right anyway. The x/ repos are supposed to be staging for the standard library (like we did with context) but in this case there has been no staging.

Does Go have a policy about the docs for the standard library including
examples of things in x/ libraries, or even external libraries? It seems
like a great set of first steps is 1) making the API as easy as envisioned
here with an acme.Listener, 2) having the net/http docs examples include
this; neither of which require the Go team to make a long term commitment.

On Thu, Sep 15, 2016 at 2:07 PM, Russ Cox notifications@github.com wrote:

Given that it's not easy today, why not make it easy with an x/ import
first and then reevaluate whether it needs the extra step to go into the
standard library? I could see an argument for being in the standard library
if it were also going to be on by default, but that doesn't seem right
anyway. The x/ repos are supposed to be staging for the standard library
(like we did with context) but in this case there has been no staging.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#17053 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAADBKmmgsAs9tI_mAzFG-nj_IR3wpMfks5qqYlUgaJpZM4J5z9d
.

"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: D1B3 ADC0 E023 8CA6

@rsc, @alex, sounds good. I'll start with the autocert.Listener type and some example docs for std.

Stop sending these to me!

torsdag 15 september 2016 skrev Brad Fitzpatrick notifications@github.com:

@rsc https://github.com/rsc, @alex https://github.com/alex, sounds
good. I'll start with the autocert.Listener type and some example docs for
std.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#17053 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACxR0MYNG5u462so5rcGRgvTjxYcjXe9ks5qqY4UgaJpZM4J5z9d
.

@raski, it looks like you were mentioned on accident above instead of @rasky. You'll have to press the "Unsubscribe" button on the right. This is a Github thing. I can't unsubscribe you.

Allowing the cache implementation (not just directory) to be overridden seems essential to me. Eg, if you are running a binary that compiles this code in on many servers (either your actual app or a custom proxy) which do not share a disk you likely do not want them all to individually and separately talk to LetsEncrypt as that will quickly exhaust your rate limits.

(Otherwise +1 on this general idea. We are a production user of github.com/hlandau/acme but hopefully this library provides similar support, or we can just keep using hlandau.)

Wouldn't it make sense to have something like a security/driver package (along the lines of database/sql/driver) so that one can use e.g. acme or any other provider that has an API and decouple the implementation from the interface? It seems to work well enough for database/sql/driver and there is no actual database driver in the language itself.

Note: I'm not saying that the package shouldn't be included, just that it might be a good idea to have it in a way that makes it possible to swap implementation.

Another argument for a separate listener vs http.Server: presumably you need to configure what domain you're requesting the cert for (vs getting a cert for whatever shows up in a Host header?) In the original snippet I guess the implication was that it derives that from Addr, but it's very common for a server to live behind a proxy and not listen on an interface that directly answers to the domain name for which the cert is needed.

I should also add that if this is done as proposed, any other ACME implementation (and there are many) will almost certainly (and should) use net/http, and thus end up importing this ACME implementation as well. That's rather unpleasant.

Thanks everyone for the feedback. There's some good stuff in here. However, given the length of this thread and the unlikelihood that anybody's read it all, I think it's time to freeze this thread until it's time for any next step, which won't be at least 4-6 months probably.

The plan at this point is just to implement more in x/crypto/acme and x/crypto/acme/autocert and add more documentation+examples to net/http for Go 1.8.

Ping @bradfitz - We would like to see the documentation.

CL https://golang.org/cl/39207 mentions this issue.