I'm a big fan of LetsEncrypt, but please describe what you need and not what your proposed fix is. The proposed fix is not acceptable (we won't be exposing internals), but I don't even know what the problem is.

Thanks Brad,

Here's a user story:

• Bob sign up with Awesome Host
• He gets a free domain that points to one of Awesome Host's servers
• He uploads some files and they go into a specific path /srv/www/bob.example.com
• Now someone visits bob.example.com.
• The server checks in it's [memory] cache, but there is no certificate
• There server checks a well-known location on disk /etc/letsencrypt/live/bob.example.com, no dice
• The server runs letsencrypt and then downloads the certificates to /etc/letsencrypt/live/bob.example.com

Instead of knowing what all hosts and certificates are beforehand, and without having to restart the server, the service can host secure websites dynamically. That's what I'm doing. It's as easy as pie in node.js because they expose an SNICallback before certificates are handled.

Since go currently requires so much statically defined up-front, the easiest way to accomplish this is to manually create a tlsConn from a net.Conn and then pass that to http.newConn.

Also, it's not exposing "internals" any more than it used to or any more than the TLS and net modules currently do (see my example). It's simply exposing a capability.

I suppose you could also add an SNICallback like node.js does, but from the code I looked at, it seems that that would be far more changes rather than taking the approach that http.ListenAndServe and http.ListenAndServeTLS approach, which would simply encapsulate capability in a composite style.

Another option perhaps would be to move a more complete version of what I've created as https.ListenAndServeSNI, which still needs an SNICallback.

Is that explanation clear?

I think you just need to implement your own net.Listener and pass that to a net.Server's Serve. The net.Listener can return any net.Conn with its own *tls.Conn validating things as needed.

So you need to present a TLS server certificate as a function of the client's presented SNI hostname?

/cc @agl

Yes, that's what I've done. But then I also need to pass it to http.NewConn.

And yes, I need to see the servername in order to know which certificate to pass, I won't know it on startup.

That's all being done in the demo I've been referencing here:
See https://github.com/coolaj86/authentication-as-a-service/blob/master/https/https.go#L64

(the repo is obviously not intended for the story I described, but it may have some similar aspects - and one thing leads to another...)

This is actually my first go project (coming from node.js).

Does crypto/tls.Config.GetCertificate not work for you? (Note: the behaviour of that changed a little since Go 1.4.)

I don't see what that has to do with the server sending the client a certificate dynamically, but I'm brand new to go.

Here's the rough code without error checking, channels, goroutines, etc:

// I start a raw TCP server
ln, err := net.Listen("tcp", ":" + config.Port)

// I begin accepting connections
conn, err := ln.Accept()

// the vhost pkg peeks at the incoming buffer and let's me see the servername
tlsConn, err := vhost.TLS(conn)

// I grab the SNI
servername := tlsConn.Host()

// Now I want to determine which certificate to issue
// 1. check some cache I create (or maybe GetCertificate?)
// 2. if the expiration is bad, jump ahead to step 5
// 3. if it doesn't exist check on the file system
// 3. if it doesn't exist try SNICallback (from user code)
// 4. if it doesn't exist, try retrieving a brand new certificate from letsencrypt.org
// 5. if that fails, return a dummy certificate

// Now make the encrypted TCP plainly readable
plainConn := tls.Server(tlsConn, tlsConfig)

// Now make it an http server request
srv := &http.Server{Handler: myHandler}
c, err := srv.NewConn(plainConn)

// Now handle that http request
go c.Serve()

When the server starts I may not have the certificate.
I want to delay creating a tls.Config object until I know the servername indication.
Then I want to handle the request as http.

What I'm trying to build is basically

type SNICallback func(domainname string) (t *tls.Config)
http.ListenAndServeSNI(addr string, sniCallback SNICallback, handler Handler) error

Except I want a build a higher-level function that would use such a function with letsencrypt. For the time being I'd just shell out to the letsencrypt python client and there's someone already working on a go client for use with caddyserver (and hopefully the code I'm working on will also find its way there).

Sorry, I didn't understand what GetCertificate was when I read through it the first time. I thought that was saying to give me the certificate that was issued to the client.

GetCertificate == SNICallback

Okay, yes, that's the piece that I was missing and didn't discover as I was greping and googling and ctrl+Fing around for "SNI" and "ServerName", and whatnot.

Updated: I had ClientCAs confused with RootCAs, now this reads correctly

I got my demo implemented and then I realized that there's not a way to specify the x509.CertPool with tls.Conn.GetCertificate.

I could keep tlsConfig.RootCAs and add to it, but then I have no way to manage it and the docs specify that I must not modify it, so I couldn't replace it.

I could solve this problem manually, but again, there's no way for me to cast net.Conn or tls.Conn to http.Server.Conn, so again, I can't.

Why would I want to manage tlsConfig.RootCAs myself?

Let's say that I'm loading hundreds or thousands of certificates over a long period of time - many of which are transient, occasionally I may want to flush the pool and start over (kilobytes turn to megabytes and so on).

Also, I assume that the x509.CertPool is inspected in such a way that duplicate chains are ignored and not added and that the only the necessary chain is sent to the browser based on the request, but the documentation doesn't explicitly state either for tls.Conn.RootCAs or x509.CertPool.

On second thought, maybe that's just me exhibiting typical control-freakishness programmer paranoia.

My app probably won't become that popular and even if I were BlueHost and I loaded every intermediate certificate in the whole world over the next 3 years without restarting the service, there are only maybe a few hundred intermediates in the world and since most of the certificates would be provisioned from the top 10%, it would only be a few dozen that actually get loaded.

What do you think?

Is there value in being able to replace CertPool?

Is there value in being able to create http.Server.Conn from net.Conn?

I'm afraid the above doesn't make any sense to me and I suspect that I'm lacking the required context.

GetCertificates is about a server's certificate chain, but an x509.CertPool is used when validating certificates. Is your server doing TLS client-auth and thus validating certificates from a client?

Again, thanks for taking the time to help.

No, I'm not validating client certificates. Apparently I meant RootCAs the whole time. The naming conventions are different in go than what I'm used to in nodeland and such.

My demo worked, so I thought I was using the right one... but since in the case of my demo there is no intermediate ca in the chain, only my fabricated root ca, it would have worked no matter what because I had the client already accept my fabricated root ca.

I realize now that I probably should have been using RootCAs. Since a browser doesn't need to receive the root ca from a server - it already has it's own list and only the intermediate cas need to be sent, I misunderstood RootCAs to mean "certificates the server uses to validate client certificates" and ClientCAs to mean "the chains that the server serves to clients to validate against their Root CAs", but now I see that it's the opposite.

Concern 1: Can't Create http.Server.Conn from net.Conn

In order to do things like count bytes of network traffic to a particular site, or examine raw headers (it seems http.Server.Conn.Request may strip certain headers - for example 'Transfer-Encoding' is stripped from client requests - which is odd, but not against the spec) or anything that relies on having access to the raw underlying socket, I need to be able to create http.Server.Conn from net.Conn so that I can spy on net.Conn or tls.Conn directly.

Concern 2: Can't replace or delete from RootCAs

Realistically, this probably isn't worth addressing if my current understanding is correct.

In short:

Also I'm always adding to RootCAs and I never have a way of removing from it or replacing it, it could grow to an arbitrary size, full of old expired certificates. However, since there probably aren't enough root and intermediate CAs in the whole world to take up more than a few megabytes, this isn't a problem as long as my assumptions that duplicate certs are ignored and the browser is sent only the subset of necessary certs in the chain.

Back the the earlier example as context:

1. Imagine a shared hosting situation where hosts are added and deleted on the fly
2. A user will automatically be provisioned certs from letsencrypt.org if no custom certs are provided (such as EV certs)

And what happens when a browser connects

1. The browser provides SNI
2. The server selects some certificates via tls.Config.GetCertificate
3. The server sends those certificates along with the chain from some (or all?) of the x509.CertPool to the browser
4. The browser doesn't recognize the signing authority of the certificate, it crawls the chain
5. If it finds an authority it knows, all green locks, EV bars, etc
6. If it doesn't find that authority, it's a big scary red page

And what happens in the code

1. tls.Conn.ClientHello.ServerName is sent by the browser, say foo.example.com
2. tls.Config.GetCertificate is called to retrieve the correct certificate
3. tls.Config.GetCertificate does not provide x509.CertPool
4. The chain for foo.example.com may be different from bar.example.net
5. To ensure the correct chain is in the pool, I must call x509.CertPool.AppendCertsFromPEM(buf)
6. I assume that internally x509.CertPool is a map
7. I assume that calling tls.Config.RootCAs.AppendCertsFromPEM(buf) is an exception to the documentation's declaration that "After tls.Config has been passed to a TLS function it must not be modified"
8. I assume that if I add a duplicate certificate chain to the pool, all duplicate certs in the chain are ignored
9. I assume that tls.Conn pulls the chain from tls.Config.RootCAs (instance of x509.CertPool) to respond to the browser's request, using only the necessary members of the chain
10. The browser receives the certificate issued by tls.Config.GetCertificate and the chain to validate against as only the relevant certs from tls.Config.RootCAs.

Concern 1: Can't Create http.Server.Conn from net.Conn

Brad is the expert here; I really know nothing. However, there is net/http.Server.Serve, which allows one to use any net.Listener as a source of connections. Since net.Listener is an interface, your concrete type can do whatever you want to the connections it produces. I suspect that would give you the flexibility that you need. (For an example, see the source code to ListenAndServeTLS.)

Concern 2: Can't replace or delete from RootCAs

There's an error in your understanding here: x509.CertPool is used only for verification. Your step three, under "And what happens when a browser connects" is thus incorrect—the certificate chain that the server returns is taken only from the result of GetCertificate. Although the return value of GetCertificate is a *Certificate, note that the structure contains a [][]byte for the actual DER data. In Go that's like a std::vector<std::string> from C++: it's one or more DER-encoded certificates. The server returns exactly those certificates to the client, and in the given order. It doesn't try to build a chain at all.

As for the Listener interface, I think I understand what you're saying - I need to create my own type with an Accept and Close method, etc, and do my magic at that layer which I then pass along.

As for the x509 stuff, does that mean that LoadX509KeyPair is not expecting just a key and server cert pair, but rather the private key and the complete bundle of all intermediate certificates as well as the server certificate?

My experience with TLS has all been in node.js, which is extremely explicit about how to load certificates - private, cert, and chain are always explicitly declared as such, not bundled - and haproxy, which is extremely lax - bundle the files any which way as long as no true root (self-signed) cas are included.

Go's conventions are different from either of theirs so I'm being humbled quite a bit to realize that I don't know these things as well as I thought.

I'll have to play around with this on Monday, but I think you've helped me understand both some of the idiomaticies of go that were eluding me and some implementation details that I had the wrong understanding of.

Thank you very much.

And it turns out that I've been doing it wrong in node.js for the past year too. It just always happened to work because I always happened to be using well-known intermediate certificate authorities.

Alas I have discovered the error of my ways with a little help from openssl.

openssl s_client -showcerts -connect example.com:443 -servername example.com -CAfile /path/to/root.pem

How embarrassing. My entire life is a lie... I'm gonna go cry in the bathroom now.

...

Thanks so much for all the help. My understanding of Go's tls handing and how interfaces work is up by about a billion and I finally got a demo that works without crazy changes. https://github.com/coolaj86/golang-https-example