net/http: EOF during TLS handshake #61721

go-aegian · 2023-08-02T20:32:57Z

The reported EOF error, happens when tls.ClientAuth is set to tls.VerifyClientCertIfGiven - if no certificate is provided or request comes from any browser it throws this error, and it should not given that only if Cert if given, unless there is a way to setup the browser to send a specific client cert?

File - /src/net/http/server.go

Current version -

if err := tlsConn.HandshakeContext(ctx); err != nil {
			// If the handshake failed due to the client not speaking
			// TLS, assume they're speaking plaintext HTTP and write a
			// 400 response on the TLS conn's underlying net.Conn.
			if re, ok := err.(tls.RecordHeaderError); ok && re.Conn != nil && tlsRecordHeaderLooksLikeHTTP(re.RecordHeader) {
				io.WriteString(re.Conn, "HTTP/1.0 400 Bad Request\r\n\r\nClient sent an HTTP request to an HTTPS server.\n")
				re.Conn.Close()
				return
			}
			c.server.logf("http: TLS handshake error from %s: %v", c.rwc.RemoteAddr(), err)
			return
		}

Propose Fix - tested on my environment - add this to the if line - && err.Error() != "EOF"

if err := tlsConn.HandshakeContext(ctx); err != nil && err.Error() != "EOF" {
			// If the handshake failed due to the client not speaking
			// TLS, assume they're speaking plaintext HTTP and write a
			// 400 response on the TLS conn's underlying net.Conn.
			if re, ok := err.(tls.RecordHeaderError); ok && re.Conn != nil && tlsRecordHeaderLooksLikeHTTP(re.RecordHeader) {
				io.WriteString(re.Conn, "HTTP/1.0 400 Bad Request\r\n\r\nClient sent an HTTP request to an HTTPS server.\n")
				re.Conn.Close()
				return
			}
			c.server.logf("http: TLS handshake error from %s: %v", c.rwc.RemoteAddr(), err)
			return
		}

The text was updated successfully, but these errors were encountered:

mauri870 · 2023-08-02T20:52:03Z

You could use err != nil && err != io.EOF. Not sure tho if this is okay to change. Do you have a reproducible example to share?

mauri870 · 2023-08-02T20:58:11Z

Please change the title to net/http: EOF during TLS handshake

I feel like an EOF can't be ignored here, but hard to say without knowing the context to reproduce this. Likely the connection was refused for a reason, maybe the client asked for a protocol that the server does not support.

If possible share the Go version you are using alongside with a reproducible test case.

Might be related to #13523, #50984

go-aegian · 2023-08-02T21:24:18Z

Changed title.

Yes that will work too - err != nil && err != io.EOF

Here is an example to run the server on https://localhost:8300/test - open a new browser session after starting the server as it only happens on handshake not when opening a new browser tab after hitting the url

create a certificates folder and add in there a ca.crt, localhost.crt and localhost.key that you can generate on your end


package main

import (
	"crypto/rand"
	"crypto/tls"
	"crypto/x509"
	"fmt"
	"log"
	"net/http"
	"os"
	"strings"
	"time"
	
	"github.com/pkg/errors"
	"golang.org/x/net/http2"
)

var Ciphers = []uint16{
	// TLS 1.3
	tls.TLS_AES_256_GCM_SHA384,
	tls.TLS_AES_128_GCM_SHA256,
	tls.TLS_CHACHA20_POLY1305_SHA256,
	
	// ECDSA is about 3 times faster than RSA on the server side.
	tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
	tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
	tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,
	tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,
	
	// RSA is slower on the server side but still widely used.
	tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
	tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
	tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,
	tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,
}
var (
	HTTP11    = "http/1.1"
	HTTP2     = "h2"
	ALPNProto = "acme-tls/1"
)

func main() {
	caFile := "certificates/ca.crt"
	caCert, errReadFile := readFile(caFile, "CA Cert")
	if errReadFile != nil {
		log.Fatalf("failed to read ca certificate %s: %v", caFile, errReadFile)
	}
	caCertPool, _ := x509.SystemCertPool()
	if caCertPool == nil {
		caCertPool = x509.NewCertPool()
	}
	if !caCertPool.AppendCertsFromPEM(caCert) {
		log.Fatalf("failed to append CA certificate %s", caFile)
	}
	
	serverCert, _ := getServerCert()
	tlsConfig := &tls.Config{
		MinVersion:       tls.VersionTLS13,
		MaxVersion:       tls.VersionTLS13,
		CipherSuites:     Ciphers,
		NextProtos:       []string{HTTP2, HTTP11, ALPNProto},
		CurvePreferences: []tls.CurveID{tls.CurveP521, tls.CurveP384, tls.CurveP256},
		Certificates:     []tls.Certificate{*serverCert},
		ClientAuth:       tls.VerifyClientCertIfGiven,
		Rand:             rand.Reader,
		RootCAs:          caCertPool,
		ClientCAs:        caCertPool,
	}
	
	tcpListener, _ := tls.Listen("tcp", ":8300", tlsConfig)
	
	s2 := &http2.Server{
		MaxHandlers:                  0,
		MaxConcurrentStreams:         0,
		MaxDecoderHeaderTableSize:    0,
		MaxEncoderHeaderTableSize:    0,
		MaxReadFrameSize:             0,
		PermitProhibitedCipherSuites: true,
		IdleTimeout:                  10 * time.Second,
		MaxUploadBufferPerConnection: 65535,
		MaxUploadBufferPerStream:     1,
		NewWriteScheduler:            nil,
		CountError:                   nil,
	}
	router := http.NewServeMux()
	router.HandleFunc("/test", func(w http.ResponseWriter, req *http.Request) {
		_, _ = fmt.Fprintf(w, "Welcome to the home page!")
	})
	server := &http.Server{
		Handler:           router,
		Addr:              fmt.Sprintf("%s:%d", "localhost", 8300),
		ReadHeaderTimeout: 120 * time.Second,
		WriteTimeout:      120 * time.Second,
		IdleTimeout:       120 * time.Second,
		ReadTimeout:       120 * time.Second,
		TLSConfig:         tlsConfig,
		MaxHeaderBytes:    1048576,
		ErrorLog:          log.New(os.Stderr, "", 0),
	}
	
	if err := http2.ConfigureServer(server, s2); err != nil {
		panic(err)
	}
	
	if err := server.Serve(tcpListener); err != nil && !errors.Is(err, http.ErrServerClosed) {
		panic(err)
	}
}

func getServerCert() (*tls.Certificate, error) {
	cert := "certificates/localhost.crt"
	
	certBytes, err := readFile(cert, "Cert")
	if err != nil {
		return nil, errors.Errorf("failed to read certificate %s: %v", cert, err)
	}
	
	key := "certificates/localhost.key"
	certKeyBytes, err := readFile(key, "Key")
	if err != nil {
		return nil, errors.Errorf("failed to read certificate key %s: %v", key, err)
	}
	
	serverCert, errCert := tls.X509KeyPair(certBytes, certKeyBytes)
	if errCert != nil {
		return nil, errors.Errorf("failed to load certificate %s and key %s: %v", cert, key, errCert)
	}
	
	return &serverCert, nil
}

func readFile(file, errorPrefix string) ([]byte, error) {
	file = strings.TrimSpace(file)
	if file == "" {
		return nil, errors.Errorf("%s file cannot be blank", errorPrefix)
	}
	
	osf, err := os.Stat(file)
	if err != nil {
		return nil, errors.Errorf("%s file not found %s: %v", errorPrefix, file, err)
	}
	
	if osf.IsDir() {
		return nil, errors.Errorf("%s file needs to specify a file in its path", errorPrefix)
	}
	if osf.Size() < 1 {
		return nil, errors.Errorf("%s file cannot be empty", errorPrefix)
	}
	
	bytes, err := os.ReadFile(file)
	if err != nil {
		return nil, errors.Errorf("failed to read %s file %s: %v", errorPrefix, file, err)
	}
	
	if len(bytes) < 1 {
		return nil, errors.Errorf("%s file %s is empty", errorPrefix, file)
	}
	
	return bytes, nil
}

mauri870 · 2023-08-02T22:27:53Z

I think it might have to do with the ciphers being offered by the client (browser in this case). If I try to access the page with firefox I also get the EOF.

If I try to offer a supported cipher there is no EOF error: openssl s_client -connect localhost:8300 -ciphersuites TLS_AES_128_GCM_SHA256 -prexit (TLSv1.3, Cipher is TLS_AES_128_GCM_SHA256)

go-aegian · 2023-08-03T12:08:20Z

If you add all ciphers still gets an EOF. What is Firefox, Chrome, Edge sending as cipher and what client cert they send?

var Ciphers = []uint16{
// TLS 1.3
tls.TLS_AES_128_GCM_SHA256,
tls.TLS_AES_256_GCM_SHA384,
tls.TLS_CHACHA20_POLY1305_SHA256,

// ECDSA is about 3 times faster than RSA on the server side.
tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,
tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,

// RSA is slower on the server side but still widely used.
tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,
tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,

// Added so all ciphers are available

tls.TLS_RSA_WITH_RC4_128_SHA,
tls.TLS_RSA_WITH_3DES_EDE_CBC_SHA,
tls.TLS_RSA_WITH_AES_128_CBC_SHA,
tls.TLS_RSA_WITH_AES_256_CBC_SHA,
tls.TLS_RSA_WITH_AES_128_CBC_SHA256,
tls.TLS_RSA_WITH_AES_128_GCM_SHA256,
tls.TLS_RSA_WITH_AES_256_GCM_SHA384,
tls.TLS_ECDHE_ECDSA_WITH_RC4_128_SHA,
tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
tls.TLS_ECDHE_RSA_WITH_RC4_128_SHA,
tls.TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,
tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,
tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,
tls.TLS_FALLBACK_SCSV,

}

dr2chase · 2023-08-03T22:07:17Z

@golang/security @neild
Not sure if this is a security question or a net question.

wangyuanwenGit · 2023-09-04T11:34:47Z

I also have the same problem occur.
I implemented a proxy server using net.http and also used terminal TLS, but I also got EOF.
And I captured the packet through wireshark, and what I saw was that the client actively closed the tcp connection (that is, sent a FIN), and then I thought that http.server just read an EOF.
This is very confusing. I can't determine what the problem is. If I access the same URL on different computers, I may not be able to get this EOF.
I have seen a lot of discussions about https EOF. Can you provide some ideas or solutions and reasons? Thank you.

rani-sharim · 2023-12-14T08:47:56Z

We have a similar symptom/issue, although a different scenario when we try to do mtls with tpmvscmgr (a virtual smartcard).
Although the browser prompts for certificate selection, the go server gets EOF when trying to read it.

I'm not sure if I should open a different issue for this, so I'll post reproduction steps here.

This happens on go versions 1.21.5 all the way back to 1.15 (at which point I stoped testing), Windows 11, Edge/Chrome/Firefox.

How to create a certificate on the key card:

1. Create TPM container :
tpmvscmgr.exe create /name myVSC /pin prompt /adminkey random /generate

2. Enroll certificate :
2.1    certreq -new
2.2   select inf model : smartcard.inf (see blow)
2.3 fill pin code from TPM
2.4 Save csr file generated

3. Generate a client certificate from this csr

4. Add it to the TMP:
4.1 certreq -accept certificate.crt
4.2 input pin code

Demo smartcard.inf

[Version]
Signature="$Windows NT$"
 
[NewRequest]
Subject = "CN=test OU=Organizational_Unit, O=Organization, L=City, S=State, C=Country"
KeySpec = 1
KeyLength = 2048
HashAlgorithm = SHA256
Exportable = FALSE
ProviderName = "Microsoft Base Smart Card Crypto Provider"
RequestType = PKCS10
KeyUsage = 0xa0

Go server code:

package main

import (
    "crypto/tls"
    "log"
    "net/http"
    "time"
)

func handleRequest(w http.ResponseWriter, r *http.Request) {
    w.WriteHeader(http.StatusOK)
    w.Write([]byte("Ok"))
}

func main() {
    tlsConfig := &tls.Config{
        ClientAuth: tls.RequireAnyClientCert,
        MinVersion: tls.VersionTLS13,
    }

    router := http.NewServeMux()
    router.HandleFunc("/", handleRequest)

    srv := &http.Server{
        Addr: ":8080",
        ReadTimeout:  60 * time.Second,
        WriteTimeout: 60 * time.Second,
        IdleTimeout:  60 * time.Second,
        ErrorLog:     log.Default(),
        TLSConfig:    tlsConfig,
        Handler:      router,
    }

    err := srv.ListenAndServeTLS("./localhost.pem", "./localhost-key.pem")
    if err != nil {
        panic(err)
    }
}

Edit: Seems like its exclusive to tls1.3. If I force degrade the protocol to tls1.2 it works as expcted.
Edit2: So after more investigating, i think its because chromium doesn't implement post handshake auth, so it's on the client side probably.

wangyuanwenGit · 2023-12-14T09:13:41Z

Oh, Thanks.
I may have found the cause of this problem, but of course it's just the cause of my problem, not necessarily the same as yours. But you can try to check it out.

First of all, the questions I have are:
One of my golang servers receives tls messages, but during the handshake phase a FIN is replied by the client after I reply to the client's clientHello message. The error printed in my server log is TLS: EOF. The key is that this does not necessarily happen. It does not appear on some computers, but it appears stably on some computers.

Cause of the problem:
After comparing the data packets on the normal computer and the abnormal computer, I found that in the Extension field, there were 10 items on the normal computer, but only 6 on the problem computer. I did not study the role of these fields in depth. After asking our operation and maintenance personnel to add these specified extensions to the certificate, the problem was indeed solved.
I'm still not sure why some computers add extensions to the same certificate and some don't. And why does the client not return a certificate error reply but directly close the session.
The above is my troubleshooting process, I hope it can help you.

Posted below are the extensions.
`
content:

extensions: 10 items
Extension (id-ce-authorityKeyIdentifier)
Extension (id-ce-subjectKeyIdentifier)
Extension (id-ce-subjectAltName)
Extension (id-ce-keyUsage)
Extension (id-ce-extKeyUsage)
Extension (id-ce-cRLDistributionPoints)
Extension (id-ce-certificatePolicies)
Extension (id-pe-authorityInfoAccess)
Extension (id-ce-basicConstraints)
Extension (SignedCertificateTimestampList)
`

go-aegian changed the title ~~http: TLS handshake error from 127.0.0.1:xxx: EOF~~ net/http: EOF during TLS handshake Aug 2, 2023

dr2chase added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Aug 3, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

net/http: EOF during TLS handshake #61721

net/http: EOF during TLS handshake #61721

go-aegian commented Aug 2, 2023

mauri870 commented Aug 2, 2023

mauri870 commented Aug 2, 2023 •

edited

go-aegian commented Aug 2, 2023

mauri870 commented Aug 2, 2023

go-aegian commented Aug 3, 2023

dr2chase commented Aug 3, 2023

wangyuanwenGit commented Sep 4, 2023

rani-sharim commented Dec 14, 2023 •

edited

wangyuanwenGit commented Dec 14, 2023

net/http: EOF during TLS handshake #61721

net/http: EOF during TLS handshake #61721

Comments

go-aegian commented Aug 2, 2023

mauri870 commented Aug 2, 2023

mauri870 commented Aug 2, 2023 • edited

go-aegian commented Aug 2, 2023

mauri870 commented Aug 2, 2023

go-aegian commented Aug 3, 2023

dr2chase commented Aug 3, 2023

wangyuanwenGit commented Sep 4, 2023

rani-sharim commented Dec 14, 2023 • edited

wangyuanwenGit commented Dec 14, 2023

mauri870 commented Aug 2, 2023 •

edited

rani-sharim commented Dec 14, 2023 •

edited