net/http: http2 with header `connection:close` causes the failure of rewinding body after connection loss #62450

leizongmin · 2023-09-05T08:59:02Z

What version of Go are you using (`go version`)?

$ go version
go version go1.20.4 darwin/amd64

Does this issue reproduce with the latest release?

Run the following code:

package main

import (
	"bytes"
	"fmt"
	"io"
	"log"
	"net/http"
	"sync"
)

func main() {
	// condition 1: request a http2 server
	const u = "https://http2.github.io/"
	const n = 30

	wg := sync.WaitGroup{}
	wg.Add(n)
	counter := 0

	makeRequest := func(u string) (*http.Response, error) {
		// condition 2: body is not nil
		body := io.NopCloser(bytes.NewBuffer([]byte("hello")))
		req, err := http.NewRequest("POST", u, body)
		if err != nil {
			return nil, err
		}

		// condition 3: add header `connection: close`
		req.Header.Set("connection", "close")

		res, err := http.DefaultClient.Do(req)
		return res, err
	}

	// condition 4: concurrent multiple requests
	for i := 0; i < n; i++ {
		go func(i int) {
			defer wg.Done()
			res, err := makeRequest(fmt.Sprintf("%s?i=%d", u, i))
			if err != nil {
				log.Println(err)
				counter++
				return
			}
			defer res.Body.Close()
		}(i)
	}

	wg.Wait()
	log.Println("total", n, "error", counter)
}

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env

What did you do?

What did you expect to see?

2023/09/05 16:52:40 total 30 error 0

What did you see instead?

2023/09/05 16:52:40 Post "https://http2.github.io/?i=29": net/http: cannot rewind body after connection loss
2023/09/05 16:52:40 Post "https://http2.github.io/?i=8": net/http: cannot rewind body after connection loss
2023/09/05 16:52:40 Post "https://http2.github.io/?i=16": net/http: cannot rewind body after connection loss
2023/09/05 16:52:40 Post "https://http2.github.io/?i=10": net/http: cannot rewind body after connection loss
2023/09/05 16:52:40 Post "https://http2.github.io/?i=5": net/http: cannot rewind body after connection loss
2023/09/05 16:52:40 Post "https://http2.github.io/?i=20": net/http: cannot rewind body after connection loss
2023/09/05 16:52:40 total 30 error 6

The text was updated successfully, but these errors were encountered:

panjf2000 · 2023-09-05T10:40:34Z

Change body := io.NopCloser(bytes.NewBuffer([]byte("hello"))) to body := bytes.NewBuffer([]byte("hello"))

The body can only be replayed after a connection loss when the body passed to net.NewRequest is one of the type *bytes.Buffer, *bytes.Reader, or *strings.Reader.

panjf2000 · 2023-09-05T14:16:51Z

Duplicate of #62453, and that issue is more general for this kind of problem, so move the discussion to #62453.

leizongmin · 2023-09-07T02:51:57Z

@panjf2000 I think this issue is a bit different from #62453's situation.
The main problem here is that when the request header contains connection: close, since the http/2 connection is multiplexed, the server will close the connection after processing the first request. The error errCannotRewind is returned when there are other requests that have not sent the body.

According to the relevant description of RFC 7540:

8.1.2.2. Connection-Specific Header Fields
HTTP/2 does not use the Connection header field to indicate connection-specific header fields; in this protocol, connection-specific metadata is conveyed by other means. An endpoint MUST NOT generate an HTTP/2 message containing connection-specific header fields; any message containing connection-specific header fields MUST be treated as malformed (Section 8.1.2.6).

The only exception to this is the TE header field, which MAY be present in an HTTP/2 request; when it is, it MUST NOT contain any value other than "trailers".

This means that an intermediary transforming an HTTP/1.x message to HTTP/2 will need to remove any header fields nominated by the Connection header field, along with the Connection header field itself. Such intermediaries SHOULD also remove other connection-specific header fields, such as Keep-Alive, Proxy-Connection, Transfer-Encoding, and Upgrade, even if they are not nominated by the Connection header field.

Note: HTTP/2 purposefully does not support upgrade to another protocol. The handshake methods described in Section 3 are believed sufficient to negotiate the use of alternative protocols.

My question is, when the http client sends a request using the http/2 protocol, should the connection request header be removed?

panjf2000 · 2023-09-11T14:56:57Z

OK, then we need to change the issue title

panjf2000 · 2023-09-11T15:55:44Z

I just went through the code and found this:

go/src/net/http/h2_bundle.go

Lines 6412 to 6423 in 9af74e7

    
           // "Connection" headers aren't allowed in HTTP/2 (RFC 7540, 8.1.2.2), 
        
           // but respect "Connection" == "close" to mean sending a GOAWAY and tearing 
        
           // down the TCP connection when idle, like we do for HTTP/1. 
        
           // TODO: remove more Connection-specific header fields here, in addition 
        
           // to "Connection". 
        
           if _, ok := rws.snapHeader["Connection"]; ok { 
        
           	v := rws.snapHeader.Get("Connection") 
        
           	delete(rws.snapHeader, "Connection") 
        
           	if v == "close" { 
        
           		rws.conn.startGracefulShutdown() 
        
           	} 
        
           }

It turns out this behavior is expected, therefore, this is the end of the discussion.

panjf2000 mentioned this issue Sep 5, 2023

net/http2: Transport received Server's graceful shutdown GOAWAY] after Request.Body was written; define Request.GetBody to avoid this error #62453

Closed

panjf2000 closed this as completed Sep 5, 2023

panjf2000 changed the title ~~net/http: cannot rewind body after connection loss~~ net/http: http2 with header connection:close causes the failure of rewinding body after connection loss Sep 11, 2023

panjf2000 reopened this Sep 11, 2023

panjf2000 closed this as completed Sep 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

net/http: http2 with header `connection:close` causes the failure of rewinding body after connection loss #62450

net/http: http2 with header `connection:close` causes the failure of rewinding body after connection loss #62450

leizongmin commented Sep 5, 2023 •

edited

panjf2000 commented Sep 5, 2023

panjf2000 commented Sep 5, 2023

leizongmin commented Sep 7, 2023

panjf2000 commented Sep 11, 2023 •

edited

panjf2000 commented Sep 11, 2023

net/http: http2 with header connection:close causes the failure of rewinding body after connection loss #62450

net/http: http2 with header connection:close causes the failure of rewinding body after connection loss #62450

Comments

leizongmin commented Sep 5, 2023 • edited

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

panjf2000 commented Sep 5, 2023

panjf2000 commented Sep 5, 2023

leizongmin commented Sep 7, 2023

panjf2000 commented Sep 11, 2023 • edited

panjf2000 commented Sep 11, 2023

net/http: http2 with header `connection:close` causes the failure of rewinding body after connection loss #62450

net/http: http2 with header `connection:close` causes the failure of rewinding body after connection loss #62450

leizongmin commented Sep 5, 2023 •

edited

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

panjf2000 commented Sep 11, 2023 •

edited