Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: x/net/http2: Add support for client-side H2C upgrade flow #46249

Open
Gerg opened this issue May 18, 2021 · 7 comments
Open

proposal: x/net/http2: Add support for client-side H2C upgrade flow #46249

Gerg opened this issue May 18, 2021 · 7 comments

Comments

@Gerg
Copy link

Gerg commented May 18, 2021

What version of Go are you using (go version)?

$ go version
go version go1.16.4 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/pivotal/.cache/go-build"
GOENV="/home/pivotal/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/pivotal/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/pivotal/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.16"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.16/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.4"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1134153495=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Issue a request to a to a h2c server (examples in #45785) with the appropriate headers to trigger a h2c upgrade:

package main

import (
        "fmt"
        "log"
        "net/http"
)

func main() {
        transport := &http.Transport{}
        client := http.Client{Transport: transport}

        req, err := http.NewRequest("GET", "http://localhost:8080", nil)
        if err != nil {
                log.Fatalf("Error making new request: %+v\n", err)
        }

        req.Header.Set("Connection", "Upgrade, HTTP2-Settings")
        req.Header.Set("Upgrade", "h2c")
        req.Header.Set("HTTP2-Settings", "AAMAAABkAARAAAAAAAIAAAAA")

        resp, err := client.Do(req)
        if err != nil {
                log.Fatalf("Error sending request: %+v\n", err)
        }

        fmt.Printf("Response:\n\t%+v\n\n", resp)
}

What did you expect to see?

The HTTP/2 response to my request.

What did you see instead?

The 101 Switching Protocols response from the h2c upgrade process.

Response:
        &{Status:101 Switching Protocols StatusCode:101 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Connection:[Upgrade] Upgrade:[h2c]] Body:0xc00000e150 ContentLength:0 TransferEncoding:[] Close:false Uncompressed:false Trailer:map[] Request:0xc000142000 TLS:<nil>}

It's fine to receive the 101 response, but it does not seem possible for the caller to recover the HTTP/2 response for the original request.

Calling http2.Transport's NewClientConn with the underlying net.Conn will not work because the resulting ClientConn will not have a clientStream with id=1. According to RFC 7540, Section 3.2, Stream 1 will be used for the response:

The HTTP/1.1 request that is sent prior to upgrade is assigned a stream identifier of 1 (see Section 5.1.1) with default priority values (Section 5.3.5). Stream 1 is implicitly "half-closed" from the client toward the server (see Section 5.1), since the request is completed as an HTTP/1.1 request. After commencing the HTTP/2 connection, stream 1 is used for the response.

Stream 1 would typically be created as part of the http2.Transport roundTrip, however we do not go though a http2 round trip in the upgrade case. Since the client does not have Stream 1, the response will be dropped by the ClientConn once we send the HTTP/2 connection prefix and start the ClientConn's read loop (#25230 takes the stream id offset into account for future http2 round trips).

Even if Stream 1 were in place, we still do not have access to the response, since it appears the logic for draining the stream's response channel is only available as part of a round trip.

We are happy to submit a change that either implements the h2c client upgrade flow or provides hooks for it to be implemented externally (possibly in x/net/http2/h2c?). Any implementation guidance you can provide would be appreciated. We will include possible implementation details in a follow-up comment.

@seankhliao seankhliao changed the title net/http: Add support for client-side H2C upgrade flow x/net/http2: Add support for client-side H2C upgrade flow May 18, 2021
@seankhliao seankhliao changed the title x/net/http2: Add support for client-side H2C upgrade flow proposal: x/net/http2: Add support for client-side H2C upgrade flow May 18, 2021
@gopherbot gopherbot added this to the Proposal milestone May 18, 2021
@Gerg
Copy link
Author

Gerg commented May 18, 2021

Implementation Option

Based on our understanding, the ideal end state would be:

  1. The original caller is able to recover the HTTP/2 response to the upgrade request (Stream 1)
  2. The http.Transport has an idle persistConn for the h2c backend with a http2.Transport set in it's alt field that can be re-used for future requests to that backend
  3. The http2.Transport has a ClientConn for the backend in its clientConnPool that can be re-used for future requests to that backend

If we were to implement this as part of the net/http package, one option would be to mirror the ALPN upgrade flow established in http2.ConfigureTransports and triggered when dialing the connection. The lifecycle of this upgrade would be different, since it would take place during a http.Transport RoundTrip instead of when the connection is first established.

As described in the above issue, we would need seed the new http2.ClientConn with Stream 1. This could replace the logic implemented in #25230, or happen through other means.

The http2.Transport could also have a method for recovering the response from Stream 1, which can then be returned from the original http.Transport round trip.

Proof of Concept

Based on the above ideas, we produced a proof of concept that implements the h2c upgrade flow within net/http: https://github.com/Gerg/net.

The core logic is:

if pconn.alt != nil {
  // HTTP/2 path.
  t.setReqCanceler(req, nil) // not cancelable with CancelRequest
  resp, err = pconn.alt.RoundTrip(req)
} else {
  resp, err = pconn.roundTrip(treq)
  if err == nil && resp.isProtocolSwitch() {
    upgradeProto := resp.Header.Get("Upgrade")
    if upgradeFn, ok := t.upgradeNextProto[upgradeProto]; ok {
      t2 := upgradeFn(cm.targetAddr, pconn.conn)
      pconn.alt = t2
      resp, err = t2.completeUpgrade(req)
    }
  }
}

Source: https://github.com/Gerg/net/blob/c49a8dd603d5276e2ce2f026762f3bb5443b9f5a/http/transport.go#L538-L545

We are happy to build out alternative implementations based on y'alls feedback and knowledge.

@seankhliao
Copy link
Member

cc @bradfitz @tombergan @empijei

@ianlancetaylor ianlancetaylor added this to Incoming in Proposals (old) May 19, 2021
@networkimprov
Copy link

cc @fraenkel

@dprotaso
Copy link

Hey @neild - would any recent golang changes (since the issue was filed years ago) now allow clients to perform the h2c upgrade flow?

@dprotaso
Copy link

dprotaso commented Oct 13, 2023

A quick investigation an I have something sorta working - though something is wrong below and I'm getting connection error: PROTOCOL_ERROR - oddly the response always returns

But technically this makes two requests not one

// You can edit this code!
// Click here and start typing.
package main

import (
	"bufio"
	"context"
	"fmt"
	"net"
	"net/http"
	"net/http/httputil"
	"time"

	"golang.org/x/net/http2"
	"golang.org/x/net/http2/h2c"
)

func main() {
	go runH2CServer()
	time.Sleep(1)
	doClient()

}

func runH2CServer() {
	handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		if r.ProtoMajor != 2 && r.Header.Get("Upgrade") != "h2c" {
			w.WriteHeader(http.StatusBadRequest)
			fmt.Fprint(w, "Expected h2c request")
			return
		}

		w.Write([]byte("hi"))
	})
	h2c := &http.Server{
		Addr:    ":3001",
		Handler: h2c.NewHandler(handler, &http2.Server{}),
	}
	fmt.Printf("Starting server, listening on port %s (h2c)\n", h2c.Addr)
	h2c.ListenAndServe()
}

func doClient() {
	dialer := &net.Dialer{}

	conn, err := dialer.DialContext(context.Background(), "tcp", "localhost:3001")
	if err != nil {
		panic(err)
	}
	defer conn.Close()
	bw := bufio.NewWriter(conn)
	br := bufio.NewReader(conn)

	req, _ := http.NewRequest("GET", "http://localhost:3001", nil)
	req.Header.Set("Connection", "Upgrade, HTTP2-Settings")
	req.Header.Set("Upgrade", "h2c")
	req.Header.Set("HTTP2-Settings", "AAMAAABkAARAAAAAAAIAAAAA")

	if err := req.Write(bw); err != nil {
		panic(err)
	}
	if err := bw.Flush(); err != nil {
		panic(err)
	}

	resp, err := http.ReadResponse(br, req)
	if err != nil {
		panic(err)
	}
	defer resp.Body.Close()

	bytes, err := httputil.DumpResponse(resp, false)
	if err != nil {
		panic(err)
	}
	fmt.Printf("Response:\n%v\n", string(bytes))

	transport := &http2.Transport{}
	h2cConn, err := transport.NewClientConn(conn)
	if err != nil {
		panic(err)
	}
	defer h2cConn.Close()

	req, _ = http.NewRequest("GET", "http://localhost:3001", nil)
	resp, err = h2cConn.RoundTrip(req)
	if err != nil {
		panic(err)
	}
	defer resp.Body.Close()
	bytes, err = httputil.DumpResponse(resp, true)
	if err != nil {
		panic(err)
	}
	fmt.Printf("Response:\n%v\n", string(bytes))
}

@dprotaso
Copy link

dprotaso commented Oct 13, 2023

I updated the above example (https://go.dev/play/p/vPEfkUEXolM) - I wasn't closing the h2c connection prior to closing the tcp connection.

Though like I stated - it actually makes two requests so overall this isn't very ideal :/

@irbekrm
Copy link
Contributor

irbekrm commented Oct 16, 2023

Just a note that the hc2 upgrade mechanism that this proposal relies on has since been deprecated https://datatracker.ietf.org/doc/html/rfc9113#:~:text=The%20%22h2c%22%20string,%C2%B6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Incoming
Development

No branches or pull requests

6 participants