Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: cannot set custom/special pseudo-headers when using HTTP/2 #32763

Open
niaow opened this issue Jun 25, 2019 · 17 comments
Open

net/http: cannot set custom/special pseudo-headers when using HTTP/2 #32763

niaow opened this issue Jun 25, 2019 · 17 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@niaow
Copy link

niaow commented Jun 25, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12.5 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jadenw/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/jadenw/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build648789117=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I attempted to implement websockets over HTTP/2, AKA RFC 8441, with net/http.

What did you expect to see?

Some mechanism of setting :protocol as defined in section 5 of RFC 8441.

What did you see instead?

There is no way to specify extra pseudo-headers in a request with net/http, at least that I can find in the header encoding code.

I am really not sure what the best way to solve this would be, or whether it is fully worthwhile. However, I would really like to be able to use a user-provided *http.Client for this, and this is the only thing I have not yet found a workaround for.

Clarification: pseudo-headers do not satisfy httpguts.ValidHeaderFieldName and therefore cannot be passed in with the regular headers

@niaow
Copy link
Author

niaow commented Jun 25, 2019

Also, a link to what I have currently of the client side of the HTTP/2 handshake: https://github.com/niaow/exp/blob/1ed5641ad76ca5761af3a652406bf49b7cac2ce3/ws/handshake.go#L248

@andybons andybons changed the title Setting Custom/Special Pseudo-Headers for HTTP/2 net/http: cannot set custom/special pseudo-headers when using HTTP/2 Jun 25, 2019
@andybons andybons added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 25, 2019
@andybons andybons added this to the Unplanned milestone Jun 25, 2019
@andybons
Copy link
Member

@bradfitz @mikioh

@mikedanese
Copy link
Contributor

This is somewhat related to #27244

@nhooyr
Copy link
Contributor

nhooyr commented Jul 25, 2019

@bradfitz how about we automatically convert HTTP/1.1 Upgrade requests into HTTP/2 if the peer supports it in *http.Transport? We'd have to perform an initial HTTP/2 request on the endpoint to check if it supports RFC 8441, cache that and then from there we can automatically decide whether or not to do HTTP/1.1 or HTTP/2 for subsequent upgrades.

Would mean my WebSocket library basically gets client side HTTP/2 WebSockets for free and httputil.ReverseProxy can proxy RFC 8441 connect requests.

@nhooyr
Copy link
Contributor

nhooyr commented Jul 25, 2019

For the server side

  1. We can just reuse http.Hijacker and return a net.Conn that tries its best to wrap around a HTTP/2 stream, unfortunately we'd have to allocate a buffo.ReadWriter every time but that shouldn't be too big a deal, most apps will want a bufio.ReadWriter when reading/writing to the stream anyway. We could add the Connection and Upgrade headers to RFC 8441 protocol connect requests too so that net/http hijacking servers all get this for free.
  2. Then once the above is implemented, we just need to always set the SETTINGS_ENABLE_CONNECT_PROTOCOL HTTP/2 setting parameter to indicate we support RFC 8441

@nhooyr
Copy link
Contributor

nhooyr commented Jul 25, 2019

There was previous discussion of this at #26937 (comment).

@odeke-em
Copy link
Member

@jadr2ddude, thank you for filing this issue! As @nhooyr mentioned, so if you are implementing Websockets I believe the first step that you want to do is to hijack the connection and then if you know you are using HTTP/2, you can use the golang.org/x/net/http2 package to create the framer and send over the respective headers by yourself. IMHO we shouldn't be changing net/http to make that change here. Please feel free to reopen though if I've missed something, thank you!

@mikedanese
Copy link
Contributor

@odeke-em how would you use the framer without having to rewrite or fork most of net/http? This needs both client and server support.

@nhooyr
Copy link
Contributor

nhooyr commented Oct 12, 2019

You can’t hijack the connection with http/2. So this needs to be reopened.

@bradfitz bradfitz reopened this Oct 12, 2019
@odeke-em
Copy link
Member

My apologies @jadr2ddude @mikedanese @nhooyr for the brainless close and non-responsiveness for the correction, I posted my response right before a long dinner. Thanks for reopening it, Brad!

@nhooyr
Copy link
Contributor

nhooyr commented Oct 13, 2019

@bradfitz I'd like to prepare some CLs for this in the coming weeks, could you please review my proposal above?

@nhooyr
Copy link
Contributor

nhooyr commented Feb 21, 2020

With @bradfitz leaving Google, who is going to be maintaining net/http? Who do I have to convince re my proposal? @andybons

@andybons
Copy link
Member

@fraenkel have any thoughts about this?

@fraenkel
Copy link
Contributor

Currently I see a couple of outstanding issues. Some we can fix, one I am not so sure about.
We currently have no way of configuring the http2 transport with any additional details which is starting to become more of an issue as we add specific http2 features that may need to be configured for a client, e.g., #31643.

The part I don't see how to handle is hijacking the http2 stream on the client side. Things should work correctly on the server side using the request.Body and ResponseWriter.

@nhooyr
Copy link
Contributor

nhooyr commented Aug 6, 2021

@fraenkel

The part I don't see how to handle is hijacking the http2 stream on the client side.

See my proposal #32763 (comment)

There's already an existing API to hijack HTTP/1.1 streams so we should be able to reuse it for HTTP/2.

@tv42
Copy link

tv42 commented Aug 6, 2021

Changing the meaning of Hijack from "take over the TCP/IP connection (so I can speak some completely different protocol)" to "take over one of the streams inside a HTTP/2 connection" sounds like a source of confusion. I'd much prefer a separate name for the new concept.

@thomas-kaltenbach
Copy link

thomas-kaltenbach commented Sep 30, 2021

Are there some plans to add the support in the next time?

We are trying to enable http/2 at our cloud foundry environment and noticed that the gorouter cannot handle http/2 websockets. I also opend issue cloudfoundry/routing-release#230.

Chrome as a client is also supporting it in the meantime https://www.chromestatus.com/feature/6251293127475200 this would lead into connection problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

9 participants