Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: Transport doesn't support specify Transfer-Encoding explicitly #33733

Closed
zema1 opened this issue Aug 20, 2019 · 1 comment
Closed

Comments

@zema1
Copy link

zema1 commented Aug 20, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12.6 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/zg/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/zg/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/zg/Development/gunkit/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/lg/cht6v2795qn9z07kmcl6h3c40000gn/T/go-build863622379=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I know http.Transport will use chunked Encoding automatically when Request has body and Request.Content-Length was set < 0. But this troubled me when I use net.http in my own penetration testing tool, which always need to build a custom chunked body. The code simply looks like the following snippet:

buf := &bytes.Buffer{}
ck := NewCustomChunkedWriter(buf)
ck.WriteString("test")
ck.Close()
req, _ = http.NewRequest("POST", "http://xx.com", buf)
req.Header.Set("Transfer-Encoding", "chunked")

Howover, it didn't work as expected. I tried to trace the roundtrip method and found that Transfer-Encoding is in a map named reqWriteExcludeHeader, which looks like a whitelist when writing request header.

And if I set req.Transfer-Encoding = []string{"chunked"}, the request body will be chunked twice! one from my customChunkedWriter, the other from the internal chunked writer.

Finally, I believe there is actually no way to make it. I think It's not a intended behavior. It should be great if it acts like Accept-Encoding.

If the Transport requests gzip on its own and gets a gzipped response, it's transparently decoded in the Response.Body. However, if the user explicitly requested gzip it is not automatically uncompressed.

Related issue: #28026

What did you expect to see?

If I send Transfer-Encoding explicitly, transport should keep the header and do nothing about the request body.

What did you see instead?

Described as above.

@bcmills bcmills changed the title net:http Transport doesn't support specify Transfer-Encoding explicitly net/http: Transport doesn't support specify Transfer-Encoding explicitly Aug 20, 2019
@bcmills
Copy link
Contributor

bcmills commented Aug 20, 2019

Duplicate of #28026

@bcmills bcmills marked this as a duplicate of #28026 Aug 20, 2019
@bcmills bcmills closed this as completed Aug 20, 2019
@golang golang locked and limited conversation to collaborators Aug 19, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants