Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: Memory usage increases on client side when server does not read request's body #43098

Open
AVoskoboinikov opened this issue Dec 9, 2020 · 1 comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@AVoskoboinikov
Copy link

AVoskoboinikov commented Dec 9, 2020

What version of Go are you using (go version)?

$ go version
go version go1.15.6 darwin/amd64

Does this issue reproduce with the latest release?

Yes, go1.15.6 is the latest release at the moment.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOOS="darwin"
GOPROXY="https://proxy.golang.org,direct"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/some/path=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Please, refer to this gist:

  • run http-server.go on one tab;
  • run http-client.go on another tab;
  • monitor output from http-client.go

What did you expect to see?

No errors in the output and amount of memory used by http-client.go does not increase.

What did you see instead?

  • Errors from http-client.go like:
    • write tcp [::1]:54091->[::1]:4040: write: protocol wrong type for socket
    • write tcp [::1]:54096->[::1]:4040: write: broken pipe
  • Amount of memory used by http-client.go increases:
http-client.go Output (errors omitted for brevity)
memo-sys-mb:    69 		 heap-inuse-mb:     1 		 heap-objects:  1436
memo-sys-mb:    71 		 heap-inuse-mb:    40 		 heap-objects:  4388
memo-sys-mb:   137 		 heap-inuse-mb:    57 		 heap-objects:  4871
...after some time
memo-sys-mb:   336 		 heap-inuse-mb:   194 		 heap-objects: 11196
memo-sys-mb:   402 		 heap-inuse-mb:   256 		 heap-objects: 18257
memo-sys-mb:   402 		 heap-inuse-mb:   305 		 heap-objects: 24123
...after some time
memo-sys-mb:  1130 		 heap-inuse-mb:   755 		 heap-objects: 43349
memo-sys-mb:  1196 		 heap-inuse-mb:   598 		 heap-objects: 22463
memo-sys-mb:  1196 		 heap-inuse-mb:   975 		 heap-objects: 70247

pprof heap snapshots

Bellow snapshots were taken from http-client.go:

Comparison between heap_start and heap_after_1g

One of "memory consumers" shown by go tool pprof -base heap_start -http localhost:1234 heap_after_1g

pprof Output
strings.Repeat
/usr/local/go/src/strings/strings.go
  Total:    483.42MB   483.42MB (flat, cum) 98.17%
    531            .          .           		panic("strings: Repeat count causes overflow") 
    532            .          .           	} 
    533            .          .            
    534            .          .           	n := len(s) * count 
    535            .          .           	var b Builder 
    536     483.42MB   483.42MB           	b.Grow(n) 
    537            .          .           	b.WriteString(s) 
    538            .          .           	for b.Len() < n { 
    539            .          .           		if b.Len() <= n/2 { 
    540            .          .           			b.WriteString(b.String()) 
    541            .          .           		} else { 

Some observations

The issue with memory disappears if either is done:

  • the server starts reading request's body:
// this must be added to the server
_, _ = ioutil.ReadAll(r.Body)
defer r.Body.Close()
  • the server stops writing response:
// this line must be removed from the server 
fmt.Fprintf(w, "{\"jsonrpc\": \"2.0\", \"error\": {\"code\": -32000, \"message\": \"Something went wrong...\"}, \"id\": \"1\"}")
  • the client stops closing connections:
// this line must be removed from the client
req.Close = true
@toothrot toothrot changed the title Memory usage increases on client side when server does not read request's body net/http: Memory usage increases on client side when server does not read request's body Dec 11, 2020
@toothrot
Copy link
Contributor

/cc @neild @bradfitz

@toothrot toothrot added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Dec 11, 2020
@toothrot toothrot added this to the Backlog milestone Dec 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

2 participants