Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crypto/tls: lots of small objects allocations on .Read when using HTTP1.1 #58249

Closed
Jorropo opened this issue Feb 2, 2023 · 4 comments
Closed
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Performance
Milestone

Comments

@Jorropo
Copy link
Member

Jorropo commented Feb 2, 2023

What version of Go are you using (go version)?

$ go version
go version devel go1.21-56a14ad4bc Wed Feb 1 10:06:08 2023 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes (go1.20)

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/hugo/.cache/go-build"
GOENV="/home/hugo/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/hugo/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/hugo/go"
GOPRIVATE=""
GOPROXY="direct"
GOROOT="/home/hugo/k/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/hugo/k/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="devel go1.21-56a14ad4bc Wed Feb 1 10:06:08 2023 +0000"
GCCGO="gccgo"
GOAMD64="v3"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build992780592=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I have a program which downloads at 2.5Gbit/s from ~12 stream concurrently and runs GC more often than I like.
After some profiling I see:
profile001

ROUTINE ======================== crypto/tls.(*Conn).readFromUntil in /home/hugo/k/go/src/crypto/tls/conn.go
   2274297    2274449 (flat, cum) 49.80% of Total
         .          .    801:func (c *Conn) readFromUntil(r io.Reader, n int) error {
         .          .    802:	if c.rawInput.Len() >= n {
         .          .    803:		return nil
         .          .    804:	}
         .          .    805:	needs := n - c.rawInput.Len()
         .          .    806:	// There might be extra input waiting on the wire. Make a best effort
         .          .    807:	// attempt to fetch it so that it can be used in (*Conn).Read to
         .          .    808:	// "predict" closeNotify alerts.
         .        152    809:	c.rawInput.Grow(needs + bytes.MinRead)
   2274297    2274297    810:	_, err := c.rawInput.ReadFrom(&atLeastReader{r, int64(needs)})
         .          .    811:	return err
         .          .    812:}

What did you expect to see?

HTTP1 is a simple protocol, I expect calls to .Read to be allocation free (after buffers have been grown). Either work inplace with the buffer passed in, or by caching buffers.

What did you see instead?

The TLS implementation seems to allocate this atLeastReader pointer very often (on each Read ?).

@gopherbot
Copy link

Change https://go.dev/cl/464835 mentions this issue: crypto/tls: store value atLeastReader in tls.Conn

@mknyszek mknyszek added Performance NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Feb 2, 2023
@mknyszek mknyszek added this to the Backlog milestone Feb 2, 2023
@mknyszek
Copy link
Contributor

mknyszek commented Feb 2, 2023

CC @golang/security

@AlexanderYastrebov
Copy link
Contributor

Seems to be a duplicate of #53142 (with a similar proposed fix https://go-review.googlesource.com/c/go/+/409334)

@Jorropo
Copy link
Member Author

Jorropo commented Feb 3, 2023

Duplicate

@Jorropo Jorropo closed this as not planned Won't fix, can't repro, duplicate, stale Feb 3, 2023
@golang golang locked and limited conversation to collaborators Feb 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Performance
Projects
None yet
Development

No branches or pull requests

4 participants