Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

crypto/tls: outBuf of Conn costs too much memory for large number of idle connections #42035

Closed
cch123 opened this issue Oct 17, 2020 · 4 comments
Closed

crypto/tls: outBuf of Conn costs too much memory for large number of idle connections #42035

cch123 opened this issue Oct 17, 2020 · 4 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Backlog

Comments

@cch123
Copy link
Contributor

cch123 commented Oct 17, 2020 •
edited

What version of Go are you using (go version)?

$ go version
1.13

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env

What did you do?

We are opening TLS on our online mesh project, which runs in a memory-limited situation. Currently, the TLS costs too much memory in some special apps which have about 100,000 connections

It can be calculated easily: 100,000 * max size of tlsConn.outBuf(16KB + some extra bytes) ≈ 1.6 GB(after go 1.12,), which can cause the process to OOM.

Most of the connections are idle, but still, use much memory because of this outBuf

What did you expect to see?

Don't use so much memory for massive idle tls connections

What did you see instead?

the process is killed by os because of OOM
@cch123 cch123 changed the title crypto.tls: outBuf cost too much memory for large number of idle connections crypto.tls: outBuf costs too much memory for large number of idle connections Oct 17, 2020
@cch123 cch123 changed the title crypto.tls: outBuf costs too much memory for large number of idle connections crypto.tls: outBuf of Conn costs too much memory for large number of idle connections Oct 17, 2020
@cch123 cch123 changed the title crypto.tls: outBuf of Conn costs too much memory for large number of idle connections crypto/tls: outBuf of Conn costs too much memory for large number of idle connections Oct 17, 2020
@cch123 cch123 mentioned this issue Oct 17, 2020
Closed
@gopherbot
Copy link

Change https://golang.org/cl/263277 mentions this issue: crypto/tls: reuse the tls Conn's outBuf to reduce memory cost on large number of tls connections

@cagedmantis cagedmantis added this to the Backlog milestone Oct 19, 2020
@cagedmantis cagedmantis added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Oct 19, 2020
@cagedmantis
Copy link
Contributor

/cc @FiloSottile

@taoyuanyuan
Copy link
Contributor

#28654

@gopherbot
Copy link

Change https://golang.org/cl/267957 mentions this issue: crypto/tls: pool Conn's outBuf to reduce memory cost of idle connections

@FiloSottile FiloSottile mentioned this issue Jan 8, 2021
Open
@golang golang locked and limited conversation to collaborators Nov 9, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Backlog
4 participants
@cagedmantis @cch123 @taoyuanyuan @gopherbot