Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

net/http: excessive mutex contention #3946

Closed
dvyukov opened this issue Aug 13, 2012 · 5 comments
Closed

net/http: excessive mutex contention #3946

dvyukov opened this issue Aug 13, 2012 · 5 comments
Labels
FrozenDueToAge

Comments

@dvyukov
Copy link
Member

dvyukov commented Aug 13, 2012 

The attached contention profile (collected with http://golang.org/cl/6443115 on
a parallel http load test) shows great amount of contention on
http.Transport.putIdleConn(), and the load test fails to scale beyond 4 cores.
I think we need try to reduce critical sections and see how it affects
performance/scalability.

Attachments:

  1. http_contention.svg (48386 bytes)
@remyoudompheng
Copy link
Contributor

Comment 1:

connectMethod.String() looks like a particularly heavy operation (calling URL.String and
strings.Join), could the attached patch be enough to solve the problem ?

Attachments:

  1. net_http_transport.patch (1207 bytes)

@dvyukov
Copy link
Member Author

dvyukov commented Aug 13, 2012

Comment 2:

http://golang.org/cl/6454142
I take your patch as a base, and added some more optimizations. I did not measure them
separately, but the resulting speedups look quite impressive (-40%).

@dvyukov
Copy link
Member Author

dvyukov commented Aug 13, 2012

Comment 3:

New contention profile after applying http://golang.org/cl/6454142

Attachments:

  1. cprof2.svg (63673 bytes)

@robpike
Copy link
Contributor

robpike commented Aug 17, 2012

Comment 4:

Labels changed: added priority-soon, removed priority-triage.

@dvyukov
Copy link
Member Author

dvyukov commented Aug 20, 2012

Comment 5:

This issue was closed by revision 20f6a8f.

Status changed to Fixed.

dvyukov added a commit that referenced this issue May 11, 2015
@dvyukov
[release-branch.go1] net/http: reduce mutex contention
17fed3c 
««« backport 43568ec5cf55
net/http: reduce mutex contention
benchmark                           old ns/op    new ns/op    delta
BenchmarkClientServerParallel          155909       154454   -0.93%
BenchmarkClientServerParallel-2         86012        82986   -3.52%
BenchmarkClientServerParallel-4         70211        55168  -21.43%
BenchmarkClientServerParallel-8         80755        47862  -40.73%
BenchmarkClientServerParallel-12        77753        51478  -33.79%
BenchmarkClientServerParallel-16        77920        50278  -35.47%
The benchmark is https://golang.org/cl/6441134
The machine is 2 x 4 HT cores (16 HW threads total).
Fixes #3946.
Now contention moves to net.pollServer.AddFD().

R=bradfitz
CC=bradfitz, dave, dsymonds, gobot, golang-dev, remyoudompheng
https://golang.org/cl/6454142

»»»
@golang golang locked and limited conversation to collaborators Jun 24, 2016
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@remyoudompheng @dvyukov @robpike @gopherbot