Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime: fatal error: MHeap_AllocLocked - MSpan not free #17412

Closed
brknstrngz opened this issue Oct 12, 2016 · 4 comments
Closed

runtime: fatal error: MHeap_AllocLocked - MSpan not free #17412

brknstrngz opened this issue Oct 12, 2016 · 4 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone
Go1.8Maybe

Comments

@brknstrngz
Copy link

brknstrngz commented Oct 12, 2016 •
edited

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.7.1 freebsd/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64" GOBIN="" GOEXE="" GOHOSTARCH="amd64" GOHOSTOS="freebsd" GOOS="freebsd" GOPATH="/root/work/go" GORACE="" GOROOT="/usr/local/go" GOTOOLDIR="/usr/local/go/pkg/tool/freebsd_amd64" CC="cc" GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -gno-record-gcc-switches" CXX="clang++" CGO_ENABLED="1"
The OS is FreeBSD 11.0-RELEASE.

What did you do?

Sadly I cannot share the code, but it is a very simplistic HTTPI API client that calls multiple API endpoints in parallel. Responses are read into buffers obtained from a sync.Pool for later decoding.

What did you expect to see?

The API client run to completion.

What did you see instead?

fatal error: MHeap_AllocLocked - MSpan not free is thrown by the runtime. Not sure this is related to #15658.

gocrash.txt
@brknstrngz brknstrngz changed the title fatal error: MHeap_AllocLocked - MSpan not free runtime: fatal error: MHeap_AllocLocked - MSpan not free Oct 12, 2016
@bradfitz
Copy link
Contributor

Does your program have any data races?

@bradfitz bradfitz added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Oct 12, 2016
@bradfitz bradfitz modified the milestones: Unplanned, Go1.8Maybe Oct 12, 2016
@brknstrngz
Copy link
Author

brknstrngz commented Oct 12, 2016 •
edited

As it turns out, it does have a race. It is strange where it happens, though. We use singleflight to serialize access to our API login call.

WARNING: DATA RACE
Write at 0x00c4201ee270 by goroutine 243:
main.(_HTTPApiClient).doAuthenticate()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:360 +0xe34
main.(_HTTPApiClient).(main.doAuthenticate)-fm()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:303 +0x45
globalsign/hvpki/vendor/github.com/golang/groupcache/singleflight.(_Group).Do()
/path/to/go/src/globalsign/hvpki/vendor/github.com/golang/groupcache/singleflight/singleflight.go:56 +0x291
main.(_HTTPApiClient).authenticate()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:303 +0xa3
main.(*HTTPApiClient).requestResource()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:414 +0xdc1
main.runHTTPApiClient.func1()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:266 +0x4d

Previous read at 0x00c4201ee270 by goroutine 70:
runtime.convT2E()
/usr/local/go/src/runtime/iface.go:155 +0x0
main.(*HTTPApiClient).requestResource()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:382 +0x490
main.runHTTPApiClient.func1()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:266 +0x4d

Goroutine 243 (running) created at:
main.runHTTPApiClient()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:271 +0x358
main.main()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:114 +0x61f

Goroutine 70 (running) created at:
main.runHTTPApiClient()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:271 +0x358
main.main()
/path/to/go/src/mycompany/myproject/mysubproject/tools/httpapiclient/httpapiclient.go:114 +0x61f

@brknstrngz
Copy link
Author

I think I know what is happening, will report back shortly.

@brknstrngz
Copy link
Author

You were spot on Brad, we did have an unguarded variable access in the singleflight callback. It had flown under the radar for a year and a half or so and only manifested itself when we doubled our goroutine count. Sorry for the noise.

@golang golang locked and limited conversation to collaborators Oct 12, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
Go1.8Maybe
Development

No branches or pull requests
3 participants
@bradfitz @brknstrngz @gopherbot