Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sync: random errors on sync.Once running on MacOS Mojave or High Serra #30453

Closed
ucirello opened this issue Feb 28, 2019 · 6 comments
Closed

sync: random errors on sync.Once running on MacOS Mojave or High Serra #30453

ucirello opened this issue Feb 28, 2019 · 6 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Go1.13

Comments

@ucirello
Copy link
Contributor

What version of Go are you using (go version)?

$ go version
go version go1.12 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/uldericofilho/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/uldericofilho/src/strongdm"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/_l/9yb78fmx0jzb8khycvrzw3qw0000gn/T/go-build415177971=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Upgraded from go1.11.5 to go1.12, recompile with same flags.
No OS changes between compiler upgrades.
XCode Version 10.1 (10B61) with all tools installed (the step Xcode forces you to take to open it the first time)
No source code changes between compiler upgrades.

What did you expect to see?

No panics.

What did you see instead?

fatal error: sync: inconsistent mutex state

goroutine 127 [running]:
runtime.throw(0x5867053, 0x1e)
	/usr/local/go/src/runtime/panic.go:617 +0x72 fp=0xc001103fa8 sp=0xc001103f78 pc=0x402f252
sync.throw(0x5867053, 0x1e)
	/usr/local/go/src/runtime/panic.go:603 +0x35 fp=0xc001103fc8 sp=0xc001103fa8 pc=0x402f1d5
sync.(*Mutex).Lock(0xc0004f0bf0)
	/usr/local/go/src/sync/mutex.go:121 +0x1ec fp=0xc001104008 sp=0xc001103fc8 pc=0x4073f9c
sync.(*Once).Do(0xc0004f0bf0, 0xc001104048)
	/usr/local/go/src/sync/once.go:40 +0x3b fp=0xc001104038 sp=0xc001104008 pc=0x407418b

The block of code that triggers the error is:

	var releaseOnce sync.Once
	release := func() { releaseOnce.Do(func() { lock.Release() }) }
	defer release()

(I could not produce a synthetic version of the code that triggers this)

Occasionally, it also panics with:

runtime: nelems=170 nalloc=86 previous allocCount=85 nfreed=65535
fatal error: sweep increased allocation count

goroutine 18 [running]:
runtime.throw(0x58677ea, 0x20)
    /usr/local/go/src/runtime/panic.go:617 +0x72 fp=0xc000068660 sp=0xc000068630 pc=0x402f422
runtime.(*mspan).sweep(0xb2a5190, 0xc0000a2000, 0x4059200)
    /usr/local/go/src/runtime/mgcsweep.go:329 +0x8da fp=0xc000068740 sp=0xc000068660 pc=0x402427a
runtime.sweepone(0x5909220)
    /usr/local/go/src/runtime/mgcsweep.go:136 +0x26e fp=0xc0000687a8 sp=0xc000068740 pc=0x402374e
runtime.bgsweep(0xc0000a2000)
    /usr/local/go/src/runtime/mgcsweep.go:73 +0xbb fp=0xc0000687d8 sp=0xc0000687a8 pc=0x402342b
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc0000687e0 sp=0xc0000687d8 pc=0x405e671
created by runtime.gcenable
    /usr/local/go/src/runtime/mgc.go:208 +0x58
@cherrymui
Copy link
Member

Interestingly, we also see a similar bug in another place. That one is not related to macOS.
Does CL https://go-review.googlesource.com/c/go/+/164118 fixes it?

@eliasnaur
Copy link
Contributor

I saw a similar error on darwin/arm64. I don't know if it is related.

https://build.golang.org/log/cea072d8d5ec55f28dad2e766457dc19e9a19b87

runtime: nelems=102 nalloc=15 previous allocCount=14 nfreed=65535
fatal error: sweep increased allocation count

goroutine 3 [running]:
runtime.throw(0x1029c72ca, 0x20)
	/private/var/folders/qq/qxn86k813bn9fjxydm095rxw0000gp/T/workdir-host-darwin-amd64-zenly-ios/go/src/runtime/panic.go:627 +0x4c fp=0x130054e10 sp=0x130054de0 pc=0x10254f2ec
runtime.(*mspan).sweep(0x10534f468, 0x102545900, 0x102576600)
	/private/var/folders/qq/qxn86k813bn9fjxydm095rxw0000gp/T/workdir-host-darwin-amd64-zenly-ios/go/src/runtime/mgcsweep.go:329 +0x8b0 fp=0x130054f30 sp=0x130054e10 pc=0x102544890
runtime.sweepone(0x102a23020)
	/private/var/folders/qq/qxn86k813bn9fjxydm095rxw0000gp/T/workdir-host-darwin-amd64-zenly-ios/go/src/runtime/mgcsweep.go:136 +0x2d4 fp=0x130054fa0 sp=0x130054f30 pc=0x102543cf4
runtime.bgsweep(0x13003c1c0)
	/private/var/folders/qq/qxn86k813bn9fjxydm095rxw0000gp/T/workdir-host-darwin-amd64-zenly-ios/go/src/runtime/mgcsweep.go:73 +0xd4 fp=0x130054fd0 sp=0x130054fa0 pc=0x102543934
runtime.goexit()
	/private/var/folders/qq/qxn86k813bn9fjxydm095rxw0000gp/T/workdir-host-darwin-amd64-zenly-ios/go/src/runtime/asm_arm64.s:1128 +0x4 fp=0x130054fd0 sp=0x130054fd0 pc=0x10257bc64
created by runtime.gcenable
	/private/var/folders/qq/qxn86k813bn9fjxydm095rxw0000gp/T/workdir-host-darwin-amd64-zenly-ios/go/src/runtime/mgc.go:208 +0x4c

...

@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Feb 28, 2019
@bcmills bcmills added this to the Go1.13 milestone Feb 28, 2019
@bcmills
Copy link
Contributor

bcmills commented Feb 28, 2019

@gopherbot, please backport to 1.12: this is an apparent regression in a core synchronization library.

@gopherbot gopherbot mentioned this issue Feb 28, 2019
Closed
@gopherbot
Copy link
Contributor

Backport issue(s) opened: #30470 (for 1.12).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/164118 mentions this issue: runtime: scan defer closure in stack scan

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/164629 mentions this issue: [release-branch.go1.12] runtime: scan defer closure in stack scan

gopherbot pushed a commit that referenced this issue Mar 1, 2019
@cherrymui @ianlancetaylor
[release-branch.go1.12] runtime: scan defer closure in stack scan
91fd14b 
With stack objects, when we scan the stack, it scans defers with
tracebackdefers, but it seems to me that tracebackdefers doesn't
include the func value itself, which could be a stack allocated
closure. Scan it explicitly.

Alternatively, we can change tracebackdefers to include the func
value, which in turn needs to change the type of stkframe.

Updates #30453.
Fixes #30470.

Change-Id: I55a6e43264d6952ab2fa5c638bebb89fdc410e2b
Reviewed-on: https://go-review.googlesource.com/c/164118
Reviewed-by: Keith Randall <khr@golang.org>
(cherry picked from commit 4f4c2a7)
Reviewed-on: https://go-review.googlesource.com/c/164629
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
@golang golang locked and limited conversation to collaborators Feb 29, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Go1.13
Development

No branches or pull requests
5 participants
@ucirello @eliasnaur @bcmills @gopherbot @cherrymui