Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: go runtime GC panic #45401

Closed
eastmoutain opened this issue Apr 6, 2021 · 2 comments
Closed

runtime: go runtime GC panic #45401

eastmoutain opened this issue Apr 6, 2021 · 2 comments
Labels
FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.

Comments

@eastmoutain
Copy link

eastmoutain commented Apr 6, 2021

What version of Go are you using (go version)?

$ go version

I have tried go1.13.4, go1.13.15, go1.14, go1.14.15

Does this issue reproduce with the latest release?

my code depends on the third part code, go1.15 and above would cause compilation fail

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/thomas/.cache/go-build"
GOENV="/home/thomas/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/thomas/gopath/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/thomas/gopath"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="/usr/bin/gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build922420275=/tmp/go-build -gno-record-gcc-switches"

What did you do?

compile my project(a gvisor varietas container) with go, run the container for some hours.

go runtime GC will panic, I have caught two different paincs:
The first one: runtime/mgcwork.go:checknonempty() and checkempty()

func (b *workbuf) checknonempty() {
    if b.nobj == 0 {
        throw("workbuf is empty")
    }
}

and

func (b *workbuf) checkempty() {
    if b.nobj != 0 {
        throw("workbuf is not empty")
    }
}

the second: runtime/mgcsweep.go:sweep()

    nfreed := s.allocCount - nalloc
    if nalloc > s.allocCount {
        print("runtime: nelems=", s.nelems, " nalloc=", nalloc, " previous allocCount=", s.allocCount, " nfreed=", nfreed, "\n")
        throw("sweep increased allocation count")
    }

What did you expect to see?

It should not panic in go runtime library

What did you see instead?

panic frequently in go runtime GC

Trials

I added a FENCE instruction in both push() and pop() function in runtime/lfstack.go

func (head *lfstack) push(node *lfnode) {
   ...
    for {
        old := atomic.Load64((*uint64)(head))
        node.next = old
        atomic.Mfence()  // mfence instruction
        if atomic.Cas64((*uint64)(head), old, new) {
            break
        }
    }
}
func (head *lfstack) pop() unsafe.Pointer {
    for {
        old := atomic.Load64((*uint64)(head))
        if old == 0 {
            return nil
        }
        node := lfstackUnpack(old)
        atomic.Mfence()  // mfence instruction
        next := atomic.Load64(&node.next)
        if atomic.Cas64((*uint64)(head), old, next) {
            return unsafe.Pointer(node)
        }
    }
}

It makes no change to the result, but it runs more time that before.

@ALTree
Copy link
Member

ALTree commented Apr 6, 2021

This won't be easy to investigate without a reproducer. Two questions:

  1. Does your program (or any of its dependencies) use cgo? Does compiling and running it with GODEBUG=cgocheck=2 raise any error?
  2. Does compiling your program with -race and then running it print any race detector warnings?

Also you haven't pasted any of the compilation(??)/runtime errors you are getting. I'd suggest doing so.

@ALTree ALTree added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Apr 6, 2021
@ALTree ALTree changed the title go runtime GC panic runtime: go runtime GC panic Apr 6, 2021
@gopherbot
Copy link

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)

@golang golang locked and limited conversation to collaborators May 6, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests

3 participants