Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/cgo: goroutine freezes on stack overflow in C code #55919

Closed
begelundmuller opened this issue Sep 28, 2022 · 4 comments
Closed

cmd/cgo: goroutine freezes on stack overflow in C code #55919

begelundmuller opened this issue Sep 28, 2022 · 4 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@begelundmuller
Copy link

begelundmuller commented Sep 28, 2022

What version of Go are you using (go version)?

$ go version
go version go1.19.1 darwin/arm64

(Also tested on go version go1.18.3 darwin/amd64)

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/Users/benjamin/Library/Caches/go-build"
GOENV="/Users/benjamin/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/benjamin/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/benjamin/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/homebrew/Cellar/go/1.19.1/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/homebrew/Cellar/go/1.19.1/libexec/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.19.1"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/l9/q2cr0hld7038bfv6qz3dhz3w0000gn/T/go-build2278872038=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Run the following main.go using CGO_ENABLED=1 go run main.go:

package main

/*
#import <stdio.h>

int foo(int n) {
	if (n == 0) {
		return 1;
	}
	return foo(n - 1) + foo(n - 1);
}

void bar() {
	foo(100000); // causes stack overflow on the non-main thread
	printf("bar\n");
}
*/
import "C"
import (
	"sync"
)

func main() {
	n := 2
	wg := sync.WaitGroup{}
	for i := 0; i < n; i++ {
		wg.Add(1)
		go func() {
			defer wg.Done()

			// This succeeds on the goroutine that gets scheduled on the main thread,
			// but freezes on the goroutine that gets scheduled on a new thread
			C.bar()
		}()
	}
	wg.Wait()
}

What did you expect to see?

I expect the code to either print bar twice to the console or to crash with a stack overflow error.

What did you see instead?

It prints bar once and hangs indefinitely.

It appears that the goroutine that does not get scheduled on the main thread freezes (at 100% CPU utilization). This is just a guess, but it seems that the thread Go creates for the second goroutine gets a smaller stack size than the main thread, and that when it encounters a stack overflow, the error does not correctly propagate back to Go.

In terms of expected behaviour, I would expect cgo calls to show the same behaviour regardless of which goroutine they're called from. Failing that, I would expect that stack overflow errors in C code crash (not freeze) the process.

For context, we encountered this bug when using marcboeker/go-duckdb, which uses cgo to call DuckDB from Go. DuckDB can run into stack overflow errors on deeply nested queries. Crashes are fine in such cases, but silent freezes at 100% CPU utilization feel dangerous.

Let me know if there's any more information I can provide to help.

@dmitshur dmitshur changed the title cgo: goroutine freezes on stack overflow in C code cmd/cgo: goroutine freezes on stack overflow in C code Sep 28, 2022
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Sep 28, 2022
@dmitshur
Copy link
Contributor

CC @golang/compiler.

@dmitshur dmitshur added this to the Backlog milestone Sep 28, 2022
@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 28, 2022
@cherrymui
Copy link
Member

when it encounters a stack overflow, the error does not correctly propagate back to Go.

The C functions do not have stack bounds check, so when it overflows the stack it corrupts the memory next to the stack. The Go runtime is not able to detect/report a C stack overflow error.

it seems that the thread Go creates for the second goroutine gets a smaller stack size than the main thread

CL https://golang.org/cl/415915 might help. cc @fengyoulin

@cherrymui
Copy link
Member

Crashes are fine in such cases, but silent freezes at 100% CPU utilization feel dangerous.

If this is a concern for you, you probably want to build the C code with sanitizers or stack protection.

@aclements
Copy link
Member

To add to what @cherrymui said, the C code doesn't run on the Go stack at all, so the stack size of the goroutine does not matter. Overflowing a C stack is undefined behavior, so anything could happen, including freezes or silent memory corruption.

Closing as this is not a Go issue.

@golang golang locked and limited conversation to collaborators Sep 28, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

5 participants