Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: race detector not working #52275

Closed
win-t opened this issue Apr 11, 2022 · 6 comments
Closed

net/http: race detector not working #52275

win-t opened this issue Apr 11, 2022 · 6 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. RaceDetector
Milestone

Comments

@win-t
Copy link

win-t commented Apr 11, 2022

What version of Go are you using (go version)?

$ go version
go version go1.18 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/win/.cache/go-build"
GOENV="/home/win/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/win/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/win/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.18"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2412875146=/tmp/go-build -gno-record-gcc-switches"

What did you do?

func main() {
	counter := 0
	panic(http.ListenAndServe(":8080", http.HandlerFunc(
		func(w http.ResponseWriter, r *http.Request) {
			counter++
			fmt.Fprintln(w, counter)
		},
	)))
}

and run it with go run -race main.go, then curl it twice

What did you expect to see?

race detector detects the race condition

What did you see instead?

race detector doesn't detect the race condition

I believe the go race detector will catch it even though I run curl sequentially because in the following code, the race detected do detect it correctly

func main() {
	counter := 0

	go func() {
		counter++
		fmt.Println(counter)
	}()

	time.Sleep(1 * time.Second)

	go func() {
		counter++
		fmt.Println(counter)
	}()

	time.Sleep(1 * time.Second)
}
@dmitshur
Copy link
Contributor

I'm not sure what guarantees the race detector makes about what it's definitely able to detect. Maybe not reporting a race when the curls were done sequentially is a feature, in that there was not actually a race (but could be if the requests arrived at the same time). Or maybe detecting such "potential races" could be a feature request.

It seems the following program also doesn't report a race (which likely doesn't happen if you wait enough seconds to be sure the first goroutine completed before providing stdin):

package main

import "fmt"

func main() {
	counter := 0

	go func() {
		counter++
		fmt.Println(counter)
	}()

	fmt.Scanln(new(string))

	go func() {
		counter++
		fmt.Println(counter)
	}()

	// Output (with -race):
	// 1
	// a
	// 2
}

CC @ianlancetaylor, @neild.

@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Apr 11, 2022
@dmitshur dmitshur added this to the Backlog milestone Apr 11, 2022
@ianlancetaylor
Copy link
Contributor

The race detector does make an effort to detect potential races. Usually when the race detector fails to report a race it is because there is an unexpected synchroniation/serialization in a library.

The later example above will indeed report a race if run as ./x < /dev/null or as echo hi | ./x. Otherwise I think it's being effectively synchronized by the runtime I/O code. That may also be what is happening in the earlier example, I'm not sure.

@ianlancetaylor
Copy link
Contributor

CC @dvyukov

@dvyukov
Copy link
Member

dvyukov commented Apr 19, 2022

All real races should be eventually detectable. In this case you play role of synchronization since you run curl sequentially ;)
If you write a stress test where the handler is invoked concurrently, the race should be detected eventually.

As Ian pointed, the false negative happens due to induced io synchronization:

src$ egrep "race\.(Acquire|Release)" syscall/*.go
syscall/syscall_unix.go:			race.Acquire(unsafe.Pointer(&ioSync))
syscall/syscall_unix.go:		race.ReleaseMerge(unsafe.Pointer(&ioSync))
syscall/syscall_unix.go:			race.Acquire(unsafe.Pointer(&ioSync))
syscall/syscall_unix.go:		race.ReleaseMerge(unsafe.Pointer(&ioSync))
syscall/syscall_unix.go:		race.ReleaseMerge(unsafe.Pointer(&ioSync))
syscall/syscall_windows.go:		race.Acquire(unsafe.Pointer(&ioSync))
syscall/syscall_windows.go:		race.ReleaseMerge(unsafe.Pointer(&ioSync))

Removing that will lead to false positives and we prefer to err on side of false negatives.

It may be possible to restructure http package to run the handler in unsynchronized goroutines. But not sure if it will work b/c the goroutine will still need to accept/read/receive before the handler and send after the handler.
Otherwise my vote for #WAI

@ianlancetaylor
Copy link
Contributor

Thanks, closing as there is nothing we can reasonably do here.

@win-t
Copy link
Author

win-t commented Apr 19, 2022

thanks for clarification

@golang golang locked and limited conversation to collaborators Apr 19, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. RaceDetector
Projects
None yet
Development

No branches or pull requests

5 participants