runtime/race: adding support for the race detector on other platforms #19273

owenwaller · 2017-02-24T17:54:46Z

Hi Everyone,

Following on from this discussion on the golang-nuts list, about supporting the race detector on ARM hardware, I'm opening this issue to track the suggestion.

The ThreadSanitizer that the Go race detector is based on supports x86_64, aarch64 (i.e.Arm8 64-bit), mips64 and powerpc64. Currently Go only supports the race detector on x86_64 hardware.

Would it be possible for a future release of Go to support the race detector on the other platforms - aarch64, mips64 and powerpc64.

Thanks

Owen

ianlancetaylor · 2017-02-24T18:09:32Z

CC @dvyukov

bradfitz · 2017-06-07T21:38:27Z

No objections, if somebody does the work.

krytarowski · 2017-12-05T19:36:24Z

I've added NetBSD/amd64 support in LLVM TSan. This could be integrated with golang CI infrastructure.

https://blog.netbsd.org/tnf/entry/the_llvm_thread_sanitizer_has

bradfitz · 2017-12-05T19:39:19Z

Note that we check in binary blobs to the Go repo:

bradfitz@gdev:~/go$ find . | grep syso$
./src/runtime/race/race_linux_amd64.syso
./src/runtime/race/race_darwin_amd64.syso
./src/runtime/race/race_windows_amd64.syso
./src/runtime/race/race_freebsd_amd64.syso

They are built using:

https://github.com/golang/build/tree/master/cmd/racebuild

krytarowski · 2017-12-05T20:55:45Z

I've repeated the steps for building race_netbsd_amd64.syso.

$ file ./src/runtime/race/race_netbsd_amd64.syso
./src/runtime/race/race_netbsd_amd64.syso: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped

My golang patch:

diff --git a/src/cmd/go/internal/work/init.go b/src/cmd/go/internal/work/init.go
index 7f894f5c6d..1df9963521 100644
--- a/src/cmd/go/internal/work/init.go
+++ b/src/cmd/go/internal/work/init.go
@@ -43,8 +43,8 @@ func instrumentInit() {
 		fmt.Fprintf(os.Stderr, "-msan is not supported on %s/%s\n", cfg.Goos, cfg.Goarch)
 		os.Exit(2)
 	}
-	if cfg.Goarch != "amd64" || cfg.Goos != "linux" && cfg.Goos != "freebsd" && cfg.Goos != "darwin" && cfg.Goos != "windows" {
-		fmt.Fprintf(os.Stderr, "go %s: -race and -msan are only supported on linux/amd64, freebsd/amd64, darwin/amd64 and windows/amd64\n", flag.Args()[0])
+	if cfg.Goarch != "amd64" || cfg.Goos != "linux" && cfg.Goos != "freebsd" && cfg.Goos != "netbsd" && cfg.Goos != "darwin" && cfg.Goos != "windows" {
+		fmt.Fprintf(os.Stderr, "go %s: -race and -msan are only supported on linux/amd64, freebsd/amd64, netbsd/amd64, darwin/amd64 and windows/amd64\n", flag.Args()[0])
 		os.Exit(2)
 	}
 
diff --git a/src/race.bash b/src/race.bash
index adf2297c2f..84b8df24c7 100755
--- a/src/race.bash
+++ b/src/race.bash
@@ -9,7 +9,7 @@
 set -e
 
 function usage {
-	echo 'race detector is only supported on linux/amd64, freebsd/amd64 and darwin/amd64' 1>&2
+	echo 'race detector is only supported on linux/amd64, freebsd/amd64, netbsd/amd64 and darwin/amd64' 1>&2
 	exit 1
 }
 
@@ -30,6 +30,11 @@ case $(uname) in
 		usage
 	fi
 	;;
+"NetBSD")
+	if [ $(uname -m) != "amd64" ]; then
+		usage
+	fi
+	;;
 *)
 	usage
 	;;

But I'm getting the following error when attempting to use it:

##### Testing packages.
# compress/bzip2 (testmain)
runtime/race(.text): relocation target malloc not defined
runtime/race(.text): relocation target malloc not defined
runtime/race(.text): relocation target free not defined
runtime/race(.text): relocation target free not defined
runtime/race(.text): relocation target malloc not defined
runtime/race(.text): relocation target malloc not defined
runtime/race(.text): relocation target malloc not defined
runtime/race(.text): relocation target malloc not defined
runtime/race(.text): relocation target malloc not defined
runtime/race(.text): relocation target malloc not defined
runtime/race(.text): relocation target malloc not defined
runtime/race(.text): relocation target malloc not defined
runtime/race(.text): relocation target environ not defined
runtime/race(.text): relocation target malloc not defined
runtime/race(.text): relocation target setrlimit not defined
runtime/race(.text): relocation target syscall not defined
runtime/race(.text): relocation target _lwp_self not defined
runtime/race(.text): relocation target syscall not defined
runtime/race(.text): relocation target __errno not defined
runtime/race(.text): relocation target exit not defined
runtime/race(.text): relocation target __libc_thr_yield not defined
/tmp/test/go/pkg/tool/netbsd_amd64/link: too many errors

Investigating.

krytarowski · 2017-12-05T21:12:52Z

@bradfitz other .syso files seem to have similar symbols undefined. Are there suggestions how to address it?

bradfitz · 2017-12-05T21:26:18Z

I have no clue. I've never built these binaries before, nor have I worked on the race detector.

Maybe @ianlancetaylor or @dvyukov know.

ianlancetaylor · 2017-12-05T22:44:15Z

Most likely you need to edit the build constraints in the flie runtime/race/race.go so that your system is recognized.

krytarowski · 2017-12-05T23:09:40Z

This has improved the situation. I'm now down to unrecognized __ps_strings.

runtime/race(.text): relocation target __ps_strings not defined
runtime/race(.text): relocation target __ps_strings not defined
runtime/race(.text): relocation target __ps_strings not defined
runtime/race(.text): undefined: "__ps_strings"
runtime/race(.text): undefined: "__ps_strings"
runtime/race(.text): undefined: "__ps_strings"

I'm researching how to fix it.

krytarowski · 2017-12-05T23:21:17Z

http://www.netbsd.org/~kamil/llvm/go-tsan-2017-12-06.txt

Passed 340 of 347 tests (97.98%, 0+, 7-)
0 expected failures (0 has not fail)

krytarowski · 2017-12-05T23:43:59Z

http://www.netbsd.org/~kamil/llvm/0001-Add-initial-TSan-NetBSD-support.patch

@bradfitz feel free to merge this with HEAD. (my CLA is signed for Google projects)

My .syso file: http://www.netbsd.org/~kamil/llvm/race_netbsd_amd64.syso

Feel free to regenerate it from scratch or take this one as is and commit to the golang repository.

I cannot focus on the racy tests soon, as I'm burdened with MSan now.

bradfitz · 2017-12-05T23:48:02Z

@krytarowski, the tree is closed right now for release. Only bug fixes can go in, not features.
See https://golang.org/wiki/Go-Release-Cycle (Unlike NetBSD, we intentionally do not have a development branch open during release freezes to encourage people to do boring stuff instead of just hacking on fun stuff)

We can merge fun things in February.

Also, we'd need to generate the syso ourselves (not shipping a random binary off the network), so you'd first need to send a change to x/build/cmd/racebuild

See https://golang.org/doc/contribute.html for how to submit a git patch to Gerrit. We only do it for people in very rare circumstance.

krytarowski · 2017-12-05T23:53:02Z

OK, thanks. In February hopefully there will be an option to get MSan aboard.

ceseo · 2017-12-06T15:35:21Z

FYI, I'm working on adding the necessary changes in LLVM TSan so we can get the race detectors on ppc64le.

(edit: sorry, little endian only)

krytarowski · 2017-12-06T17:03:38Z

Great to see ppc64le development around. FreeBSD is improving their POWER8 support lately.. I would love to see it with NetBSD as well (but hw price isn't easily reachable for a hobbyist).

ceseo · 2017-12-06T20:34:30Z

@krytarowski if you need machine access, you may try a POWER8 VM in one of our OpenPOWER Mini Clouds, like this.

krytarowski · 2017-12-08T04:43:09Z

Personally I prefer to use a real hardware when porting the kernel and userland. But thanks!

gopherbot · 2018-05-11T21:38:11Z

Change https://golang.org/cl/112881 mentions this issue: cmd/racebuild: add linux/ppc64le

gopherbot · 2018-05-11T21:38:15Z

Change https://golang.org/cl/112880 mentions this issue: cmd/racebuild: add netbsd/amd64

Updates golang/go#24354. Updates golang/go#19273. Updates golang/go#24322. Change-Id: Ia67fde51d7698ca94d86c4697fd153a551a8ceee Reviewed-on: https://go-review.googlesource.com/112880 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Updates golang/go#19273. Updates golang/go#24354. Updates golang/go#23731. Change-Id: Iad8870b265e7e3b56b5219d3367ccef70dcc0679 Reviewed-on: https://go-review.googlesource.com/112881 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

gopherbot · 2018-07-10T23:14:45Z

Change https://golang.org/cl/99835 mentions this issue: runtime/cgo: Add initial NetBSD Thread Sanitizer support

Recognize NetBSD in: - go/internal/work/init.go - race.bash - runtime/race/race.go Add __ps_strings symbol in runtime/cgo/netbsd.go as this is used internally in the TSan library for NetBSD and used for ReExec(). Tested on NetBSD/amd64 v. 8.99.12. Around 98% tests are passing for the ./race.bash target. Updates #19273 Change-Id: Ic0e48d2fb159a7868aab5e17156eeaca1225e513 GitHub-Last-Rev: d6e0827 GitHub-Pull-Request: #24322 Reviewed-on: https://go-review.googlesource.com/99835 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

ianlancetaylor changed the title ~~Adding support for the race detector on other platforms~~ runtime/race: adding support for the race detector on other platforms Feb 24, 2017

bradfitz added this to the Go1.9Maybe milestone Feb 24, 2017

bradfitz added the help wanted label Feb 24, 2017

dvyukov added the RaceDetector label Feb 24, 2017

bradfitz modified the milestones: Go1.10, Go1.9Maybe Jun 7, 2017

bradfitz added the FeatureRequest label Jun 7, 2017

rsc modified the milestones: Go1.10, Unplanned Nov 29, 2017

bradfitz mentioned this issue Dec 5, 2017

x/build: update NetBSD builders to NetBSD 8-beta #20852

Closed

basheerk mentioned this issue Jan 23, 2018

Enabling -race flag depending on arch kata-containers/tests#33

Merged

bradfitz mentioned this issue Apr 26, 2018

x/build: update NetBSD builder image to 8.0_RC1 #25109

Closed

bradfitz mentioned this issue Jul 16, 2018

runtime/race: complete race detector on netbsd/amd64 #26403

Open

gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 7, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

runtime/race: adding support for the race detector on other platforms #19273

runtime/race: adding support for the race detector on other platforms #19273

owenwaller commented Feb 24, 2017

ianlancetaylor commented Feb 24, 2017

bradfitz commented Jun 7, 2017

krytarowski commented Dec 5, 2017

bradfitz commented Dec 5, 2017

krytarowski commented Dec 5, 2017

krytarowski commented Dec 5, 2017

bradfitz commented Dec 5, 2017

ianlancetaylor commented Dec 5, 2017

krytarowski commented Dec 5, 2017 •

edited

krytarowski commented Dec 5, 2017

krytarowski commented Dec 5, 2017

bradfitz commented Dec 5, 2017

krytarowski commented Dec 5, 2017

ceseo commented Dec 6, 2017 •

edited

krytarowski commented Dec 6, 2017

ceseo commented Dec 6, 2017

krytarowski commented Dec 8, 2017

gopherbot commented May 11, 2018

gopherbot commented May 11, 2018

gopherbot commented Jul 10, 2018

runtime/race: adding support for the race detector on other platforms #19273

runtime/race: adding support for the race detector on other platforms #19273

Comments

owenwaller commented Feb 24, 2017

ianlancetaylor commented Feb 24, 2017

bradfitz commented Jun 7, 2017

krytarowski commented Dec 5, 2017

bradfitz commented Dec 5, 2017

krytarowski commented Dec 5, 2017

krytarowski commented Dec 5, 2017

bradfitz commented Dec 5, 2017

ianlancetaylor commented Dec 5, 2017

krytarowski commented Dec 5, 2017 • edited

krytarowski commented Dec 5, 2017

krytarowski commented Dec 5, 2017

bradfitz commented Dec 5, 2017

krytarowski commented Dec 5, 2017

ceseo commented Dec 6, 2017 • edited

krytarowski commented Dec 6, 2017

ceseo commented Dec 6, 2017

krytarowski commented Dec 8, 2017

gopherbot commented May 11, 2018

gopherbot commented May 11, 2018

gopherbot commented Jul 10, 2018

krytarowski commented Dec 5, 2017 •

edited

ceseo commented Dec 6, 2017 •

edited