Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "cmd/cgo/internal/testsanitizers" && test == "TestTSAN"

--- FAIL: TestTSAN (7.58s)
    --- FAIL: TestTSAN/tsan (1.49s)
        tsan_test.go:77: /workdir/tmp/TestTSAN2652834654/tsan exited with signal: segmentation fault

— watchflakes

This isn't very actionable at the moment. We'll wait for more failures.

It seems odd to be building TestTSAN with the -race option. Those tests link in the gcc thread sanitizer library and the -race option links in the LLVM thread sanitizer code from the race syso file. Should they be expected to work together?

That is a good point. This will link together two different copies of the TSAN code. The TSAN code has the same sources, so it will often work, but if there are version discrepancies anything could happen.

Change https://go.dev/cl/524896 mentions this issue: cmd/cgo/internal/testsanitizers: don't test a sanitizer we are using

@laboger, what leads you to believe that the binaries built by the test are linking in both sanitizers?

As far as I can tell, the configure function doesn't take into account the settings with which the test binary itself was built. (But perhaps it should?)

Or maybe these settings are leaking in through an env file or a GOFLAGS environment variable?
(But I don't see a GOFLAGS variable in the environment logged in https://build.golang.org/log/790a7852a56dfa511475f6e00e42da18cbc23a26.)

@laboger, what leads you to believe that the binaries built by the test are linking in both sanitizers?

I was involved in adding the support for the -race option on ppc64le, so I know the -race option causes the .syso files from runtime/race to be linked into the binaries. Those syso files are built from LLVM's thread sanitizer code.

In the last year or so I had to debug some of the TestTSAN tests on ppc64le because we were seeing flaky results and discovered that those tests were using cgo to link in the sanitizer libraries that come with gcc.

For example, cmd/cgo/internal/testsanitizers/testdata/tsan.go and all the other tsan tests in this directory contain something like this:

package main

// This program produced false race reports when run under the C/C++
// ThreadSanitizer, as it did not understand the synchronization in
// the Go code.

/*
#cgo CFLAGS: -fsanitize=thread
#cgo LDFLAGS: -fsanitize=thread

These flags cause the gcc sanitizer library code to be linked in.

I know the -race option causes the .syso files from runtime/race to be linked into the binaries.

I agree that that is correct. However, I don't see how the -race option would be set for the binaries built by cmd/cgo/internal/testsanitizers.TestTSAN, unless it crept in via GOFLAGS or .config/go/env (which AFAICT is not the case for the failures reported here by watchflakes).

Those binaries are built by calling go build here:
https://cs.opensource.google/go/go/+/master:src/cmd/cgo/internal/testsanitizers/tsan_test.go;l=68;drc=a674ab1961ca75d902ebbdf0a3d363501fd017ae

In the case above where you show the failures, it is running on the linux-amd64-race builder? Doesn't that mean it is running everything with the -race option? Maybe I misunderstood what that means.

The -race builder does indeed build all of the packages and tests using the -race option. So the testsanitizers.test binary itself is built with -race and linked against the .syso files.

However, testsanitizers.test itself calls out to go build to build another binary, which it then runs in order to perform the actual test. And that other binary is not built with -race, because nothing in the test (or the builder environment) causes that to happen.