cmd/go: TestCgoConsistentResults fails with clang-3.9 #19964

bradfitz · 2017-04-13T17:48:34Z

@jessfraz and I upgraded the linux-*-clang builders for the first time in ages.

They now have clang-3.9 and now builds are failing with:

https://build.golang.org/log/e66348d8581d2f80e7d801961cf8814d9af1bed4

--- FAIL: TestCgoConsistentResults (4.47s)
	go_test.go:260: running testgo [build -o /tmp/gotest555473365/cgotest1 cgotest]
	go_test.go:260: running testgo [build -x -o /tmp/gotest555473365/cgotest2 cgotest]
	go_test.go:279: standard error:
	go_test.go:280: WORK=/tmp/go-build283999844
		mkdir -p $WORK/cgotest/_obj/
		mkdir -p $WORK/
		cd /tmp/workdir/go/src/cmd/go/testdata/src/cgotest
		CGO_LDFLAGS="-g" "-O2" /tmp/workdir/go/pkg/tool/linux_amd64/cgo -objdir $WORK/cgotest/_obj/ -importpath cgotest -- -I $WORK/cgotest/_obj/ -g -O2 m.go
		cd $WORK
		/usr/bin/clang -fdebug-prefix-map=a=b -c trivial.c
		/usr/bin/clang -gno-record-gcc-switches -c trivial.c
		cd /tmp/workdir/go/src/cmd/go/testdata/src/cgotest
		/usr/bin/clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I $WORK/cgotest/_obj/ -g -O2 -o $WORK/cgotest/_obj/_cgo_export.o -c $WORK/cgotest/_obj/_cgo_export.c
		/usr/bin/clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I $WORK/cgotest/_obj/ -g -O2 -o $WORK/cgotest/_obj/m.cgo2.o -c $WORK/cgotest/_obj/m.cgo2.c
		/usr/bin/clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I $WORK/cgotest/_obj/ -g -O2 -o $WORK/cgotest/_obj/_cgo_main.o -c $WORK/cgotest/_obj/_cgo_main.c
		/usr/bin/clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -o $WORK/cgotest/_obj/_cgo_.o $WORK/cgotest/_obj/_cgo_main.o $WORK/cgotest/_obj/_cgo_export.o $WORK/cgotest/_obj/m.cgo2.o -g -O2
		/tmp/workdir/go/pkg/tool/linux_amd64/cgo -dynpackage cgotest -dynimport $WORK/cgotest/_obj/_cgo_.o -dynout $WORK/cgotest/_obj/_cgo_import.go
		cd $WORK
		/usr/bin/clang -no-pie -c trivial.c
		cd /tmp/workdir/go/src/cmd/go/testdata/src/cgotest
		/usr/bin/clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -o $WORK/cgotest/_obj/_all.o $WORK/cgotest/_obj/_cgo_export.o $WORK/cgotest/_obj/m.cgo2.o -g -O2 -Wl,-r -nostdlib -Wl,--build-id=none
		/tmp/workdir/go/pkg/tool/linux_amd64/compile -o $WORK/cgotest.a -trimpath $WORK -p cgotest -buildid da01d0527cc40b26d8afc2e889ecf3cc74ca325a -D _/tmp/workdir/go/src/cmd/go/testdata/src/cgotest -I $WORK -pack $WORK/cgotest/_obj/_cgo_gotypes.go $WORK/cgotest/_obj/m.cgo1.go $WORK/cgotest/_obj/_cgo_import.go
		pack r $WORK/cgotest.a $WORK/cgotest/_obj/_all.o # internal
		mkdir -p /tmp/gotest555473365/
		mv $WORK/cgotest.a /tmp/gotest555473365/cgotest2
		
	go_test.go:3398: building cgotest twice did not produce the same output
FAIL
FAIL	cmd/go	60.574s

/cc @ianlancetaylor

The text was updated successfully, but these errors were encountered:

jessfraz · 2017-04-13T17:52:58Z

hmm i can test with 3.5 and just verify for sanity that was working otherwise might be something weird...

…

On Thu, Apr 13, 2017 at 1:49 PM, Brad Fitzpatrick ***@***.***> wrote: @jessfraz <https://github.com/jessfraz> and I upgraded the linux-*-clang builders for the first time in ages. They now have clang-3.9 and now builds are failing with: https://build.golang.org/log/e66348d8581d2f80e7d801961cf8814d9af1bed4 --- FAIL: TestCgoConsistentResults (4.47s) go_test.go:260: running testgo [build -o /tmp/gotest555473365/cgotest1 cgotest] go_test.go:260: running testgo [build -x -o /tmp/gotest555473365/cgotest2 cgotest] go_test.go:279: standard error: go_test.go:280: WORK=/tmp/go-build283999844 mkdir -p $WORK/cgotest/_obj/ mkdir -p $WORK/ cd /tmp/workdir/go/src/cmd/go/testdata/src/cgotest CGO_LDFLAGS="-g" "-O2" /tmp/workdir/go/pkg/tool/linux_amd64/cgo -objdir $WORK/cgotest/_obj/ -importpath cgotest -- -I $WORK/cgotest/_obj/ -g -O2 m.go cd $WORK /usr/bin/clang -fdebug-prefix-map=a=b -c trivial.c /usr/bin/clang -gno-record-gcc-switches -c trivial.c cd /tmp/workdir/go/src/cmd/go/testdata/src/cgotest /usr/bin/clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I $WORK/cgotest/_obj/ -g -O2 -o $WORK/cgotest/_obj/_cgo_export.o -c $WORK/cgotest/_obj/_cgo_export.c /usr/bin/clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I $WORK/cgotest/_obj/ -g -O2 -o $WORK/cgotest/_obj/m.cgo2.o -c $WORK/cgotest/_obj/m.cgo2.c /usr/bin/clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -I $WORK/cgotest/_obj/ -g -O2 -o $WORK/cgotest/_obj/_cgo_main.o -c $WORK/cgotest/_obj/_cgo_main.c /usr/bin/clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -o $WORK/cgotest/_obj/_cgo_.o $WORK/cgotest/_obj/_cgo_main.o $WORK/cgotest/_obj/_cgo_export.o $WORK/cgotest/_obj/m.cgo2.o -g -O2 /tmp/workdir/go/pkg/tool/linux_amd64/cgo -dynpackage cgotest -dynimport $WORK/cgotest/_obj/_cgo_.o -dynout $WORK/cgotest/_obj/_cgo_import.go cd $WORK /usr/bin/clang -no-pie -c trivial.c cd /tmp/workdir/go/src/cmd/go/testdata/src/cgotest /usr/bin/clang -I . -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -o $WORK/cgotest/_obj/_all.o $WORK/cgotest/_obj/_cgo_export.o $WORK/cgotest/_obj/m.cgo2.o -g -O2 -Wl,-r -nostdlib -Wl,--build-id=none /tmp/workdir/go/pkg/tool/linux_amd64/compile -o $WORK/cgotest.a -trimpath $WORK -p cgotest -buildid da01d0527cc40b26d8afc2e889ecf3cc74ca325a -D _/tmp/workdir/go/src/cmd/go/testdata/src/cgotest -I $WORK -pack $WORK/cgotest/_obj/_cgo_gotypes.go $WORK/cgotest/_obj/m.cgo1.go $WORK/cgotest/_obj/_cgo_import.go pack r $WORK/cgotest.a $WORK/cgotest/_obj/_all.o # internal mkdir -p /tmp/gotest555473365/ mv $WORK/cgotest.a /tmp/gotest555473365/cgotest2 go_test.go:3398: building cgotest twice did not produce the same output FAIL FAIL cmd/go 60.574s /cc @ianlancetaylor <https://github.com/ianlancetaylor> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#19964>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABYNbBJnVbOv-ME8hdyrDJGpkWuKQen4ks5rvmBCgaJpZM4M9ARL> .

-- Jessie Frazelle 4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3 pgp.mit.edu <http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3>

jessfraz · 2017-04-13T18:59:34Z

ya I can reproduce with 3.9 only

ianlancetaylor · 2017-04-13T23:05:12Z

CC @neild

n · 2017-04-14T01:04:14Z

Looks like clang-3.9 is including path information in the binary.

82,83c82,83
< /tmp/go-build662498961/cgotest/_obj/_cgo_export.c
< /tmp/go-build662498961/cgotest/_obj/m.cgo2.c
---
> /tmp/go-build714166219/cgotest/_obj/_cgo_export.c
> /tmp/go-build714166219/cgotest/_obj/m.cgo2.c

Stripping cgotest seems to resolve the issue.

3427c3427
< 	ldflags = append(ldflags, "-Wl,-s,-r", "-nostdlib")
---
> 	ldflags = append(ldflags, "-Wl,-r", "-nostdlib")

go test -run TestCgoConsistentResults go_test.go
ok  	command-line-arguments	23.184s

Add: Probably not the solution. Just more info on the problem.

jessfraz · 2017-04-14T09:09:05Z

Whoa good find!

…

On Apr 13, 2017 21:04, "Nick McCann" ***@***.***> wrote: Looks like clang-3.9 is including path information in the binary. 82,83c82,83 < /tmp/go-build662498961/cgotest/_obj/_cgo_export.c < /tmp/go-build662498961/cgotest/_obj/m.cgo2.c --- > /tmp/go-build714166219/cgotest/_obj/_cgo_export.c > /tmp/go-build714166219/cgotest/_obj/m.cgo2.c Stripping cgotest seems to resolve the issue. 3427c3427 < ldflags = append(ldflags, "-Wl,-s,-r", "-nostdlib") --- > ldflags = append(ldflags, "-Wl,-r", "-nostdlib") go test -run TestCgoConsistentResults go_test.go ok command-line-arguments 23.184s — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#19964 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABYNbGlXkg9KDqUVq9RD-yI7NPrihqkAks5rvsYxgaJpZM4M9ARL> .

amenzhinsky · 2017-04-17T21:02:28Z

I think using the CGO_LDFLAGS env var would suffice. Changing default ldflags is not a good idea because we wouldn't be able to turn stripping down.

bradfitz · 2017-04-22T01:15:41Z

We're using -fdebug-prefix-map though, right?

I checked that our image supports that flag, so this should be true:

        if b.gccSupportsFlag("-fdebug-prefix-map=a=b") {                                                             | 
                a = append(a, "-fdebug-prefix-map="+b.WorkDir+"=/tmp/go-build")                                      | 
        }                                                                                                            |

(Ignore the gcc part of the method name... it just checks the C compiler supporting the given flag.`

But you're saying that the linker is adding filenames?

Does the gcc linker not add filenames to its output? Which option is that? Maybe we're just missing the clang equivalent flags in the place where we ask gcc to not add filenames?

It seems that stripping the whole binary is overkill. Is that what we do with gcc?

/cc @ianlancetaylor

ianlancetaylor · 2017-04-22T04:57:17Z

A filename that ends in .c would not be added by the linker. It's possible that -fdebug-prefix-map only affects debug info, and that this file name is appearing because the compiler is emitting a .file pseudo-op. In GCC -fdebug-prefix-map is applied to the .file pseudo-op, but perhaps in clang it is not. That's just a guess, though.

I only have clang 3.6 installed myself, and it does not seem to support -fdebug-prefix-map at all. According to a web search the option was added to clang in version 3.8. Note that the test always passes with a compiler that does not support that option--the test checks for that.

Hmmm, I just realized that I can use gomote to test clang 3.9. And I am correct: the -fdebug-prefix-map option is being applied to the debug info but not to the initial .file pseudo-op.

My only current thought for how to fix this is that instead of compiling $WORK/a/b/foo.c, we should change the command to (cd $WORK/a/b && clang -g -c foo.c). We'll still want -fdebug-prefix-map, I think, but that should fix the .file pseudo-op.

gopherbot · 2017-04-25T04:27:24Z

CL https://golang.org/cl/41629 mentions this issue.

bradfitz added help wanted NeedsFix The path to resolution is known, but the work has not been done. Testing An issue that has been verified to require only test changes, not just a test failure. labels Apr 13, 2017

bradfitz added this to the Go1.9 milestone Apr 13, 2017

gopherbot closed this as completed in 8ae60dc Apr 25, 2017

golang locked and limited conversation to collaborators Apr 25, 2018

gopherbot added the FrozenDueToAge label Apr 25, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: TestCgoConsistentResults fails with clang-3.9 #19964

cmd/go: TestCgoConsistentResults fails with clang-3.9 #19964

bradfitz commented Apr 13, 2017

jessfraz commented Apr 13, 2017 via email

jessfraz commented Apr 13, 2017

ianlancetaylor commented Apr 13, 2017

n commented Apr 14, 2017 •

edited

jessfraz commented Apr 14, 2017 via email

amenzhinsky commented Apr 17, 2017 •

edited

bradfitz commented Apr 22, 2017

ianlancetaylor commented Apr 22, 2017

gopherbot commented Apr 25, 2017

cmd/go: TestCgoConsistentResults fails with clang-3.9 #19964

cmd/go: TestCgoConsistentResults fails with clang-3.9 #19964

Comments

bradfitz commented Apr 13, 2017

jessfraz commented Apr 13, 2017 via email

jessfraz commented Apr 13, 2017

ianlancetaylor commented Apr 13, 2017

n commented Apr 14, 2017 • edited

jessfraz commented Apr 14, 2017 via email

amenzhinsky commented Apr 17, 2017 • edited

bradfitz commented Apr 22, 2017

ianlancetaylor commented Apr 22, 2017

gopherbot commented Apr 25, 2017

n commented Apr 14, 2017 •

edited

amenzhinsky commented Apr 17, 2017 •

edited