Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go test -S repeated symbols for darwin #20976

Closed
campoy opened this issue Jul 10, 2017 · 19 comments
Closed

cmd/go: go test -S repeated symbols for darwin #20976

campoy opened this issue Jul 10, 2017 · 19 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.

Comments

@campoy
Copy link
Contributor

campoy commented Jul 10, 2017

What did you do?

I ran the benchmark defined in this code with -gcflags "-S" in order to see the generated assembly.

package profiling

import "testing"

func div(a, b int) int {
	return int(float64(a) / float64(b))
}

var s int

func BenchmarkDiv(b *testing.B) {
	for i := 0; i < b.N; i++ {
		s = div(2, 3)
	}
}

I ran the benchmark with the command:

go test -bench=Div -gcflags "-S" 2>& out.text2

The output contains the definition of BenchmarkDiv and div twice, and their content differs.
See full output here.

# github.com/campoy/go-tooling-workshop/3-dynamic-analysis/3-profiling/jic
"".div STEXT nosplit size=41 args=0x18 locals=0x0
	0x0000 00000 (benchmarks_test.go:19)	TEXT	"".div(SB), NOSPLIT, $0-24
	0x0000 00000 (benchmarks_test.go:19)	FUNCDATA	$0, gclocals·54241e171da8af6ae173d69da0236748(SB)
	0x0000 00000 (benchmarks_test.go:19)	FUNCDATA	$1, gclocals·33cdeccccebe80329f1fdbee7f5874cb(SB)
	0x0000 00000 (benchmarks_test.go:19)	MOVQ	"".a+8(SP), AX
	0x0005 00005 (benchmarks_test.go:20)	XORPS	X0, X0
	0x0008 00008 (benchmarks_test.go:20)	CVTSQ2SD	AX, X0
	0x000d 00013 (benchmarks_test.go:20)	MOVQ	"".b+16(SP), AX
	0x0012 00018 (benchmarks_test.go:20)	XORPS	X1, X1
	0x0015 00021 (benchmarks_test.go:20)	CVTSQ2SD	AX, X1
	0x001a 00026 (benchmarks_test.go:20)	DIVSD	X1, X0
	0x001e 00030 (benchmarks_test.go:20)	CVTTSD2SQ	X0, AX
	0x0023 00035 (benchmarks_test.go:20)	MOVQ	AX, "".~r2+24(SP)
	0x0028 00040 (benchmarks_test.go:20)	RET
	0x0000 48 8b 44 24 08 0f 57 c0 f2 48 0f 2a c0 48 8b 44  H.D$..W..H.*.H.D
	0x0010 24 10 0f 57 c9 f2 48 0f 2a c8 f2 0f 5e c1 f2 48  $..W..H.*...^..H
	0x0020 0f 2c c0 48 89 44 24 18 c3                       .,.H.D$..
"".BenchmarkDiv STEXT nosplit size=67 args=0x8 locals=0x0
	0x0000 00000 (benchmarks_test.go:25)	TEXT	"".BenchmarkDiv(SB), NOSPLIT, $0-8
	0x0000 00000 (benchmarks_test.go:25)	FUNCDATA	$0, gclocals·a36216b97439c93dafebe03e7f0808b5(SB)
	0x0000 00000 (benchmarks_test.go:25)	FUNCDATA	$1, gclocals·33cdeccccebe80329f1fdbee7f5874cb(SB)
	0x0000 00000 (benchmarks_test.go:25)	MOVQ	"".b+8(SP), AX
	0x0005 00005 (benchmarks_test.go:25)	MOVL	$0, CX
	0x0007 00007 (benchmarks_test.go:26)	JMP	54
	0x0009 00009 (benchmarks_test.go:27)	MOVL	$2, DX
	0x000e 00014 (benchmarks_test.go:27)	XORPS	X0, X0
	0x0011 00017 (benchmarks_test.go:27)	CVTSQ2SD	DX, X0
	0x0016 00022 (benchmarks_test.go:27)	MOVL	$3, BX
	0x001b 00027 (benchmarks_test.go:27)	XORPS	X1, X1
	0x001e 00030 (/Users/cam# github.com/campoy/go-tooling-workshop/3-dynamic-analysis/3-profiling/jic
"".div STEXT nosplit size=41 args=0x18 locals=0x0
	0x0000 00000 (benchmarks_test.go:19)	TEXT	"".div(SB), NOSPLIT, $0-24
	0x0000 00000 (benchmarks_test.go:19)	FUNCDATA	$0, gclocals·54241e171da8af6ae173d69da0236748(SB)
	0x0000 00000 (benchmarks_test.go:19)	FUNCDATA	$1, gclocals·33cdeccccebe80329f1fdbee7f5874cb(SB)
	0x0000 00000 (benchmarks_test.go:19)	MOVQ	"".a+8(SP), AX
	0x0005 00005 (benchmarks_test.go:20)	XORPS	X0, X0
	0x0008 00008 (benchmarks_test.go:20)	CVTSQ2SD	AX, X0
	0x000d 00013 (benchmarks_test.go:20)	MOVQ	"".b+16(SP), AX
	0x0012 00018 (benchmarks_test.go:20)	XORPS	X1, X1
	0x0015 00021 (benchmarks_test.go:20)	CVTSQ2SD	AX, X1
	0x001a 00026 (benchmarks_test.go:20)	DIVSD	X1, X0
	0x001e 00030 (benchmarks_test.go:20)	CVTTSD2SQ	X0, AX
	0x0023 00035 (benchmarks_test.go:20)	MOVQ	AX, "".~r2+24(SP)
	0x0028 00040 (benchmarks_test.go:20)	RET
	0x0000 48 8b 44 24 08 0f 57 c0 f2 48 0f 2a c0 48 8b 44  H.D$..W..H.*.H.D
	0x0010 24 10 0f 57 c9 f2 48 0f 2a c8 f2 0f 5e c1 f2 48  $..W..H.*...^..H
	0x0020 0f 2c c0 48 89 44 24 18 c3                       .,.H.D$..
"".BenchmarkDiv STEXT nosplit size=67 args=0x8 locals=0x0
	0x0000 00000 (benchmarks_test.go:25)	TEXT	"".BenchmarkDiv(SB), NOSPLIT, $0-8
	0x0000 00000 (benchmarks_test.go:25)	FUNCDATA	$0, gclocals·a36216b97439c93dafebe03e7f0808b5(SB)
	0x0000 00000 (benchmarks_test.go:25)	FUNCDATA	$1, gclocals·33cdeccccebe80329f1fdbee7f5874cb(SB)
	0x0000 00000 (benchmarks_test.go:25)	MOVQ	"".b+8(SP), AX
	0x0005 00005 (benchmarks_test.go:25)	MOVL	$0, CX
	0x0007 00007 (benchmarks_test.go:26)	JMP	54
	0x0009 00009 (benchmarks_test.go:27)	MOVL	$2, DX
	0x000e 00014 (benchmarks_test.go:27)	XORPS	X0, X0
	0x0011 00017 (benchmarks_test.go:27)	CVTSQ2SD	DX, X0
	0x0016 00022 (benchmarks_test.go:27)	MOVL	$3, BX
	0x001b 00027 (benchmarks_test.go:27)	XORPS	X1, X1
	0x001e 00030 (benchmarks_test.go:27)	CVTSQ2SD	BX, X1
	0x0023 00035 (benchmarks_test.go:27)	DIVSD	X1, X0
	0x0027 00039 (benchmarks_test.go:27)	CVTTSD2SQ	X0, SI
	0x002c 00044 (benchmarks_test.go:27)	MOVQ	SI, "".s(SB)
	0x0033 00051 (benchmarks_test.go:26)	INCQ	CX
	0x0036 00054 (benchmarks_test.go:26)	MOVQ	240(AX), DX
	0x003d 00061 (benchmarks_test.go:26)	CMPQ	CX, DX
	0x0040 00064 (benchmarks_test.go:26)	JLT	9
	0x0042 00066 (benchmarks_test.go:29)	RET
	0x0000 48 8b 44 24 08 31 c9 eb 2d ba 02 00 00 00 0f 57  H.D$.1..-......W
	0x0010 c0 f2 48 0f 2a c2 bb 03 00 00 00 0f 57 c9 f2 48  ..H.*.......W..H
	0x0020 0f 2a cb f2 0f 5e c1 f2 48 0f 2c f0 48 89 35 00  .*...^..H.,.H.5.
	0x0030 00 00 00 48 ff c1 48 8b 90 f0 00 00 00 48 39 d1  ...H..H......H9.
	0x0040 7c c7 c3                                         |..
	rel 47+4 t=15 "".s+0

What did you expect to see?

I expected to see each symbol defined only once in the output.

What did you see instead?

Two slightly different definitions of the same functions div and BenchmarkDiv.

Does this issue reproduce with the latest release (go1.8.3)?

Yes, but only with darwin. Tried running the same with linux on Docker with golang:1.8.3 and everything worked correctly.

System details

go version devel +eab99a8d54 Mon Jun 26 21:12:22 2017 +0000 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/campoy"
GORACE=""
GOROOT="/Users/campoy/src/golang.org/x/go"
GOTOOLDIR="/Users/campoy/src/golang.org/x/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/yt/3jw3ts594kd9w6t6dw7rp6y40043p6/T/go-build685643079=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOROOT/bin/go version: go version devel +eab99a8d54 Mon Jun 26 21:12:22 2017 +0000 darwin/amd64
GOROOT/bin/go tool compile -V: compile version devel +eab99a8d54 Mon Jun 26 21:12:22 2017 +0000 X:framepointer
uname -v: Darwin Kernel Version 16.6.0: Fri Apr 14 16:21:16 PDT 2017; root:xnu-3789.60.24~6/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.12.5
BuildVersion:	16F73
lldb --version: lldb-370.0.42
  Swift-3.1
@davecheney
Copy link
Contributor

I can't reproduce the issue on my darwin machine,

zapf(~/src/profiling) % go test -bench=Div -gcflags "-S"  2>&1 | grep Div
"".BenchmarkDiv STEXT nosplit size=67 args=0x8 locals=0x0
        0x0000 00000 (/Users/dfc/src/profiling/benchdiv.go:11)  TEXT    "".BenchmarkDiv(SB), NOSPLIT, $0-8
go.info."".BenchmarkDiv SDWARFINFO size=46
        0x0000 02 22 22 2e 42 65 6e 63 68 6d 61 72 6b 44 69 76  ."".BenchmarkDiv
        rel 17+8 t=1 "".BenchmarkDiv+0
        rel 25+8 t=1 "".BenchmarkDiv+67
go.range."".BenchmarkDiv SDWARFRANGE size=0

Note: i'm not running the latest release of osx, so i'm also not running the latest version of xcdeo

zapf(~/src/profiling) % system_profiler SPSoftwareDataType
Software:

    System Software Overview:

      System Version: OS X 10.11.6 (15G1510)
      Kernel Version: Darwin 15.6.0
zapf(~/src/profiling) % /usr/bin/xcodebuild -version
Xcode 8.0
Build version 8A218a

This might be relevant.

@campoy
Copy link
Contributor Author

campoy commented Jul 10, 2017

My system is running:

$ system_profiler SPSoftwareDataType            
Software:

    System Software Overview:

      System Version: macOS 10.12.5 (16F73)
      Kernel Version: Darwin 16.6.0

Maybe that's why?

@bradleyfalzon
Copy link

I'm running the same:

$ system_profiler SPSoftwareDataType
Software:

    System Software Overview:

      System Version: macOS 10.12.5 (16F73)
      Kernel Version: Darwin 16.6.0

Output:

$ go test -bench=Div -gcflags "-S"  2>&1 | grep Div
"".BenchmarkDiv t=1 size=81 args=0x8 locals=0x0
	0x0000 00000 (/Users/bradleyf/go/src/github.com/bradleyfalzon/test/issue-20976/profiling.go:11)	TEXT	"".BenchmarkDiv(SB), $0-8
go.info."".BenchmarkDiv t=45 size=48
	0x0000 02 22 22 2e 42 65 6e 63 68 6d 61 72 6b 44 69 76  ."".BenchmarkDiv
	rel 17+8 t=1 "".BenchmarkDiv+0
	rel 25+8 t=1 "".BenchmarkDiv+81

@campoy
Copy link
Contributor Author

campoy commented Jul 10, 2017

Do you have XCode installed, @bradleyfalzon?

I do not:

$ gcc --version

Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

@bradleyfalzon
Copy link

bradleyfalzon commented Jul 10, 2017

No, pretty vanilla workstation:

$ gcc --version
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

All output: https://gist.github.com/bradleyfalzon/3be20a61b0567d23e0d7dd027a4ea512

@campoy
Copy link
Contributor Author

campoy commented Jul 10, 2017

Well ... there has to be something different. What's your go version?

@bradleyfalzon
Copy link

$ go version
go version go1.8.3 darwin/amd64
$ go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/bradleyf/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/q5/zxqf32v92r388478vjyd70hh0000gn/T/go-build800668811=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

@campoy
Copy link
Contributor Author

campoy commented Jul 10, 2017

Have you installed go by downloading the installer?
I wonder if that might be the difference, I'm compiling from source code using:

GOROOT_BOOTSTRAP=$GOPATH/src/golang.org/x/go1.7.4 ./make.bash

@campoy
Copy link
Contributor Author

campoy commented Jul 10, 2017

I just installed Go from scratch using the pkg installer but I still can reproduce the issue.

@bradleyfalzon
Copy link

Have you installed go by downloading the installer?

Yes, it's using https://storage.googleapis.com/golang/go1.8.3.darwin-amd64.pkg

It's just that and your source code installed today. I had to create $GOPATH/{src,pkg,bin} myself.

@hirochachacha
Copy link
Contributor

I can reproduce this, however I'm wondering about 2>& out.text2.
If I replace it with 2> out.text2, the problem is gone.

@ALTree ALTree changed the title go test -S repeated symbols for darwin cmd/go: go test -S repeated symbols for darwin Jul 11, 2017
@ALTree ALTree added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 11, 2017
@hirochachacha
Copy link
Contributor

I meant this is not a bug.

@ALTree
Copy link
Member

ALTree commented Jul 11, 2017

It seems like you are mistakenly writing the output two times, to the same file. First pass is interrupted after 2 functions and then there's a full disassembly written to the file.

@hirochachacha
Copy link
Contributor

hirochachacha commented Jul 11, 2017

$ ls foobar
ls: foobar: No such file or directory
$ ls foobar 2>& /dev/stdout
ls: foobar: No such file or directory
ls: foobar: No such file or directory

So, this is not a bug if the initial report is correct.

EDIT:

I am using zsh. It seems that 2>& is not valid on other shells.

@ALTree
Copy link
Member

ALTree commented Jul 11, 2017

Nice. My shell says "bash: out.text2: ambiguous redirect":

$ go test -bench=Div -gcflags "-S" 2>& out.text2
bash: out.text2: ambiguous redirect

@ianlancetaylor
Copy link
Contributor

Are you really writing literally 2>& out.text? Which shell are you using? That syntax is not supported by either POSIX sh or bash.

@robpike
Copy link
Contributor

robpike commented Jul 12, 2017

The desired effect comes from

command >x 2>&1

in the one true shell. This modem noise burst says put stdout to x, and make 2 a dup of 1. But what you have written makes little sense to me beyond saying, "this is a mistake".

@campoy
Copy link
Contributor Author

campoy commented Jul 12, 2017

After some extra investigation this is clearly a weird behavior on the zsh side.

@campoy campoy closed this as completed Jul 12, 2017
@campoy
Copy link
Contributor Author

campoy commented Jul 12, 2017

For those interested in more details, it seems 2>& out redirects twice to out and writes it without sync which causes random garbling.

A simple program that shows the behavior:

package main

import (
	"fmt"
	"os"
)

func main() {
	out := ""
	for i := 0; i < 1000; i++ {
		out += "stderr"
	}
	fmt.Fprintln(os.Stdout, "stdout")
	fmt.Fprintln(os.Stderr, out)
}

@golang golang locked and limited conversation to collaborators Jul 12, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

8 participants