Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/link: TestDWARF fails with the latest tip version #25392

Closed
YoshikiShibata opened this issue May 14, 2018 · 13 comments
Closed

cmd/link: TestDWARF fails with the latest tip version #25392

YoshikiShibata opened this issue May 14, 2018 · 13 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker
Milestone

Comments

@YoshikiShibata
Copy link

What did you do?

Just try to rebuild the latest tip version with all.bash

What did you expect to see?

No error on the build.

What did you see instead?

ok  	cmd/internal/test2json	0.252s
--- FAIL: TestDWARF (5.71s)
    --- FAIL: TestDWARF/testprogcgo (3.89s)
    	dwarf_test.go:145: did not find file:line for 0x40b9630 (main.main)
FAIL
FAIL	cmd/link	6.191s
ok  	cmd/link/internal/ld	96.030s

Does this issue reproduce with the latest release (go1.10.2)?

N/A

System details

go version devel +eeff8fa453 Mon May 14 20:33:00 2018 +0000 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/yoshiki/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/yoshiki/gocode:/Users/yoshiki/exercises/gpl:/Users/yoshiki/oak"
GORACE=""
GOROOT="/Users/yoshiki/tools/go"
GOTMPDIR=""
GOTOOLDIR="/Users/yoshiki/tools/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/pb/c7_4_d355ng5zfm4r_jg2gg00000gn/T/go-build881657275=/tmp/go-build -gno-record-gcc-switches -fno-common"
GOROOT/bin/go version: go version devel +eeff8fa453 Mon May 14 20:33:00 2018 +0000 darwin/amd64
GOROOT/bin/go tool compile -V: compile version devel +eeff8fa453 Mon May 14 20:33:00 2018 +0000
uname -v: Darwin Kernel Version 17.5.0: Fri Apr 13 19:32:32 PDT 2018; root:xnu-4570.51.2~1/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.13.4
BuildVersion:	17E202
lldb --version: lldb-900.0.50.1
  Swift-4.0
@meirf
Copy link
Contributor

meirf commented May 15, 2018

I am able to reproduce on mac (darwin amd64 10.13.3).
This was introduced by 7bac2a9. (@dr2chase)

@agnivade agnivade changed the title cmd/link: TestDWARF fails with the lastest tip version cmd/link: TestDWARF fails with the latest tip version May 15, 2018
@agnivade
Copy link
Contributor

I am unable to reproduce this on linux.

$gotip test -v cmd/link
=== RUN   TestDWARF
=== RUN   TestDWARF/testprog
=== RUN   TestDWARF/testprogcgo
--- PASS: TestDWARF (0.72s)
    --- PASS: TestDWARF/testprog (0.26s)
    --- PASS: TestDWARF/testprogcgo (0.37s)
=== RUN   TestDWARFiOS
--- SKIP: TestDWARFiOS (0.00s)
	dwarf_test.go:162: skipping on non-darwin/amd64 platform
=== RUN   TestLargeSymName
--- PASS: TestLargeSymName (0.00s)
=== RUN   TestIssue21703
--- PASS: TestIssue21703 (0.08s)
=== RUN   TestLargeText
--- SKIP: TestLargeText (0.00s)
	linkbig_test.go:24: Skipping large text section test in short mode or on amd64

@agnivade agnivade added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker labels May 15, 2018
@agnivade agnivade added this to the Go1.11 milestone May 15, 2018
@dr2chase
Copy link
Contributor

I am unable to reproduce this on Darwin.
I've been running all.bash all along.
I did recently upgrade my XCode (yesterday-recently).

go version devel +c64c2ce812 Mon May 14 16:12:59 2018 -0400 darwin/amd64

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/drchase/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/drchase/GoogleDrive/work/gocode"
GORACE=""
GOROOT="/Users/drchase/GoogleDrive/work/go"
GOTMPDIR=""
GOTOOLDIR="/Users/drchase/GoogleDrive/work/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/gr/vvb66dqx6jl6lh8wckfd5p9w0095tn/T/go-build292785485=/tmp/go-build -gno-record-gcc-switches -fno-common"

uname -v
Darwin Kernel Version 17.5.0: Fri Apr 13 19:32:32 PDT 2018; root:xnu-4570.51.2~1/RELEASE_X86_64

lldb --version
lldb-902.0.79.2
  Swift-4.1

I'll see if I can figure out what other information would be helpful to figure out what's causing this.

@thanm
Copy link
Contributor

thanm commented May 15, 2018

FWIW testDWARF is called with multiple build modes -- it would be helpful to know which one is failing (also version info for clang).

@dr2chase
Copy link
Contributor

Than, according to what I read, this is TestDWARF calling testDwarf with "" mode, so nothing special.
My attempt to reproduce by hand:

cd go/src/cmd/link
go build -o foo.exe ../../runtime/testdata/testprogcgo

nm foo.exe | grep main.main
00000000040b9c00 t main.main
# 40b9c00 is the address of interest
# gobjdump is from MacPorts binutils

gobjdump --dwarf=decodedline foo.exe | more

..../go/src/runtime/testdata/testprogcgo/main.go:[++]
untime/testdata/testprogcgo/main.go           24           0x40b9c00
untime/testdata/testprogcgo/main.go           24           0x40b9c18
untime/testdata/testprogcgo/main.go           25           0x40b9c2f
untime/testdata/testprogcgo/main.go           25           0x40b9c36
untime/testdata/testprogcgo/main.go           26           0x40b9c47

gobjdump -D foo.exe | more

00000000040b9c00 <main.main>:
 40b9c00:       65 48 8b 0c 25 30 00    mov    %gs:0x30,%rcx
 40b9c07:       00 00 
 40b9c09:       48 8d 44 24 e0          lea    -0x20(%rsp),%rax
 40b9c0e:       48 3b 41 10             cmp    0x10(%rcx),%rax
 40b9c12:       0f 86 cf 01 00 00       jbe    40b9de7 <main.main+0x1e7>
 40b9c18:       48 81 ec a0 00 00 00    sub    $0xa0,%rsp
 40b9c1f:       48 89 ac 24 98 00 00    mov    %rbp,0x98(%rsp)
 
gobjdump --dwarf=rawline foo.exe | more

  [0x0001e751]  Set is_stmt to 1
  [0x0001e752]  Special opcode 72: advance Address by 7 to 0x40b9bf8 and Line by -2 to 18
  [0x0001e753]  Advance PC by 7 to 0x40b9bff
  [0x0001e755]  Extended opcode 1: End of Sequence

  [0x0001e758]  Extended opcode 2: set Address to 0x40b9c00
  [0x0001e763]  Set File Name to entry 3 in the File Name Table
  [0x0001e765]  Advance Line by 23 to 24
  [0x0001e767]  Copy
  [0x0001e768]  Set prologue_end to true
  [0x0001e769]  Special opcode 244: advance Address by 24 to 0x40b9c18 and Line by 0 to 24
  [0x0001e76a]  Special opcode 235: advance Address by 23 to 0x40b9c2f and Line by 1 to 25
  [0x0001e76b]  Set is_stmt to 0

This looks good to me.

@YoshikiShibata
Copy link
Author

The problem still occurs on my MacBook. The following is the version information of clang:

MacBook-61:src yoshiki$ clang --version
Apple LLVM version 9.0.0 (clang-900.0.38)
Target: x86_64-apple-darwin17.5.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
MacBook-61:src yoshiki$

@YoshikiShibata
Copy link
Author

I tried with MacBook Air. The problem didn't occur. The version information of clang on the MacBook Air is following:

Yoshiki-no-MacBook-Air:~ yoshiki$ clang --version
Apple LLVM version 9.1.0 (clang-902.0.39.1)
Target: x86_64-apple-darwin17.5.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
Yoshiki-no-MacBook-Air:~ yoshiki$ 

The clang on my MacBook Air is newer than one on my MackBook.

@YoshikiShibata
Copy link
Author

After upgrading XCode and clang, the problem disappeared.

MacBook-61:~ yoshiki$ clang --version
Apple LLVM version 9.1.0 (clang-902.0.39.1)
Target: x86_64-apple-darwin17.5.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
MacBook-61:~ yoshiki$

@agnivade
Copy link
Contributor

Thanks for the analysis @YoshikiShibata !

@dr2chase - This matches with your findings since you mentioned that you upgraded xcode recently.

@dr2chase
Copy link
Contributor

You didn't perhaps save the binary produced with the older version of XCode? I'd love to see what it did differently.

@aarzilli
Copy link
Contributor

I think this issue should be reopened. It looks like some (most?) versions of dsymutil do not support (but see below) version 3 for debug_line, including the most recent version of dsymutil on 10.12.

If 10.12 is still considered supported this breaks backwards compatibility.

Now, Saying that dsymutil doesn't support v3 is incomplete, it seems that it actually does, but only if you lie to it and say it's v2. For example this is what the line table of a compilation unit produced by clang looks like in the final executable:

  Offset:                      0x2b3fc
  Length:                      154
  DWARF Version:               2
  Prologue Length:             30
  Minimum Instruction Length:  1
  Initial value of 'is_stmt':  1
  Line Base:                   -5
  Line Range:                  14
  Opcode Base:                 13

 Opcodes:
  Opcode 1 has 0 args
  Opcode 2 has 1 args
  Opcode 3 has 1 args
  Opcode 4 has 1 args
  Opcode 5 has 1 args
  Opcode 6 has 0 args
  Opcode 7 has 0 args
  Opcode 8 has 0 args
  Opcode 9 has 1 args
  Opcode 10 has 0 args
  Opcode 11 has 0 args
  Opcode 12 has 1 args

Note how it says version 2 but has more than 9 standard opcodes.

The choice is between lying, pretending 10.12 doesn't exist or reverting the prologueEnd thing.

@ysmolski
Copy link
Member

ysmolski commented May 20, 2018

It is still broken for me.

go version devel +cc09212f59 Sun May 20 00:09:28 2018 +0000 darwin/amd64

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/thorn/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/thorn/go"
GORACE=""
GOROOT="/Users/thorn/golang"
GOTMPDIR=""
GOTOOLDIR="/Users/thorn/golang/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/km/50gy6_q557v6_7vxbf9b29v00000gn/T/go-build027945066=/tmp/go-build -gno-record-gcc-switches -fno-common"

% uname -v
Darwin Kernel Version 15.6.0: Tue Jan 30 11:45:51 PST 2018; root:xnu-3248.73.8~1/RELEASE_X86_64

@dr2chase
Copy link
Contributor

I can confirm that setting the DWARF version on .debug_lines to 2 still works okay on 10.13.4 with corresponding XCode.

@golang golang locked and limited conversation to collaborators May 20, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker
Projects
None yet
Development

No branches or pull requests

8 participants