debug/elf: does not properly apply DWARF RELA relocations #40879

vikmik · 2020-08-19T03:55:51Z

What version of Go are you using (`go version`)?

go version go1.14.4 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/vic/.cache/go-build"
GOENV="/home/vic/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/vic/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.14"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.14/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/vic/go/src/test/test/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build635738080=/tmp/go-build -gno-record-gcc-switches"

What did you do?

EDIT: I've since added a standalone repro case here:
>>>>>>> #40879 (comment) <<<<<<<

I'm trying to get the PC ranges from the Linux kernel DWARF information:

1) Get the debug version of any mainstream Linux Kernel image, uncompressed (disclaimer: it's big!)

wget http://http.us.debian.org/debian/pool/main/l/linux/linux-image-4.19.0-10-amd64-dbg_4.19.132-1_amd64.deb
ar x linux-image-4.19.0-10-amd64-dbg_4.19.132-1_amd64.deb
tar xf data.tar.xz
# The vmlinux image should be at usr/lib/debug/vmlinux-4.19.0-10-amd64

2) Get the DIE offset for a compilation unit (here I'm using ./init/main.c)

$ llvm-dwarfdump usr/lib/debug/vmlinux-4.19.0-10-amd64 --name=./init/main.c | grep DW_TAG_compile_unit
0x000217db: DW_TAG_compile_unit

3) Try to use the debug/elf and debug/dwarf libraries to print the PC ranges for this CU:

package main

import (
	"debug/elf"
	"fmt"
	"log"
)

func main() {
	file, err := elf.Open("usr/lib/debug/vmlinux-4.19.0-10-amd64")
	if err != nil {
		log.Fatal(err)
	}
	defer file.Close()

	dw, err := file.DWARF()
	if err != nil {
		log.Fatal(err)
	}

	reader := dw.Reader()
	reader.Seek(0x000217db) // Offset value is from llvm-dwarfdump output

	// Read entry at that offset
	entry, err := reader.Next()
	if err != nil {
		log.Fatal(err)
	}

	// Read PC ranges from DIE
	ranges, err := dw.Ranges(entry)
	if err != nil {
		log.Fatal(err)
	}

	for i := range ranges {
		fmt.Printf("range %d: [0x%x, 0x%x]\n", i, ranges[i][0], ranges[i][1])
	}
}

What did you expect to see?

One can use llvm-dwarfdump to show the correct PC ranges for this DIE:

$ llvm-dwarfdump usr/lib/debug/vmlinux-4.19.0-10-amd64 --name=./init/main.c
usr/lib/debug/vmlinux-4.19.0-10-amd64:	file format ELF64-x86-64

0x000217db: DW_TAG_compile_unit
[...]
              DW_AT_ranges	(0x000023e0
                 [0xffffffff810020b0, 0xffffffff81002983)
                 [0xffffffff81002983, 0xffffffff8100298b)
                 [0xffffffff82457574, 0xffffffff82457589)
                 [0xffffffff82457589, 0xffffffff8245759b)
                 [0xffffffff8245759b, 0xffffffff824575ad)
                 [0xffffffff824575ad, 0xffffffff824575d9)
                 [0xffffffff824575d9, 0xffffffff82457605)
                 [0xffffffff82457605, 0xffffffff82457693)
                 [0xffffffff82457693, 0xffffffff824576e9)
                 [0xffffffff824576e9, 0xffffffff8245773e)
                 [0xffffffff8245773e, 0xffffffff824578d3)
                 [0xffffffff8100298b, 0xffffffff810029b9)
                 [0xffffffff81722909, 0xffffffff817229b3)
                 [0xffffffff810029b9, 0xffffffff810029f7)
                 [0xffffffff810029f7, 0xffffffff81002a26)
                 [0xffffffff824578d3, 0xffffffff824578fb)
                 [0xffffffff824578fb, 0xffffffff82457920)
                 [0xffffffff82457920, 0xffffffff82457978)
                 [0xffffffff82457978, 0xffffffff824579d3)
                 [0xffffffff824579d3, 0xffffffff82457aa7)
                 [0xffffffff82457aa7, 0xffffffff82457ab8)
                 [0xffffffff81002a26, 0xffffffff81002a6d)
                 [0xffffffff82457ab8, 0xffffffff82457ac2)
                 [0xffffffff82457ac2, 0xffffffff82457aea)
                 [0xffffffff82457aea, 0xffffffff82457b5b)
                 [0xffffffff82457b5b, 0xffffffff82457b61)
                 [0xffffffff82457b61, 0xffffffff82457b67)
                 [0xffffffff82457b67, 0xffffffff82457b6d)
                 [0xffffffff82457b6d, 0xffffffff82457b73)
                 [0xffffffff82457b73, 0xffffffff824580a2)
                 [0xffffffff824580a2, 0xffffffff824582ba)
                 [0xffffffff817229b3, 0xffffffff81722ab2))
[...]

What did you see instead?

The displayed ranges seem to be completely off:

$ ./test
range 0: [0x20b0, 0x2983]
range 1: [0x2983, 0x298b]
range 2: [0x574, 0x589]
range 3: [0x589, 0x59b]
range 4: [0x59b, 0x5ad]
range 5: [0x5ad, 0x5d9]
range 6: [0x5d9, 0x605]
range 7: [0x605, 0x693]
range 8: [0x693, 0x6e9]
range 9: [0x6e9, 0x73e]
range 10: [0x73e, 0x8d3]
range 11: [0x298b, 0x29b9]
range 12: [0x722909, 0x7229b3]
range 13: [0x29b9, 0x29f7]
range 14: [0x29f7, 0x2a26]
range 15: [0x8d3, 0x8fb]
range 16: [0x8fb, 0x920]
range 17: [0x920, 0x978]
range 18: [0x978, 0x9d3]
range 19: [0x9d3, 0xaa7]
range 20: [0xaa7, 0xab8]
range 21: [0x2a26, 0x2a6d]
range 22: [0xab8, 0xac2]
range 23: [0xac2, 0xaea]
range 24: [0xaea, 0xb5b]
range 25: [0xb5b, 0xb61]
range 26: [0xb61, 0xb67]
range 27: [0xb67, 0xb6d]
range 28: [0xb6d, 0xb73]
range 29: [0xb73, 0x10a2]
range 30: [0x10a2, 0x12ba]
range 31: [0x7229b3, 0x722ab2]

What seems to be the problem?

There seems to be a problem with the way relocations are applied to the PC ranges.

Here are the relevant relocations for this DIE - looked up with readelf -r. The offset 0x000023e0 is shown above in the DW_AT_ranges attribute.

Relocation section '.rela.debug_ranges' at offset 0x1f561d40 contains 923208 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
0000000023e0  000100000001 R_X86_64_64       ffffffff81000000 .text + 20b0
0000000023e8  000100000001 R_X86_64_64       ffffffff81000000 .text + 2983
0000000023f0  000100000001 R_X86_64_64       ffffffff81000000 .text + 2983
0000000023f8  000100000001 R_X86_64_64       ffffffff81000000 .text + 298b
000000002400  001400000001 R_X86_64_64       ffffffff82457000 .init.text + 574
000000002408  001400000001 R_X86_64_64       ffffffff82457000 .init.text + 589
000000002410  001400000001 R_X86_64_64       ffffffff82457000 .init.text + 589
000000002418  001400000001 R_X86_64_64       ffffffff82457000 .init.text + 59b
000000002420  001400000001 R_X86_64_64       ffffffff82457000 .init.text + 59b
000000002428  001400000001 R_X86_64_64       ffffffff82457000 .init.text + 5ad
000000002430  001400000001 R_X86_64_64       ffffffff82457000 .init.text + 5ad
000000002438  001400000001 R_X86_64_64       ffffffff82457000 .init.text + 5d9
000000002440  001400000001 R_X86_64_64       ffffffff82457000 .init.text + 5d9
000000002448  001400000001 R_X86_64_64       ffffffff82457000 .init.text + 605
000000002450  001400000001 R_X86_64_64       ffffffff82457000 .init.text + 605
000000002458  001400000001 R_X86_64_64       ffffffff82457000 .init.text + 693
000000002460  001400000001 R_X86_64_64       ffffffff82457000 .init.text + 693
000000002468  001400000001 R_X86_64_64       ffffffff82457000 .init.text + 6e9
000000002470  001400000001 R_X86_64_64       ffffffff82457000 .init.text + 6e9
000000002478  001400000001 R_X86_64_64       ffffffff82457000 .init.text + 73e
000000002480  001400000001 R_X86_64_64       ffffffff82457000 .init.text + 73e
000000002488  001400000001 R_X86_64_64       ffffffff82457000 .init.text + 8d3
000000002490  000100000001 R_X86_64_64       ffffffff81000000 .text + 298b
000000002498  000100000001 R_X86_64_64       ffffffff81000000 .text + 29b9
0000000024a0  000100000001 R_X86_64_64       ffffffff81000000 .text + 722909
0000000024a8  000100000001 R_X86_64_64       ffffffff81000000 .text + 7229b3

As we can see, these relocations are relative to the .text and .init.text sections. If we do the math, adding Addend to Sym.Value, the result is exactly the output from llvm-dwarfdump above.

However, the Go program behaves differently. After looking into it, the culprit seems to be the following, in debug/elf/file.go:

// relocSymbolTargetOK decides whether we should try to apply a
// relocation to a DWARF data section, given a pointer to the symbol
// targeted by the relocation. Most relocations in DWARF data tend to
// be section-relative, but some target non-section symbols (for
// example, low_PC attrs on subprogram or compilation unit DIEs that
// target function symbols), and we need to include these as well.
// Return value is a pair (X,Y) where X is a boolean indicating
// whether the relocation is needed, and Y is the symbol value in the
// case of a non-section relocation that needs to be applied.
func relocSymbolTargetOK(sym *Symbol) (bool, uint64) {
        if ST_TYPE(sym.Info) == STT_SECTION {
                return true, 0
        }
        if sym.Section != SHN_UNDEF && sym.Section < SHN_LORESERVE {
                return true, sym.Value
        }
        return false, 0
}

Not sure if I'm reading this correctly, but the first if statement seems to assume that relocations relative to sections are either absolute relocations, or that the section addresses are always 0. But this is not the case here.

After removing this offending if statement, the output of the above program is correct again and matches the llvm-dwarfdump output:

$ ./test
range 0: [0xffffffff810020b0, 0xffffffff81002983]
range 1: [0xffffffff81002983, 0xffffffff8100298b]
range 2: [0xffffffff82457574, 0xffffffff82457589]
range 3: [0xffffffff82457589, 0xffffffff8245759b]
range 4: [0xffffffff8245759b, 0xffffffff824575ad]
range 5: [0xffffffff824575ad, 0xffffffff824575d9]
range 6: [0xffffffff824575d9, 0xffffffff82457605]
range 7: [0xffffffff82457605, 0xffffffff82457693]
range 8: [0xffffffff82457693, 0xffffffff824576e9]
range 9: [0xffffffff824576e9, 0xffffffff8245773e]
range 10: [0xffffffff8245773e, 0xffffffff824578d3]
range 11: [0xffffffff8100298b, 0xffffffff810029b9]
range 12: [0xffffffff81722909, 0xffffffff817229b3]
range 13: [0xffffffff810029b9, 0xffffffff810029f7]
range 14: [0xffffffff810029f7, 0xffffffff81002a26]
range 15: [0xffffffff824578d3, 0xffffffff824578fb]
range 16: [0xffffffff824578fb, 0xffffffff82457920]
range 17: [0xffffffff82457920, 0xffffffff82457978]
range 18: [0xffffffff82457978, 0xffffffff824579d3]
range 19: [0xffffffff824579d3, 0xffffffff82457aa7]
range 20: [0xffffffff82457aa7, 0xffffffff82457ab8]
range 21: [0xffffffff81002a26, 0xffffffff81002a6d]
range 22: [0xffffffff82457ab8, 0xffffffff82457ac2]
range 23: [0xffffffff82457ac2, 0xffffffff82457aea]
range 24: [0xffffffff82457aea, 0xffffffff82457b5b]
range 25: [0xffffffff82457b5b, 0xffffffff82457b61]
range 26: [0xffffffff82457b61, 0xffffffff82457b67]
range 27: [0xffffffff82457b67, 0xffffffff82457b6d]
range 28: [0xffffffff82457b6d, 0xffffffff82457b73]
range 29: [0xffffffff82457b73, 0xffffffff824580a2]
range 30: [0xffffffff824580a2, 0xffffffff824582ba]
range 31: [0xffffffff817229b3, 0xffffffff81722ab2]

The text was updated successfully, but these errors were encountered:

ianlancetaylor · 2020-08-19T04:00:32Z

I'm a bit surprised that a vmlinux image has any relocations. Why isn't the image fully linked?

vikmik · 2020-08-19T04:10:30Z

So I'm a bit out of my depth here - could that be related to CONFIG_RELOCATABLE ?

(EDIT: removed non-sensical info)

--

Otherwise, the previous iteration of the code in debug/elf (prior to https://go-review.googlesource.com/c/go/+/195679/6/src/debug/elf/file.go ) had a comment saying:

		// There are relocations, so this must be a normal
		// object file, and we only look at section symbols,
		// so we assume that the symbol value is 0.

I'm really no ELF/DWARF expert, but I don't understand the rationale behind "we assume that the symbol value is 0"

vikmik · 2020-08-19T06:53:36Z

Digged into it a bit more - this is due to CONFIG_X86_NEED_RELOCS in the kernel config (indeed related to CONFIG_RELOCATABLE):

https://github.com/torvalds/linux/blob/18445bf405cb331117bc98427b1ba6f12418ad17/arch/x86/Makefile#L198

ifdef CONFIG_X86_NEED_RELOCS
LDFLAGS_vmlinux := --emit-relocs --discard-none
else

so it looks like passing --emit-relocs to the linker can potentially cause breakage, if I can trigger the emission of a DW_AT_ranges attribute. I'll try to provide a self-contained test-case that doesn't depend on a 500MiB ELF file

vikmik · 2020-08-19T07:26:24Z

Ok, here's a standalone repro case that doesn't depend on a vmlinux image: https://github.com/vikmik/dwarf-relocation-golang-bug

This builds 2 binaries, one with --emit-relocs and one without. I added a test script that shows the difference between the 2.
I used a linker script to force the use of multiple code sections (with one hardcoded address), to make sure .debug_ranges / DW_AT_ranges is used.

You should be able to just run make to see the issue

vic@lapdog: ~/go/src/github.com/vikmik/dwarf-relocation-golang-bug$ make
cd testdata && make
make[1]: Entering directory '/home/vic/go/src/github.com/vikmik/dwarf-relocation-golang-bug/testdata'
gcc -g repro.c repro.ld -Wl,--emit-relocs -Wl,--discard-none -o bad
/usr/bin/ld: warning: repro.ld contains output sections; did you forget -T?
gcc -g repro.c repro.ld -o good
/usr/bin/ld: warning: repro.ld contains output sections; did you forget -T?
make[1]: Leaving directory '/home/vic/go/src/github.com/vikmik/dwarf-relocation-golang-bug/testdata'
go build
-------------------------
Testing testdata/good...
Actual ranges:
[0x1125,0x1137)
[0xc0ffee,0xc0fff9)

Ranges derived from Golang's debug/elf + debug/dwarf
[0x1125, 0x1137)
[0xc0ffee, 0xc0fff9)

-------------------------
Testing testdata/bad...
Actual ranges:
[0x1125,0x1137)
[0xc0ffee,0xc0fff9)

Ranges derived from Golang's debug/elf + debug/dwarf
[0xe5, 0xf7)
[0x0, 0xb)
-------------------------

vikmik · 2020-08-19T08:15:33Z

For now I suppose we'll have to fork debug/elf to make our code happy.
However I'm not sure that removing the if statement that checks the symbol type (and return 0 if it's a section) is the right thing to do. It seems sane to me, but I am also fairly confident that I'm missing something :) If you have any advice to share, I'm all ears.

Thanks! Happy to provide more info if needed

thomasdullien · 2020-08-19T10:28:06Z

FWIW, I suspect the reason the kernel is not fully linked is related to KASLR

…cations commit 72ec930 added basic support for relocations, but assumed that the symbol value would be 0, likely because .debug_info always has address == 0 in the ELF section headers. commit df855da added further support for relocations, but explicitly encoded the original assumption that section addresses would be 0. This commit removes that assumption, and adds support for relocations relative to sections that have non-zero addresses. Typically, sections that are part of a LOAD program segment are such sections. For example, .debug_ranges relocations could be relative to .text, which usually has an address > 0. This addresses golang#40879, causing relocations for .debug_ranges to be incorrectly applied.

…cations commit 72ec930 added basic support for relocations, but assumed that the symbol value would be 0, likely because .debug_info always has address == 0 in the ELF section headers. commit df855da added further support for relocations, but explicitly encoded the original assumption that section addresses would be 0. This commit removes that assumption, and adds support for relocations relative to sections that have non-zero addresses. Typically, sections that are part of a LOAD program segment are such sections. For example, .debug_ranges relocations could be relative to .text, which usually has an address > 0. This fixes golang#40879, as relocations for .debug_ranges were incorrectly applied due to the above, ignoring the base section address.

…resses commit 72ec930 added basic support for relocations, but assumed that the symbol value would be 0, likely because .debug_info always has address == 0 in the ELF section headers. commit df855da added further support for relocations, but explicitly encoded the original assumption that section addresses would be 0. This change removes that assumption: all relocations will now be properly computed based on the target symbol value even when that symbol is a section. Typically, sections that are part of a LOAD program segment are such sections. For example, .debug_ranges relocations could be relative to .text, which usually has an address > 0. This fixes golang#40879, as relocations for .debug_ranges were incorrectly applied due to the above, ignoring the base section address.

…resses commit 72ec930 added basic support for relocations, but assumed that the symbol value would be 0, likely because .debug_info always has address == 0 in the ELF section headers. commit df855da added further support for relocations, but explicitly encoded the original assumption that section addresses would be 0. This change removes that assumption: all relocations will now be properly computed based on the target symbol value even when that symbol is a section. Typically, sections that are part of a LOAD program segment are such sections. For example, .debug_ranges relocations could be relative to .text, which usually has an address > 0. Fixes golang#40879 : as relocations for .debug_ranges were incorrectly applied due to the above, ignoring the base section address.

gopherbot · 2020-08-25T22:56:21Z

Change https://golang.org/cl/250559 mentions this issue: debug/elf: support relocations relative to sections with non-zero addresses

…resses commit 72ec930 added basic support for relocations, but assumed that the symbol value would be 0, likely because .debug_info always has address == 0 in the ELF section headers. commit df855da added further support for relocations, but explicitly encoded the original assumption that section addresses would be 0. This change removes that assumption: all relocations will now be properly computed based on the target symbol value even when that symbol is a section. Typically, sections that are part of a LOAD program segment have non-zero addresses. For example, .debug_ranges relocations could be relative to .text, which usually has an address > 0. Fixes golang#40879 : as relocations for .debug_ranges were incorrectly applied due to the above, ignoring the base section address.

…resses commit 72ec930 added basic support for relocations, but assumed that the symbol value would be 0, likely because .debug_info always has address == 0 in the ELF section headers. CL 195679 added further support for relocations, but explicitly encoded the original assumption that section addresses would be 0. This change removes that assumption: all relocations will now be properly computed based on the target symbol value even when that symbol is a section with a non-zero address. Typically, sections that are part of a LOAD program segment have non-zero addresses. For example, .debug_ranges relocations could be relative to .text, which usually has an address > 0. Fixes golang#40879

ianlancetaylor changed the title ~~Bug: debug/elf does not properly apply DWARF relocations~~ debug/elf: does not properly apply DWARF RELA relocations Aug 19, 2020

ianlancetaylor added help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Aug 19, 2020

ianlancetaylor added this to the Backlog milestone Aug 19, 2020

vikmik mentioned this issue Aug 25, 2020

debug/elf: support relocations relative to sections with non-zero addresses #41038

Closed

gopherbot closed this as completed in e01a226 Aug 31, 2020

golang locked and limited conversation to collaborators Aug 31, 2021

gopherbot added the FrozenDueToAge label Aug 31, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

debug/elf: does not properly apply DWARF RELA relocations #40879

debug/elf: does not properly apply DWARF RELA relocations #40879

vikmik commented Aug 19, 2020 •

edited

ianlancetaylor commented Aug 19, 2020

vikmik commented Aug 19, 2020 •

edited

vikmik commented Aug 19, 2020 •

edited

vikmik commented Aug 19, 2020 •

edited

vikmik commented Aug 19, 2020

thomasdullien commented Aug 19, 2020 •

edited

gopherbot commented Aug 25, 2020

debug/elf: does not properly apply DWARF RELA relocations #40879

debug/elf: does not properly apply DWARF RELA relocations #40879

Comments

vikmik commented Aug 19, 2020 • edited

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

What seems to be the problem?

ianlancetaylor commented Aug 19, 2020

vikmik commented Aug 19, 2020 • edited

vikmik commented Aug 19, 2020 • edited

vikmik commented Aug 19, 2020 • edited

vikmik commented Aug 19, 2020

thomasdullien commented Aug 19, 2020 • edited

gopherbot commented Aug 25, 2020

vikmik commented Aug 19, 2020 •

edited

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

vikmik commented Aug 19, 2020 •

edited

vikmik commented Aug 19, 2020 •

edited

vikmik commented Aug 19, 2020 •

edited

thomasdullien commented Aug 19, 2020 •

edited