debug/elf: fatal error: runtime: out of memory #45599

fangxlmr · 2021-04-16T14:01:25Z

What version of Go are you using (`go version`)?

$ go version
go version go1.15.5 linux/amd64

Does this issue reproduce with the latest release?

YES

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/data/leonxlfang/.cache/go-build"
GOENV="/data/leonxlfang/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/data/leonxlfang/go/pkg/mod"
GONOPROXY="git.code.oa.com,git.woa.com"
GONOSUMDB="git.code.oa.com,git.woa.com"
GOOS="linux"
GOPATH="/data/leonxlfang/go"
GOPRIVATE="git.code.oa.com,git.woa.com"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/data/leonxlfang/.local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/data/leonxlfang/.local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.3"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/data/leonxlfang/Test/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build789926490=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Try to parse an ELF file. The file has valid header but other sections are broken somehow.

Example code:

package main

import (
	"debug/elf"
	"fmt"
	"os"
)

func main() {
	// pathname := "/bin/ls"
	pathname := "./pvrsrvkm.ko"
	f1, err := os.Open(pathname)
	if err != nil {
		panic(fmt.Sprintf("error opening %s: %s", pathname, err))
	}
	f2, err := elf.NewFile(f1)
	if err != nil {
		panic(fmt.Sprintf("error creating new ELF struct: %s", err))
	}
	for _, section := range f2.Sections {
		fmt.Println(section.Name)
	}
}

Example file: pvrsrvkm.ko.zip

What did you expect to see?

NO PANIC AT LEAST! Or panic with informative messages.

readelf command neither panics nor causes out of memory.

readelf pvrsrvkm.ko Output

ELF Header: Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 Class: ELF64 Data: 2's complement, little endian Version: 1 (current) OS/ABI: UNIX - System V ABI Version: 0 Type: REL (Relocatable file) Machine: AArch64 Version: 0x1 Entry point address: 0x0 Start of program headers: 0 (bytes into file) Start of section headers: 804368 (bytes into file) Flags: 0x0 Size of this header: 64 (bytes) Size of program headers: 0 (bytes) Number of program headers: 0 Size of section headers: 64 (bytes) Number of section headers: 39 Section header string table index: 36 readelf: pvrsrvkm.ko: Warning: Section 0 has an out of range sh_link value of 1572864 readelf: pvrsrvkm.ko: Warning: Section 1 has an out of range sh_link value of 3145728 readelf: pvrsrvkm.ko: Warning: Section 2 has an out of range sh_link value of 16842752 readelf: pvrsrvkm.ko: Warning: Section 3 has an out of range sh_link value of 6291456 readelf: pvrsrvkm.ko: Warning: Section 4 has an out of range sh_link value of 7864320 readelf: pvrsrvkm.ko: Warning: Section 5 has an out of range sh_link value of 16842752 readelf: pvrsrvkm.ko: Warning: Section 6 has an out of range sh_link value of 11010048 readelf: pvrsrvkm.ko: Warning: Section 7 has an out of range sh_link value of 12582912 readelf: pvrsrvkm.ko: Warning: Section 8 has an out of range sh_link value of 16842752 readelf: pvrsrvkm.ko: Warning: Section 9 has an out of range sh_link value of 15728640 readelf: pvrsrvkm.ko: Warning: Section 10 has an out of range sh_link value of 17301504 readelf: pvrsrvkm.ko: Warning: Section 11 has an out of range sh_link value of 16842752 readelf: pvrsrvkm.ko: Warning: Section 12 has an out of range sh_link value of 20447232 readelf: pvrsrvkm.ko: Warning: Section 13 has an out of range sh_link value of 22020096 readelf: pvrsrvkm.ko: Warning: Section 14 has an out of range sh_link value of 16842752 readelf: pvrsrvkm.ko: Warning: Section 15 has an out of range sh_link value of 7864320 readelf: pvrsrvkm.ko: Warning: Section 16 has an out of range sh_link value of 106430464 readelf: pvrsrvkm.ko: Warning: Section 17 has an out of range sh_link value of 16842752 readelf: pvrsrvkm.ko: Warning: Section 18 has an out of range sh_link value of 33030144 readelf: pvrsrvkm.ko: Warning: Section 21 has an out of range sh_link value of 620756992 readelf: pvrsrvkm.ko: Warning: Section 23 has an out of range sh_link value of 620756992 readelf: pvrsrvkm.ko: Warning: Section 25 has an out of range sh_link value of 620756992 readelf: pvrsrvkm.ko: Warning: Section 27 has an out of range sh_link value of 620756992 readelf: pvrsrvkm.ko: Warning: Section 30 has an out of range sh_link value of 620756992 readelf: pvrsrvkm.ko: Error: Reading 0x2000000000 bytes extends past end of file for string table

Section Headers:
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[ 0] 00010000: <unkn 0000000000100000 1000001010000
0000000195640000 0000000000000000 xop 1572864 0 470626161077059584
[ 1] NULL 0000000002b40000 00300000
0010000001010000 0010000001010000 xox 3145728 0 3670016
[ 2] NULL 0598000001010000 00000000
0000000000480000 0000000000500000 o 16842752 524288 2359361536
[ 3] 00010000: <unkn 0000000000580000 1000001010000
0000000194180000 0000000000000000 xxop 6291456 0 470626161077059584
[ 4] NULL 0000000001c00000 00780000
0010000001010000 0010000001010000 xox 7864320 0 8388608
[ 5] NULL 0598000001010000 00000000
0000000000900000 0000000000980000 xo 16842752 524288 2393505792
[ 6] 00010000: <unkn 0000000000a00000 1000001010000
00000001a4b80000 0000000000000000 xop 11010048 0 470626161077059584
[ 7] NULL 0000000003540000 00c00000
0010000001010000 0010000001010000 xox 12582912 0 13107200
[ 8] NULL 0598000001010000 00000000
0000000000d80000 0000000000e00000 o 16842752 524288 2393505792
[ 9] 00010000: <unkn 0000000000e80000 1000001010000
00000001a2a00000 0000000000000000 xxop 15728640 0 470626161077059584
[10] NULL 00000000051c0000 01080000
0010000001010000 0010000001010000 xox 17301504 0 17825792
[11] NULL 0598000001010000 00000000
0000000001200000 0000000001280000 xo 16842752 524288 2393505792
[12] 00010000: <unkn 0000000001300000 1000001010000
00000001a08c0000 0000000000000000 xop 20447232 0 470626161077059584
[13] NULL 0000000003e40000 01500000
0010000001010000 0010000001010000 xox 22020096 0 22544384
[14] NULL 0598000001010000 00000000
0000000000000000 0000000000280000 o 16842752 786432 0
[15] 000c0000: <unkn 0000000000500000 c000001010000
0000000000580000 0000000000400000 o 7864320 0 3377699737370624
[16] NULL 0000000000280000 00000000
000e000001010000 000e000001010000 xoxx 106430464 0 524288
[17] NULL 000e000001010000 07c80000
0000000000180000 0000000000200000 o 16842752 917504 139460608
[18] 000e0000: <unkn 0000000001200000 602000001010000
0000000000000000 0000000000000000 xo 33030144 0 398287092062420992
[19] NULL 0000000000000000 00000000
0000000000000000 0000000000000000 0 0 0
[20] 00010000: <unkn 0000000000000000 00400000
00000003ec9c0000 0000000000000000 xx 0 0 262144
[21] 00040000: <unkn 0000000000000000 70a0d900000
000004e738000000 0000000018000000 o 620756992 16777216 134217728
[22] 01000000: <unkn 0000000000000000 3ecdc000000
0000000654000000 0000000000000000 o 0 0 67108864
[23] 04000000: <unkn 0000000000000000 bf1c8000000
0000000d38000000 0000000018000000 p 620756992 50331648 134217728
[24] 01000000: <unkn 0000000000000000 3f330000000
0000000084000000 0000000000000000 o 0 0 67108864
[25] 04000000: <unkn 0000000000000000 bff00000000
0000000150000000 0000000018000000 p 620756992 83886080 134217728
[26] 01000000: <unkn 0000000000000000 3f3b4000000
0000000474000000 0000000000000000 o 0 0 67108864
[27] 04000000: <unkn 0000000000000000 c0050000000
0000000690000000 0000000018000000 p 620756992 117440512 134217728
[28] 07000000: <unkn 0000000000000000 3f828000000
0000000024000000 0000000000000000 o 0 0 67108864
[29] 01000000: <unkn 0000000028000000 3f850000000
0000000020000000 0000000000000000 o 0 0 134217728
[30] 04000000: <unkn 0000000000000000 c06e0000000
0000000060000000 0000001800000000 p 620756992 218103808 34359738378
[31] 00000081: <unkn 0000000000000000 3f87000000000
000028c800000000 0000000000000000 Wx 0 0 34359738368
[32] 0000007c: <unkn 0000000000000000 c074000000000
000029b800000000 0000001800000000 Xx 0 37 34359738380
[33] 00000089: <unkn 0000000000000000 4213800000000
0001451700000000 0000000100000000 Wxxx 0 0 4294967296
[34] 00000098: <unkn 0000000000000000 5664f00000000
0000002900000000 0000000000000000 Wx 0 0 4294967296
[35] 000000aa: <unkn 0000000000000000 5667800000000
000000f000000000 0000000000000000 Wx 0 0 34359738368
[36] 000000b8: <unkn 0000000000000000 5676800000000
0000002000000000 0000000000000000 Wx 0 0 34359738368
[37] 000000b3: <unkn 0000000000000000 c30f800000000
0000004800000000 0000001800000000 Xx 0 37 34359738385
[38] 000000c0: <unkn 0000000000000000 5678800000000
0000007000000000 0000000000000000 Wx 0 0 34359738368
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
L (link order), O (extra OS processing required), G (group), T (TLS),
C (compressed), x (unknown), o (OS specific), E (exclude),
p (processor specific)

There are no section groups in this file.

There are no program headers in this file.

There are no relocations in this file.

The decoding of unwind sections for machine type AArch64 is not currently supported.

No version information found in this file.

What did you see instead?

Program panics with stack trace

fatal error: runtime: out of memory

runtime stack:
runtime.throw(0x4db49c, 0x16)
        /data/leonxlfang/.local/go/src/runtime/panic.go:1117 +0x72
runtime.sysMap(0xc004000000, 0x2000000000, 0x5a6ab0)
        /data/leonxlfang/.local/go/src/runtime/mem_linux.go:169 +0xc6
runtime.(*mheap).sysAlloc(0x58e960, 0x2000000000, 0x4293b7, 0x58e968)
        /data/leonxlfang/.local/go/src/runtime/malloc.go:729 +0x1e5
runtime.(*mheap).grow(0x58e960, 0x1000000, 0x0)
        /data/leonxlfang/.local/go/src/runtime/mheap.go:1346 +0x85
runtime.(*mheap).allocSpan(0x58e960, 0x1000000, 0xc000060100, 0x7f92b3607d30)
        /data/leonxlfang/.local/go/src/runtime/mheap.go:1173 +0x609
runtime.(*mheap).alloc.func1()
        /data/leonxlfang/.local/go/src/runtime/mheap.go:910 +0x59
runtime.systemstack(0x462994)
        /data/leonxlfang/.local/go/src/runtime/asm_amd64.s:379 +0x66
runtime.mstart()
        /data/leonxlfang/.local/go/src/runtime/proc.go:1246

goroutine 1 [running]:
runtime.systemstack_switch()
        /data/leonxlfang/.local/go/src/runtime/asm_amd64.s:339 fp=0xc000057a88 sp=0xc000057a80 pc=0x462ac0
runtime.(*mheap).alloc(0x58e960, 0x1000000, 0x101, 0x40)
        /data/leonxlfang/.local/go/src/runtime/mheap.go:904 +0x85 fp=0xc000057ad8 sp=0xc000057a88 pc=0x425065
runtime.(*mcache).allocLarge(0x7f92b35fd108, 0x2000000000, 0x7f92b3600101, 0x203000)
        /data/leonxlfang/.local/go/src/runtime/mcache.go:224 +0x97 fp=0xc000057b30 sp=0xc000057ad8 pc=0x415877
runtime.mallocgc(0x2000000000, 0x4bb3c0, 0x7f92b3608801, 0x10)
        /data/leonxlfang/.local/go/src/runtime/malloc.go:1078 +0x925 fp=0xc000057bb8 sp=0xc000057b30 pc=0x40cf85
runtime.makeslice(0x4bb3c0, 0x2000000000, 0x2000000000, 0x577e00)
        /data/leonxlfang/.local/go/src/runtime/slice.go:98 +0x6c fp=0xc000057be8 sp=0xc000057bb8 pc=0x4489ec
debug/elf.(*Section).Data(0xc000133200, 0xc00007a9c0, 0x4facb8, 0x5a5520, 0x4b67e0, 0xc00001f400)
        /data/leonxlfang/.local/go/src/debug/elf/file.go:105 +0x45 fp=0xc000057c50 sp=0xc000057be8 pc=0x4ab405
debug/elf.NewFile(0x4fa090, 0xc00000e028, 0x0, 0x7f9200000000, 0xc00000e028)
        /data/leonxlfang/.local/go/src/debug/elf/file.go:456 +0xff6 fp=0xc000057ed0 sp=0xc000057c50 pc=0x4ac8b6
main.main()
        /data/leonxlfang/Test/elf.go:16 +0xee fp=0xc000057f88 sp=0xc000057ed0 pc=0x4ae8ae
runtime.main()
        /data/leonxlfang/.local/go/src/runtime/proc.go:225 +0x256 fp=0xc000057fe0 sp=0xc000057f88 pc=0x4355f6
runtime.goexit()
        /data/leonxlfang/.local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc000057fe8 sp=0xc000057fe0 pc=0x4647c1
exit status 2

The text was updated successfully, but these errors were encountered:

fangxlmr · 2021-04-16T14:01:58Z

Root cuase of this specific issue is pretty obvious, but I was wondering if debug package will plan to introduce more strict checking and validation to prevent this (and other similar problems) from happening?

mknyszek · 2021-04-16T15:10:00Z

In that panic, the debug/elf package is trying to allocator a slice of size ~128 GiB. Why? I'm not sure. That might be a bug.

mknyszek · 2021-04-16T15:10:27Z

CC @ianlancetaylor via https://dev.golang.org/owners

ianlancetaylor · 2021-04-19T17:54:30Z

It's a corrupt ELF file. The question is to what extent debug/elf should protect itself against corrupt files. It's hard to know what limits to set, as ELF files can indeed be very very large.

I'm certainly open to suggestions.

sanzhang098 · 2021-04-20T02:41:46Z

We do have some choice:

Using strict checking in debug/elf which leads to complex codes in there
Choosing another elf parser for untrusted input elf

prattmic · 2021-04-20T12:12:28Z

When I wrote a loader for untrusted ELF files in Go a few years ago, I avoided using debug/elf for exactly this reason (or, rather, used it only for types/constants). debug/elf is not currently suited to parsing corrupt/malicious ELF files. This is only one of numerous places where it passed values directly from the file to make, easily causing such OOMs.

Should debug/elf try to be safe against malicious ELFs? I'm not sure; that is a pretty high bar to meet, and safety also limits the API to some extent. For example, the crashing function here, Section.Data is not really safe even for a legitimate, very large, ELF, which could contain a multi-GB section that will OOM the process when read in entirety.

ianlancetaylor · 2021-04-20T19:46:41Z

I am in favor of better checking in debug/elf, where possible.

It occurs to me that in this particular case we can compare the number of sections with the size of the file. If the file is too small, we know it is corrupt before we try to allocate memory.

fangxlmr · 2021-04-21T07:16:01Z

I am in favor of better checking in debug/elf, where possible.

Agree. I'd like have debug/elf at least NO (unpredictable) PANIC as well. But as above post said, to what degree and how?

It occurs to me that in this particular case we can compare the number of sections with the size of the file. If the file is too small, we know it is corrupt before we try to allocate memory.

Yeah, I thought this before as well. But the thing is NewFile(r io.ReaderAt) (*File, error) accepts a io.ReaderAt inerface, so how do we exactly know the file size? Read until EOF? Sounds no good.

ianlancetaylor · 2021-04-21T20:12:27Z

We can have NewFile do a type assertion to see if it is given something that supports Stat, as will be true when NewFile is called from elf.Open. If we want to get really fancy we can use a type assertion to see if it supports Seek, as is likely, and then seek to the end to find the offset.

gopherbot · 2021-11-08T10:15:29Z

Change https://golang.org/cl/362255 mentions this issue: debug/elf: read section contents incrementally in Section.Data()

ianlancetaylor · 2022-03-18T01:06:09Z

More or less a dup of #47653.

gopherbot · 2022-06-23T16:11:50Z

Change https://go.dev/cl/413775 mentions this issue: debug/elf: avoid elf.Open crashing on malformed files

gopherbot · 2022-06-23T17:51:21Z

Change https://go.dev/cl/408679 mentions this issue: debug/elf: use saferio to read section data

mknyszek added the NeedsInvestigation label Apr 16, 2021

mknyszek added this to the Backlog milestone Apr 16, 2021

ianlancetaylor modified the milestones: Backlog, Unplanned Apr 19, 2021

randall77 mentioned this issue Jul 28, 2021

debug/elf: parsing Malware Elf File causes a panic in debug/elf.NewFile #47445

Closed

fweimer-rh mentioned this issue Nov 8, 2021

debug/elf: read section contents incrementally in Section.Data() #49437

Closed

DzmitryKN mentioned this issue Jun 23, 2022

debug/elf: avoid elf.Open crashing on malformed files #53517

Closed

gopherbot added the compiler/runtime label Jul 13, 2022

mknyszek added this to Go Compiler / Runtime Jul 13, 2022

mknyszek removed this from Go Compiler / Runtime Jul 13, 2022

gopherbot closed this as completed in 7367aed Aug 11, 2022

This was referenced Sep 9, 2022

archive, image, debug, encoding, x/net/html: DO NOT PANIC #47653

Open

debug/elf: timeout/oom in NewFile #54967

Closed

golang locked and limited conversation to collaborators Aug 11, 2023

gopherbot added the FrozenDueToAge label Aug 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

debug/elf: fatal error: runtime: out of memory #45599

debug/elf: fatal error: runtime: out of memory #45599

fangxlmr commented Apr 16, 2021

fangxlmr commented Apr 16, 2021

mknyszek commented Apr 16, 2021

mknyszek commented Apr 16, 2021

ianlancetaylor commented Apr 19, 2021

sanzhang098 commented Apr 20, 2021

prattmic commented Apr 20, 2021

ianlancetaylor commented Apr 20, 2021

fangxlmr commented Apr 21, 2021 •

edited

Loading

ianlancetaylor commented Apr 21, 2021

gopherbot commented Nov 8, 2021

ianlancetaylor commented Mar 18, 2022

gopherbot commented Jun 23, 2022

gopherbot commented Jun 23, 2022

debug/elf: fatal error: runtime: out of memory #45599

debug/elf: fatal error: runtime: out of memory #45599

Comments

fangxlmr commented Apr 16, 2021

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

fangxlmr commented Apr 16, 2021

mknyszek commented Apr 16, 2021

mknyszek commented Apr 16, 2021

ianlancetaylor commented Apr 19, 2021

sanzhang098 commented Apr 20, 2021

prattmic commented Apr 20, 2021

ianlancetaylor commented Apr 20, 2021

fangxlmr commented Apr 21, 2021 • edited Loading

ianlancetaylor commented Apr 21, 2021

gopherbot commented Nov 8, 2021

ianlancetaylor commented Mar 18, 2022

gopherbot commented Jun 23, 2022

gopherbot commented Jun 23, 2022

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

fangxlmr commented Apr 21, 2021 •

edited

Loading