Could you clarify what the problem is?

the objfile tool fails to located the pclntab and fails to print embedded symbols.

What is "the objfile tool"? go tool objdump? And what is "embedded symbols"?

go/src/cmd/internal/objfile/objfile.go
return gosym.NewTable(symtab, gosym.NewLineTable(pclntab, textStart))

That symtab is the .gosymtab section, instead of .symtab section. How do you strip the binary so it has strips that section? Also, that section is actually empty anyway in recent versions of Go.

Could you clarify what program/command you run that you see unexpected result? Thanks.

Sure, sorry my wording wasn't exact. The issue is this:

go tool objdump ./mtisfun_win_stripped
objdump: disassemble ./fmtisfun_win_stripped: no runtime.pclntab symbol found

The binary in question has its .symtab section zeroed out (sometimes this is referred to as stripping symbols):

The reason this matters is because internally the objfile package attempts to resolve the pclntab by using this .symtab section:

which calls (for PE's)

That symbol resolution fails, and therefore no golang symbols are parsed and the objdump tool faults. This is a poor failure case though as the pclntab section is (usually) still there! It can be located by performing a bytescan over the .text section for the magic pclntab magic. When this byte scan is done the objdump tool is able to work perfectly fine even on binaries where symbols are stripped like this.

The patch is fairly simple, in objfile.go PCLineTable():

	textStart, symtab, pclntab, err := e.raw.pcln()
	if err != nil {
		textStart, textData, err := e.raw.text()
		// little endian go12magic
		pclntab_idx := bytes.Index(textData, []byte("\xFB\xFF\xFF\xFF\x00\x00"))
		if pclntab_idx == -1 {
			// big endian go12magic
			pclntab_idx = bytes.Index(textData, []byte("\xFF\xFF\xFF\xFB\x00\x00"))
			if pclntab_idx == -1 {
				// little endian go116magic
				pclntab_idx = bytes.Index(textData, []byte("\xFA\xFF\xFF\xFF\x00\x00"))
				if pclntab_idx == -1 {
					// big endian go116magic
					pclntab_idx = bytes.Index(textData, []byte("\xFF\xFF\xFF\xFA\x00\x00"))
					if pclntab_idx == -1 {
						return nil, err
					}
				}
			}
		}

		// The symbol table of the PE is stripped, but we still have the pclntab!
		fmt.Printf("found pclntab via signature! %X", textStart+uint64(pclntab_idx))

                // we don't know the epclntab but that's fine, the parser won't read extra!
		return gosym.NewTable([]byte{}, gosym.NewLineTable(textData[pclntab_idx:], textStart))
	}

As an aside, if the members of the LineTable:

and the symtab Table:

could be capitalized to be exported I would appreciate it! I use this for some internal security tooling and have to maintain a set of patches to export these internal details.

How exactly was your symbol table stripped? Can you show us the exact commands you used starting with building your binary?
What platform are you on?

I'm not a fan of finding sections by looking for magic numbers. Magic numbers are useful if you know where they are and are just verifying them (e.g. at the start of file), but looking for them among a large corpus has a serious false positive problem. For example, the instruction sequence:

	movq	0xfbffffff(%rcx), %rcx
	addb	%al, (%rax)

Has one of your magic sequences in it.

I work in security and often deal with malicious samples that are stripped by bad actors, this isn't a process i do. However you can generate a binary without symbols via: go build -ldflags="-s -w" <files>. I agree that searching for magic bytes is not ideal, of course there are false positives. This particular magic isn't exactly the greatest, i would argue it needs more entropy and should probably be a little longer, but it's possible to deal with false positives like this as the parser will simply fail and we can try all candidates until one works. It's also know that this section is within the .text section which avoid a whole host of problems with data colliding.

A future version of go could be really nice if it added a magic for the moduledata structure too :)

What platform are you on?

Still need an answer to this.

However you can generate a binary without symbols via: go build -ldflags="-s -w"

Works fine for me (darwin/amd64, go1.13.6):

$ go build -ldflags="-s -w" hello.go
$ go tool objdump hello
... lots of disassembly ...

Why use go tool objdump instead of binutil's objdump for this? If it was generated by a malicious actor, how do you even know the stripped binary is a Go binary?

On ELF, we look for ".gosymtab" section (NOT ".symtab"), instead of "runtime.symtab" symbol ( https://tip.golang.org/src/cmd/internal/objfile/elf.go#L67 ). Can we do the same for PE?

I also don't think that looking for magic numbers are the right answer here.

@randall77 I am on windows/linux but it's kind of irrelevant here as the symbol table (.gosymtab, .symtab, whatever) can be stripped and the pclntab still be present. This issue highlights that the symbol table (.gosymtab, .symtab, whatever) is used to locate the .pclntab section which can fail. This failure to locate the pclntab doesn't mean it isn't there however, and we can find it to still make objdump useful.

Why use go tool objdump instead of binutil's objdump for this

I specifically care about getting the pclntab metadata here as it gives function names and start/stop boundaries.

how do you even know the stripped binary is a Go binary

as a reverse engineer it is incredibly easily to tell by just looking at the assembly patterns in IDA PRO

there's at least two unhandled failure cases:

1. When the symbol table itself is fully missing, ex: entire .gosymtab (ELF) or .symtab (PE) is just straight up missing
2. When the symbol table is there, but zero/malformed and the symbols cannot be located within these tables. You will get different errors for both of these cases

Now, the symbol table being corrupt/stomped doesn't matter as the pclntab is independent of that.

Please don't confuse ".gosymtab" and ".symtab". They are always two different sections, on any platform (PE/ELF/Mach-O/etc.). What strip tool do you use? I don't think it should strip ".gosymtab" section on any platform. (However, the section should be empty anyway.)

"runtime.symtab" is a symbol, which should be at the beginning of the ".gosymtab" section. Please be careful with the names. Thanks.

@randall77 I am on windows/linux but it's kind of irrelevant here

It's very relevant. Now that I know "windows", I can reproduce:

$ GOOS=windows go build -ldflags="-s -w" hello.go
$ go tool objdump hello.exe
objdump: disassemble hello.exe: no runtime.pclntab symbol found

@cherrymui I am simply using the linker flags i specified earlier. I am sorry if I am not using the correct terms, I am referring to .gosymtab as a section for linux elf, and .symtab as a section for PE. These are what i refer to as the symbol tab as it seems GO uses them to store symbols inside of (such as to locate runtime.pclntab)

@randall77 same happens for linux but it fails finding the symbol table entirely:

go tool objdump hello
objdump: disassemble hello: no symbol section

I don't think I understand the motivation for fixing this at all. For the Go project generally, that is.
A user tried to remove symbolic information from their binary. We should respect that wish and not intentionally resurrect that symbolic information despite their wishes.

The fact that they are a malicious user is unfortunate, but not IMO enough motivation to have our general-purpose tools sidestep someone's attempt to remove symbolic information.
Besides, this is an arms race. As soon as we implement this, the obfuscator will notice and just zero out more stuff. I don't think we want to go down that road. Consider yourself lucky that our tools don't show that symbolic information by default but it is still hidden in there somewhere for you.

(It might be better to have go tool objdump dump the whole text section as one big assembly function instead of erroring out, though.)

I am referring to .gosymtab as a section for linux elf, and .symtab as a section for PE

They should not be the same section. They should serve completely different purpose. If they are, it seems a bug in the linker.

to locate runtime.pclntab

What I'm proposing is that we should not try to locate runtime.pclntab at all. We should put the pclntab in a separate section and locate that section instead, which we do for ELF and Mach-O.

(It might be better to have go tool objdump dump the whole text section as one big assembly function instead of erroring out, though.)

Agreed. This seems what the system objdump does, at least on some platforms.

@randall77 fair enough, i think any tool should try as hard as possible before it fails, but that's not a philosophical direction i control. There's a real bug in the linux case though i think as the .gosymtab section being present or not is irrelevant (and it errors early)

The windows case is a bit trickier, @cherrymui your suggestion of going to a section based discovery vs a symbol based discovery I like personally. It avoids this co-dependancy of symbols <-> pclntab and also bring the windows file format inline with what linux and macho do

Closing as it's been shown there's no interest in resolving this. Hopefully this serves as documentation to some future person though. ✌️