If you have time, can you see if this is a regression since Go 1.9.x? That is, was Go 1.9.x better in this regard?

It isn't really an option, because last time my computer became unresponsive and I had to force shutdown, so I wouldn't like to be doing this a second time.

In the meantime, I tested this on a Linux/amd64 machine and it works as intended, producing the assembly code quickly.
This leads me into thinking that there is some sort of memory leak in the objdump code in Windows.

Interesting. Or maybe a bug in Windows linking.

If you're willing to risk/experiment a little, you could try again on windows with:

package main

const big = 10

func main() {
	x := [big]int{1}
	_ = x
}

And see what happens as you increase the size of big little by little.

cc @alexbrainman

Then, I used 'go tool objdump ... > out.s' on it, the memory (8GB) usage started climbing (reaching 100%) and then my computer froze.
https://play.golang.org/p/EK8zDu5vVQN

Sure. I can reproduce this. My computer gets pretty slow, but I kill objdump with ctrl+c and then everything goes back to normal. While frozen my CPU is around 15%, but my both memory and disk usage go full. I have Windows 10 with 8G of memory and flimsy hard disk.

If you're willing to risk/experiment a little, you could try again on windows with:

The last successful run of this program was with big of 1000000. I used github.com/alexbrainman/time to measure the run, and it outputs:

real	0m1.074s
user	0m0.016s
sys	0m0.000s

I do not have time to debug this today or in the near future. But happy to try suggestions.

Thank you.

Alex

I can go to ~2000000 (with memory usage at 80%) and it takes ~30s.

Sounds like the next step might be to add a -memprofile flag to objdump so that we can find out where the allocs are coming from. I’m AFK for a bit, but can do so in a bit if no one else beats me to it.

I have a lead on optimizing cmd/objdump generally. I'll mail some CLs soon, but I have a lot on my plate and not much Go time.

In the meantime, I noticed something windows-specific that perhaps Alex could take a look at.

$ GOOS=darwin GOARCH=amd64 go build -o big_darwin bigobj.go
$ GOOS=windows GOARCH=amd64 go build -o big_windows bigobj.go
$ go tool objdump big_windows | wc -l
  315135
$ go tool objdump big_darwin | wc -l
   72496
$ ls -l big_darwin big_windows 
-rwxr-xr-x  1 josh  staff  1403040 Apr 11 12:57 big_darwin
-rwxr-xr-x  1 josh  staff  1371648 Apr 11 12:57 big_windows

Though the darwin and windows executables are similar in size, the windows objdump output is 4x larger. Basically all of the difference appears to be output for a TEXT type.*(SB) symbol, which is present on windows and not on darwin. Seems like that symbol should be DATA or RODATA, not TEXT; maybe the windows linker is getting this wrong?

I have a lot on my plate and not much Go time.

Same here. I will try and debug this on weekend.

Alex

Change https://golang.org/cl/106697 mentions this issue: os: allocate buffer lazily in Expand

Change https://golang.org/cl/106798 mentions this issue: debug/gosym: intern LineTable strings

Oops—forgot to tag this issue from https://go-review.googlesource.com/c/go/+/106978

@josharian I am not convinced that the problem is objdump or windows. I think the problem is how Go compiler generates the executable.

$ cat a.go
package main

const big = 1000000

func main() {
        x := [big]int{1}
        _ = x
}
$ go build -o a && go tool nm -size -sort=size a | head -n5
  478700     189114 r runtime.pclntab
  4b59a0      65744 D runtime.trace
  4b1ae0      16064 D runtime.semtable
  4ae460      13944 D runtime.mheap_
  4ac460       8192 D runtime.timers
go tool nm: signal: broken pipe
$ cat a.go
package main

const big = 10000000

func main() {
        x := [big]int{1}
        _ = x
}
$ go build -o a && go tool nm -size -sort=size a | head -n5
  477f00   80000000 R main.statictmp_0
 50c3b20     189122 r runtime.pclntab
 51009a0      65744 D runtime.trace
 50fcae0      16064 D runtime.semtable
 50f9460      13944 D runtime.mheap_
go tool nm: signal: broken pipe
$ cat a.go
package main

const big = 100000000

func main() {
        x := [big]int{1}
        _ = x
}
$ go build -o a && go tool nm -size -sort=size a | head -n5
  477f00  800000000 R main.statictmp_0
2ff68f20     189122 r runtime.pclntab
2ffa69a0      65744 D runtime.trace
2ffa2ae0      16064 D runtime.semtable
2ff9f460      13944 D runtime.mheap_
go tool nm: signal: broken pipe
$

Note how, as I make big const bigger by, we see growing in size main.statictmp_0 symbols appearing in executable. Is that expected? Any command will stop working, if we are trying to use the command to generate or parse file that is larger and larger.

The main.statictmp_0 symbols is generated by cmd/compile command - you can see it if you run go tool compile -o /dev/null -S a.go. I don't know much about cmd/compile. Perhaps you have some ideas why it is happening.

Alex

That is #24724. But objdump only processes text symbols (types T and t), and main.statictmp_0 is a readonly data symbol (R). So it is not responsible for this particular issue.

I have some as-yet-mailed CLs to significantly cut objdump’s memory usage. They will help. But the reason windows is currently so much worse than Linux is because the type.* symbol is incorrectly marked as text; of that I am reasonably confident.

Actually, if you would, can you try the same comparison I did, but running on windows: cross compile the code for both darwin and windows and objdump it, and compare the speeds?

Actually, if you would, can you try the same comparison I did, but running on windows: cross compile the code for both darwin and windows and objdump it, and compare the speeds?

You are correct, objdump is much slower on windows executable:

c:\Users\Alex\dev\src\issue\go\24725>set GOOS=darwin

c:\Users\Alex\dev\src\issue\go\24725>go build -o big_darwin a.go

c:\Users\Alex\dev\src\issue\go\24725>set GOOS=windows

c:\Users\Alex\dev\src\issue\go\24725>go build -o big_windows a.go

c:\Users\Alex\dev\src\issue\go\24725>go tool objdump big_darwin > NUL

c:\Users\Alex\dev\src\issue\go\24725>go tool objdump big_windows > NUL

c:\Users\Alex\dev\src\issue\go\24725>dir big_*
 Volume in drive C has no label.
 Volume Serial Number is 9012-A870

 Directory of c:\Users\Alex\dev\src\issue\go\24725

14/04/2018  03:53 PM        81,414,560 big_darwin
14/04/2018  03:54 PM        81,383,424 big_windows
               2 File(s)    162,797,984 bytes
               0 Dir(s)  398,134,525,952 bytes free

c:\Users\Alex\dev\src\issue\go\24725>

I will see if I can understand why.

Alex

I will see if I can understand why.

I explained the difference between the two binaries here: #24725 (comment)

As for why that 4x size difference impacts memory usage so much, that's the partly fault of text/tabwriter. I'm working on that.

I explained the difference between the two binaries here: #24725 (comment)

I can see these symbols

44cb00     114848 T runtime.rodata
44cb00     114848 T type.*        
44cb00     114848 T runtime.types 

on windows. And they are not present on darwin. I will try to understand why linker is adding these symbols on windows.

Alex

Seems like that symbol should be DATA or RODATA, not TEXT; maybe the windows linker is getting this wrong?

Go linker can really put symbols into one of 2 sections: .text or .data

a:     file format pei-x86-64

Sections:
Idx Name          Size      VMA               LMA               File off  Algn
  0 .text         04ceaef7  0000000000401000  0000000000401000  00000600  2**4
                  CONTENTS, ALLOC, LOAD, READONLY, CODE, DATA
  1 .data         00002200  00000000050ec000  00000000050ec000  04ceb600  2**4
                  CONTENTS, ALLOC, LOAD, DATA
  2 .debug_abbrev 000001b5  000000000510c000  000000000510c000  04ced800  2**0
                  CONTENTS, READONLY, DEBUGGING
  3 .debug_line   0000c262  000000000510d000  000000000510d000  04ceda00  2**0
                  CONTENTS, READONLY, DEBUGGING
  4 .debug_frame  00009b2c  000000000511a000  000000000511a000  04cf9e00  2**0
                  CONTENTS, READONLY, DEBUGGING
  5 .debug_pubnames 000021ae  0000000005124000  0000000005124000  04d03a00  2**0
                  CONTENTS, READONLY, DEBUGGING
  6 .debug_pubtypes 0000746e  0000000005127000  0000000005127000  04d05c00  2**0
                  CONTENTS, READONLY, DEBUGGING
  7 .debug_gdb_scripts 00000027  000000000512f000  000000000512f000  04d0d200  2**0
                  CONTENTS, READONLY, DEBUGGING
  8 .debug_info   00037e3a  0000000005130000  0000000005130000  04d0d400  2**0
                  CONTENTS, READONLY, DEBUGGING
  9 .debug_loc    0003bfc2  0000000005168000  0000000005168000  04d45400  2**0
                  CONTENTS, READONLY, DEBUGGING
 10 .debug_ranges 0000e110  00000000051a4000  00000000051a4000  04d81400  2**0
                  CONTENTS, READONLY, DEBUGGING
 11 .idata        000003c4  00000000051b3000  00000000051b3000  04d8f600  2**2
                  CONTENTS, ALLOC, LOAD, DATA
 12 .symtab       0000cfe7  00000000051b4000  00000000051b4000  04d8fa00  2**2
                  CONTENTS, READONLY

type.* is in .text section at this moment. Should we put it into .data ?

Alex

Change https://golang.org/cl/106979 mentions this issue: text/tabwriter: reduce allocations from tracking cells

type.* is in .text section at this moment. Should we put it into .data ?

I believe so. .text is usually used for executable code; .data is used for, well, data. type.* is data.

I believe so. .text is usually used for executable code; .data is used for, well, data. type.* is data.

.text section stores code and read-only data on windows. .data stores data that changes during program execution - initialized and zero-initialized.

Why do you think objdump is slow because type.* symbol is in .text section? I still did not have time to actually see objdump performance profile.

Alex

So windows doesn't have an rodata section?
That's where type.* goes on linux.

Basically all of the difference appears to be output for a TEXT type.*(SB) symbol, which is present on windows and not on darwin.

I can see it myself now. And there are of couple other symbols that are huge:

TEXT type.*(SB)
TEXT runtime.rodata(SB)
TEXT runtime.types(SB)

Obviously, they are not code, so objdump should not attempt disassembling them.

Seems like that symbol should be DATA or RODATA, not TEXT; maybe the windows linker is getting this wrong?

Like I said before, we only have .data and .text sections at the moment. If we put them into data, then they won't be read-only. So we would have create new section for these. But perhaps there is an alternative way to mark them as not code for objdump - I did not get to that part of the code yet.

So windows doesn't have an rodata section?
That's where type.* goes on linux.

No. Go executables never had read-only-data section. We never felt the need. Perhaps we should

Alex

It wouldn't be the end of the world to just put those symbols in .data and call it a day.
Longer term it would be nice to introduce read-only data.

It wouldn't be the end of the world to just put those symbols in .data and call it a day.

Do you mean, for pe file writer in cmd/link to adjust both .text and .data sections to make .data section bigger to include read-only symbols? I am not sure that would be much easier than creating new read-only-data section between .text and .data sections.

Another alternative is to adjust cmd/internal/objfile/peFile.text function to return not complete .text section, but only the beginning up until runtime.etext symbol. That would be trivial enough to do. While we are there we could also adjust cmd/internal/objfile/peFile.symbols to use 'R' for symbols in .text section, if their address is less than address of runtime.etext. What do you think?

We would also need some test. Not sure about that, so suggestions are welcome.

Alex

Do you mean, for pe file writer in cmd/link to adjust both .text and .data sections to make .data section bigger to include read-only symbols?

Right, put the readonly symbols in the .data section instead of the .text section.

Your suggestion sounds fine as well. (With the modfication that we use R for symbols greater than etext?) This assumes that we do correctly segregate code from readonly data in the .text section.

Change https://golang.org/cl/108595 mentions this issue: cmd/internal/objfile: stop counting text symbols at runtime.etext

@marigonzes and @josharian please try https://go-review.googlesource.com/#/c/go/+/108595

It helps a lot, but I can still see it is not as efficient as with ELF file reading. My CL does not make Go linker write smaller PE ".text" section, so there is some quite large overhead reading huge ".text" section generated by example from #24725 (comment) I don't see how we can remove that allocation, because debug/pe.Section.Data returns []byte. So the only way to overcome that problem is to change Go linker to stop writing non-text symbols into PE ".text" section.

@randall77 and @josharian let me know if my CL is good enough. If not, we don't want my CL submitted, and I should try and change the linker.

Thank you

Alex

My two cents is that, from the perspective of this issue, your CL suffices. It brings windows objdump performance more or less in line with linux/darwin. (I haven't looked at the code, though, just its effect.)

From a long term perspective, I do think we should put readonly data in a readonly section. It looks like they exist. A bit of googling finds something like a PE spec, which contains this line:

.rdata , Read-only initialized data , IMAGE_SCN_CNT_INITIALIZED_DATA | IMAGE_SCN_MEM_READ

which seems about like what we're looking for.

From a long term perspective, I do think we should put readonly data in a readonly section. It looks like they exist.

Sure, it can be done. I am just worried it might affect / break more things. And I am not sure how long it will take me to make the change. But I will spent some time to see how complicated it gets.

Alex

I tried implementing separate (from .text symbols) read-only data PE section. And everything seems to work (with some problems here and there), except external linker.

When I build executable using external linker, and then run the app, the app crashes because runtime.checkASM returns false. When I print masks and shifts symbol addresses, I can see that they are not aligned to 16 bytes.

But I do not see how they get aligned. Where is the code that tells external linker to have them aligned? Mind you I don't see who align them with internal linker either. They are aligned with internal linker, but maybe that is just a fluke. How do I align them with external linker? What would be a good approach?

Maybe @ianlancetaylor you could help me here. Thank you.

Alex

@ianlancetaylor do you have any suggestions for #24725 (comment) ? I still hope I could implement read-only data PE section for go1.11.

Thank you.

Alex

@ianlancetaylor please reply to my #24725 (comment)

Thank you.

Alex

Sorry for the slow reply. I'm well behind on mail.

When using cmd/link, large variables like masks are aligned to the maximum alignment of the architecture, as set by the Maxalign field of ld.Arch initliazed in cmd/link/internal/*/asm.go. For amd64 this is 32, so all variables larger than 32 bytes will be aligned to a 32 byte boundary. The code for this is the symalign function in cmd/link/internal/ld/data.go. When using external linking these symbols should be properly aligned within each section, and the section's alignment should be set to the maximum alignment of the symbols placed in that section. The external linker is then expected to honor that alignment.

At least, that's how it works for ELF. When I look at the PE code, though, it looks like PE always sets the alignment of each output section to be 32 bytes, regardless of the input alignment requirements. I'm basing that on the use of IMAGE_SCN_ALIGN_32BYTES and on the lack of any reference to sect.Align.

So my first guess is that you aren't setting IMAGE_SCN_ALIGN_32BYTES in the new section that you are creating.

Sorry for the slow reply. I'm well behind on mail.

Same. :-)

When using external linking these symbols should be properly aligned within each section

So I should see masks aligned to 32 inside whatever section of go.o it lives?

the section's alignment should be set to the maximum alignment of the symbols placed in that section.

I read this as section with masks symbol should be marked as IMAGE_SCN_ALIGN_32BYTES. Correct?

When I look at the PE code, though, it looks like PE always sets the alignment of each output section to be 32 bytes, regardless of the input alignment requirements. I'm basing that on the use of IMAGE_SCN_ALIGN_32BYTES and on the lack of any reference to sect.Align.

Yes, that is what I see too. All go.o sections are tagged with IMAGE_SCN_ALIGN_32BYTES.

So my first guess is that you aren't setting IMAGE_SCN_ALIGN_32BYTES in the new section that you are creating.

I am pretty sure I do mark my new session with IMAGE_SCN_ALIGN_32BYTES. But I will check my code and post my findings here when I have time. Unless you have other ideas.

Thank you.

Alex

Change https://golang.org/cl/115975 mentions this issue: cmd/link: split pe .text section into .text and .rdata