Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: long symbol names for instantiated generics => large object files (though not executables) #50438

Open
csgura opened this issue Jan 5, 2022 · 9 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. generics Issue is related to generics NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@csgura
Copy link

csgura commented Jan 5, 2022

What version of Go are you using (go version)?

$ go version
go version devel go1.18-f154f8b Tue Jan 4 22:27:20 2022 +0000 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/gura/Library/Caches/go-build"
GOENV="/Users/gura/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/gura/go/pkg/mod"
GONOPROXY="*.uangel.com"
GONOSUMDB="*.uangel.com"
GOOS="darwin"
GOPATH="/Users/gura/go"
GOPRIVATE="*.uangel.com"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/gura/sdk/gotip"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/gura/sdk/gotip/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="devel go1.18-f154f8b Tue Jan 4 22:27:20 2022 +0000"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/gura/test/fp/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/y1/bngm83dj5_5dcsgh9yrwvpm00000gp/T/go-build228120003=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

$ git clone -b build-cache https://github.com/csgura/fp.git
$ cd fp
$ gotip clean -cache
$ du -hs ~/Library/Caches/go-build/
8.0K	/Users/gura/Library/Caches/go-build/

$ gotip test ./...

$ du -hs ~/Library/Caches/go-build/
1.4G	/Users/gura/Library/Caches/go-build/

What did you expect to see?

A build cache directory of a reasonable size ,
or
Warning about incorrectly used generic type.

What did you see instead?

$ du -hs ~/Library/Caches/go-build/
1.4G	/Users/gura/Library/Caches/go-build/

Please excuse my poor English.

It seems to be related to this issue as well. #50204

I have written some algebraic data types to test the generic of Go 1.18.
( https://github.com/csgura/fp.git )
After running the tests, I noticed that the build cache was using a very large amount of disk.
I guess the cause lies in the some recursive type ( HList and curried Func ) and the interface type that uses the type parameter.

When I modified the code so that a generic interface type does not return other generic interface type,
The size of the build cache has been significantly reduced.
This fix is applied in the master branch.

$ git checkout master
$ gotip clean -cache
$ gotip test ./...
$ du -hs ~/Library/Caches/go-build/
176M	/Users/gura/Library/Caches/go-build/

It will not become a problem right now, but I think it will become a big problem as more and more projects use generics.

@rsc
Copy link
Contributor

rsc commented Jan 5, 2022

It would help if you could provide a small single source file that produces a large output using

go tool compile -o x.a x.go

@csgura
Copy link
Author

csgura commented Jan 6, 2022

Here is a single file of about 1500 lines.

https://gotipplay.golang.org/p/kMBLHxigsDo

$ gotip clean -cache
$ gotip build
$ gotip tool compile -o x.a main.go
$ du -hs ~/Library/Caches/go-build/
105M    /Users/gura/Library/Caches/go-build/

$ ls -al
total 142016
drwxr-xr-x   6 gura  staff       192 Jan  6 11:34 .
drwxr-xr-x  15 gura  staff       480 Jan  6 10:15 ..
-rwxr-xr-x   1 gura  staff   2501872 Jan  6 11:34 main
-rw-r--r--   1 gura  staff        27 Jan  6 10:16 go.mod
-rw-r--r--   1 gura  staff     26695 Jan  6 11:31 main.go
-rw-r--r--   1 gura  staff  70173912 Jan  6 11:34 x.a

@ianlancetaylor ianlancetaylor changed the title cmd/compile: Size of 'Caches/go-build' directory is too large. cmd/compile: size of 'Caches/go-build' directory is too large Jan 6, 2022
@ianlancetaylor ianlancetaylor changed the title cmd/compile: size of 'Caches/go-build' directory is too large cmd/compile: object file much larger than final executable Jan 6, 2022
@ianlancetaylor
Copy link
Contributor

With the test case above, go tool compile foo.go produces a file that is 70173802 bytes. go tool link foo.o produces a file that is 2400985 bytes. So the final executable is about 3.5% of the size of the object file.

The largest symbol in the object file is

type..namedata.*main.Tuple2[go.shape.interface { Concat(main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]) main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; Drop(int) main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; DropWhile(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) bool) main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; Duplicate() main.Tuple2[main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0],main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]]; Exists(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) bool) bool; Filter(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) bool) main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; FilterNot(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) bool) main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; Find(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) bool) main.Option[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; ForAll(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) bool) bool; Foreach(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0)); HasNext() bool; IsEmpty() bool; MakeString(string) string; Map(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) interface {}) main.Iterator[interface {}]; Next() go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0; NextOption() main.Option[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; NonEmpty() bool; Partition(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) bool) main.Tuple2[main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0],main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]]; Reduce(main.Monoid[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]) go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0; Span(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) bool) main.Tuple2[main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0],main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]]; Take(int) main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; TakeWhile(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) bool) main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; TapEach(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0)) main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; ToList() main.List[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; ToSeq() main.Seq[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0] }_0,go.shape.interface { Concat(main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]) main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; Drop(int) main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; DropWhile(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) bool) main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; Duplicate() main.Tuple2[main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0],main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]]; Exists(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) bool) bool; Filter(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) bool) main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; FilterNot(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) bool) main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; Find(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) bool) main.Option[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; ForAll(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) bool) bool; Foreach(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0)); HasNext() bool; IsEmpty() bool; MakeString(string) string; Map(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) interface {}) main.Iterator[interface {}]; Next() go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0; NextOption() main.Option[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; NonEmpty() bool; Partition(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) bool) main.Tuple2[main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0],main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]]; Reduce(main.Monoid[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]) go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0; Span(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) bool) main.Tuple2[main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0],main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]]; Take(int) main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; TakeWhile(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0) bool) main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; TapEach(func(go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0)) main.Iterator[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; ToList() main.List[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0]; ToSeq() main.Seq[go.shape.interface { Failed() main.Try[error]; Foreach(func(int)); Get() int; IsFailure() bool; IsSuccess() bool; Iterator() main.Iterator[int]; Or(func() main.Try[int]) main.Try[int]; OrElse(int) int; OrElseGet(func() int) int; Recover(func(error) int) main.Try[int]; RecoverWith(func(error) main.Try[int]) main.Try[int]; String() string; ToOption() main.Option[int]; ToSeq() main.Seq[int]; Unapply() (int, error) }_0] }_1].

It's 30111 bytes (according to go tool nm). It does not appear in the final executable. In fact, none of the 100 largest symbols in the object file appear in the executable.

This may be working as expected, but CC @randall77 @danscales in case it is not.

@csgura
Copy link
Author

csgura commented Jan 6, 2022

https://gotipplay.golang.org/p/8WZz12Ay6vz

It is almost the same code, but the Option, Try , HCons and Iterator types are changed to a struct type.

drwxr-xr-x  5 gura  staff       160 Jan  6 14:46 .
drwxr-xr-x  7 gura  staff       224 Jan  6 14:07 ..
-rwxr-xr-x  1 gura  staff   2114816 Jan  6 14:42 opti
-rw-r--r--  1 gura  staff     23637 Jan  6 14:42 opti.go
-rw-r--r--  1 gura  staff  17436020 Jan  6 14:46 x.a

The object file size has been reduced to 17436020.

It seems that object files are created inefficiently when generic interfaces refer to each other.
I know that it works as expected,
but wouldn't it be better to output a warning message when an object file is created inefficiently like that?

Because I think It would be better not to use type parameters with interface types in the production to reduce compile time.

@danscales
Copy link
Contributor

As Ian's symbol example shows, the symbols can be quite large for the names of instantiated functions/methods, when the type arguments are instantiated types that are nested and the descriptions of some of the underlying types (e.g. interfaces) are large. For the name of a shape type (the proxy type standing for all the types that a particular instantiation will handle), we use the standard printing (via LinkString) of the shared underlying type (e.g go.shape.int64_0 or go.shape.interface { M(); String() string }_0 or go.shape.struct { p.x int8; p.y float64 }_0. Using the name of the underlying type is easiest so that we have a unique name across packages. For Go 1.19, we could try to use shorter, unique, but memo-ized names, but then we would probably need extra coordination across compilations (using the build cache?). Or, if a shape name gets really large (> 50 characters), maybe we could replace it with an MD5 hash?

@ianlancetaylor
Copy link
Contributor

wouldn't it be better to output a warning message when an object file is created inefficiently like that?

No. This is something for a release note, not a compiler warning. Everything works correctly, it just takes more space than expected. The compiler never issues warnings anyhow.

@cagedmantis cagedmantis added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. generics Issue is related to generics labels Jan 7, 2022
@cagedmantis cagedmantis added this to the Go1.18 milestone Jan 7, 2022
@akutz
Copy link

akutz commented Jan 25, 2022

FWIW, I've written a few tests to show that while package archives are definitely larger (usually 2x), the resulting binary executable shows no real difference -- https://github.com/akutz/go-generics-the-hard-way/blob/main/06-benchmarks/03-file-sizes.md.

@ianlancetaylor
Copy link
Contributor

@danscales This is in the 1.18 milestone; time to move to 1.19? Thanks.

@danscales
Copy link
Contributor

Yes, I'll move to 1.19. Thanks!

@danscales danscales modified the milestones: Go1.18, Go1.19 Jan 29, 2022
@thanm thanm modified the milestones: Go1.19, Go1.20 May 17, 2022
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 13, 2022
@mknyszek mknyszek modified the milestones: Go1.20, Backlog Nov 30, 2022
@adonovan adonovan changed the title cmd/compile: object file much larger than final executable cmd/compile: long symbol names for instantiated generics => large object files (though not executables) Jan 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. generics Issue is related to generics NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
Development

No branches or pull requests

9 participants