Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/link: buildmode=shared with many packages produces "file name too long" error #38330

Closed
jdizzle opened this issue Apr 9, 2020 · 4 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@jdizzle
Copy link

jdizzle commented Apr 9, 2020

What version of Go are you using (go version)?

$ go version
go version go1.14.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jdel/.cache/go-build"
GOENV="/home/jdel/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/jdel/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/jdel/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build427238640=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I want to build a shared library with a subset of the standard library, but many packages. This is the command I'm using, which seems to be the appropriate way to attempt this according to "go help buildmode".

./bin/go install -buildmode=shared -linkshared archive/tar archive/zip bufio bytes compress/flate compress/gzip container/heap container/list context crypto crypto/aes crypto/cipher crypto/des crypto/dsa crypto/ecdsa crypto/elliptic crypto/hmac crypto/internal/randutil crypto/internal/subtle crypto/md5 crypto/rand crypto/rc4 crypto/rsa crypto/sha1 crypto/sha256 crypto/sha512 crypto/subtle crypto/tls crypto/x509 crypto/x509/pkix database/sql database/sql/driver encoding encoding/asn1 encoding/base64 encoding/binary encoding/csv encoding/hex encoding/json encoding/pem encoding/xml errors expvar flag fmt go/ast go/parser go/scanner go/token hash hash/crc32 html html/template image image/color image/internal/imageutil image/jpeg internal/bytealg internal/cpu internal/fmtsort internal/nettrace internal/poll internal/race internal/singleflight internal/syscall/unix internal/testlog io io/ioutil log log/syslog math math/big math/bits math/cmplx math/rand mime mime/multipart mime/quotedprintable net net/http net/http/httptest net/http/httptrace net/http/httputil net/http/internal net/http/pprof net/textproto net/url os os/exec os/signal os/user path path/filepath reflect regexp regexp/syntax runtime runtime/cgo runtime/debug runtime/internal/atomic runtime/internal/math runtime/internal/sys runtime/pprof runtime/trace sort strconv strings sync sync/atomic syscall testing text/tabwriter text/template text/template/parse time unicode unicode/utf16 unicode/utf8 unsafe

This is the failure I get:

# /tmp/go-build985075268/b143/libarchive-tar,archive-zip,bufio,bytes,compress-flate,compress-gzip,container-heap,container-list,context,crypto,crypto-aes,crypto-cipher,crypto-des,crypto-dsa,crypto-ecdsa,crypto-elliptic,crypto-hmac,crypto-internal-randutil,crypto-internal-subtle,crypto-md5,crypto-rand,crypto-rc4,crypto-rsa,crypto-sha1,crypto-sha256,crypto-sha512,crypto-subtle,crypto-tls,crypto-x509,crypto-x509-pkix,database-sql,database-sql-driver,encoding,encoding-asn1,encoding-base64,encoding-binary,encoding-csv,encoding-hex,encoding-json,encoding-pem,encoding-xml,errors,expvar,flag,fmt,go-ast,go-parser,go-scanner,go-token,hash,hash-crc32,html,html-template,image,image-color,image-internal-imageutil,image-jpeg,internal-bytealg,internal-cpu,internal-fmtsort,internal-nettrace,internal-poll,internal-race,internal-singleflight,internal-syscall-unix,internal-testlog,io,io-ioutil,log,log-syslog,math,math-big,math-bits,math-cmplx,math-rand,mime,mime-multipart,mime-quotedprintable,net,net-http,net-http-httptest,net-http-httptrace,net-http-httputil,net-http-internal,net-http-pprof,net-textproto,net-url,os,os-exec,os-signal,os-user,path,path-filepath,reflect,regexp,regexp-syntax,runtime,runtime-cgo,runtime-debug,runtime-internal-atomic,runtime-internal-math,runtime-internal-sys,runtime-pprof,runtime-trace,sort,strconv,strings,sync,sync-atomic,syscall,testing,text-tabwriter,text-template,text-template-parse,time,unicode,unicode-utf16,unicode-utf8,unsafe.so
/home/jdel/go/pkg/tool/linux_amd64/link: cannot create $WORK/b143/libarchive-tar,archive-zip,bufio,bytes,compress-flate,compress-gzip,container-heap,container-list,context,crypto,crypto-aes,crypto-cipher,crypto-des,crypto-dsa,crypto-ecdsa,crypto-elliptic,crypto-hmac,crypto-internal-randutil,crypto-internal-subtle,crypto-md5,crypto-rand,crypto-rc4,crypto-rsa,crypto-sha1,crypto-sha256,crypto-sha512,crypto-subtle,crypto-tls,crypto-x509,crypto-x509-pkix,database-sql,database-sql-driver,encoding,encoding-asn1,encoding-base64,encoding-binary,encoding-csv,encoding-hex,encoding-json,encoding-pem,encoding-xml,errors,expvar,flag,fmt,go-ast,go-parser,go-scanner,go-token,hash,hash-crc32,html,html-template,image,image-color,image-internal-imageutil,image-jpeg,internal-bytealg,internal-cpu,internal-fmtsort,internal-nettrace,internal-poll,internal-race,internal-singleflight,internal-syscall-unix,internal-testlog,io,io-ioutil,log,log-syslog,math,math-big,math-bits,math-cmplx,math-rand,mime,mime-multipart,mime-quotedprintable,net,net-http,net-http-httptest,net-http-httptrace,net-http-httputil,net-http-internal,net-http-pprof,net-textproto,net-url,os,os-exec,os-signal,os-user,path,path-filepath,reflect,regexp,regexp-syntax,runtime,runtime-cgo,runtime-debug,runtime-internal-atomic,runtime-internal-math,runtime-internal-sys,runtime-pprof,runtime-trace,sort,strconv,strings,sync,sync-atomic,syscall,testing,text-tabwriter,text-template,text-template-parse,time,unicode,unicode-utf16,unicode-utf8,unsafe.so: open $WORK/b143/libarchive-tar,archive-zip,bufio,bytes,compress-flate,compress-gzip,container-heap,container-list,context,crypto,crypto-aes,crypto-cipher,crypto-des,crypto-dsa,crypto-ecdsa,crypto-elliptic,crypto-hmac,crypto-internal-randutil,crypto-internal-subtle,crypto-md5,crypto-rand,crypto-rc4,crypto-rsa,crypto-sha1,crypto-sha256,crypto-sha512,crypto-subtle,crypto-tls,crypto-x509,crypto-x509-pkix,database-sql,database-sql-driver,encoding,encoding-asn1,encoding-base64,encoding-binary,encoding-csv,encoding-hex,encoding-json,encoding-pem,encoding-xml,errors,expvar,flag,fmt,go-ast,go-parser,go-scanner,go-token,hash,hash-crc32,html,html-template,image,image-color,image-internal-imageutil,image-jpeg,internal-bytealg,internal-cpu,internal-fmtsort,internal-nettrace,internal-poll,internal-race,internal-singleflight,internal-syscall-unix,internal-testlog,io,io-ioutil,log,log-syslog,math,math-big,math-bits,math-cmplx,math-rand,mime,mime-multipart,mime-quotedprintable,net,net-http,net-http-httptest,net-http-httptrace,net-http-httputil,net-http-internal,net-http-pprof,net-textproto,net-url,os,os-exec,os-signal,os-user,path,path-filepath,reflect,regexp,regexp-syntax,runtime,runtime-cgo,runtime-debug,runtime-internal-atomic,runtime-internal-math,runtime-internal-sys,runtime-pprof,runtime-trace,sort,strconv,strings,sync,sync-atomic,syscall,testing,text-tabwriter,text-template,text-template-parse,time,unicode,unicode-utf16,unicode-utf8,unsafe.so: file name too long

What did you expect to see?

This to succeed and produce a library file.

What did you see instead?

"File name too long" error. The problem is the package names are all concatenated into the file name, which bloats up quickly. I can manually work around this with "go tool link" but is there a more canonical way?

More context:
I want to split a large executable into plugins to reduce memory footprint at run time under certain circumstances. The plan is:

  • Create a plugin with all the functions I desire.
  • Create a shared library with all intersection of dependent packages between the plugin and my exe. This is to prevent redundantly bloating plugin and exe with identical text for the common packages.
@ianlancetaylor ianlancetaylor changed the title buildmode=shared with many packages produces "file name too long" error cmd/link: buildmode=shared with many packages produces "file name too long" error Apr 10, 2020
@ianlancetaylor ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Apr 10, 2020
@ianlancetaylor ianlancetaylor added this to the Backlog milestone Apr 10, 2020
@ianlancetaylor
Copy link
Contributor

CC @mwhudson

@mwhudson
Copy link
Contributor

Yes, this requires some cmd/go changes to work I think. I'm not sure what the best answer would be, take a hash of the package names instead?

@ianlancetaylor
Copy link
Contributor

As I recall the name is kind of irrelevant, isn't it? If so, then maybe a hash would be good. Or maybe the first couple of packages and then a hash, to give people a chance of knowing which file is which.

@seankhliao
Copy link
Member

Obsoleted by #47788

@golang golang locked and limited conversation to collaborators Oct 27, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

5 participants