Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/link: link failure caused by duplicated LDFLAGS passed to external linker #25930

Open
albertjin opened this issue Jun 17, 2018 · 10 comments
Open
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@albertjin
Copy link

What version of Go are you using (go version)?

go1.10.3

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

macOS/amd64, doing cross compile for android/arm64

What did you do?

export GOOS=android
export GOARCH=arm64
export CC=/path/to/androd/gcc
export CXX=/path/to/androd/g++
export CGO_ENABLED=1
export CGO_LDFLAGS="-Wl,--version-script=/the/version-script/export.txt"
go build -buildmode=c-shared -ldflags="-s -w -v" -v -o libxxx.so foo/bar

What did you expect to see?

The shared library is generated without any error.

What did you see instead?

In the output line

0.19 host link: ...

"-Wl,--version-script=/the/version-script/export.txt" is repeated 4 time and there are 3 errors,

anonymous version tag cannot be combined with other version tags

Note that if the version script contains a version name, the error message is 'duplicate version tag xxx'.

@albertjin
Copy link
Author

For a quick fix, change the line in src/cmd/link/internal/ld/lib.go

	argv = append(argv, ldflag...)

as the following,

	{
		added := map[string]bool{}
		for _, s := range ldflag {
			if added[s] {
				continue
			}
			added[s] = true
			argv = append(argv, s)
		}
	}

@albertjin albertjin changed the title Link failure caused by duplicated LDFLAGS passed to cgo linker cmd/go: Link failure caused by duplicated LDFLAGS passed to cgo linker Jun 17, 2018
@albertjin albertjin changed the title cmd/go: Link failure caused by duplicated LDFLAGS passed to cgo linker cmd/link: Link failure caused by duplicated LDFLAGS passed to cgo linker Jun 17, 2018
@tklauser tklauser added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 18, 2018
@tklauser
Copy link
Member

/cc @ianlancetaylor

@ianlancetaylor ianlancetaylor changed the title cmd/link: Link failure caused by duplicated LDFLAGS passed to cgo linker cmd/link: link failure caused by duplicated LDFLAGS passed to cgo linker Jun 18, 2018
@ianlancetaylor ianlancetaylor changed the title cmd/link: link failure caused by duplicated LDFLAGS passed to cgo linker cmd/link: link failure caused by duplicated LDFLAGS passed to external linker Jun 18, 2018
@ianlancetaylor
Copy link
Contributor

The suggested fix above is unfortunately too simple, as some options like -Bstatic may appear multiple times, and eliminating them would change the meaning of the link command. I don't know how to fix this without understanding the external linker options.

@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Jun 18, 2018
@albertjin
Copy link
Author

albertjin commented Jun 19, 2018

While analyzing the code for a safe fix, I realized that using #cgo directive in one .go file also solves the problem. This directive will not be replicated for each imported package, so there is just one after all auto generated //go:cgo_ldflag ... directives are merged back from each package.

It looks like this,

/*
#cgo LDFLAGS: "-Wl,--version-script=${SRCDIR}/export.txt"
//...
*/
#import "C"
// ...

Note that this flag is not in the white list and requires the following environment variable,

CGO_LDFLAGS_ALLOW="-Wl,--version-script=[a-zA-Z0-9+-_/.]+"

It seems that CGO_LDFLAGS is about a global configuration option of the Go toolchain. It should better not be used for each project.

@albertjin
Copy link
Author

albertjin commented Sep 24, 2018

@ianlancetaylor, I got a new idea for a general fix.

  1. Add a mark tag at the moment getting CGO_LDFLAGS array from env
  2. Remove the tag and clean up all duplicated arrays at linking

Note that the tag format is (a const uuid + length of the whole CGO_LDFLAGS array with the tag prefix).

patch_cgo_ldflags.diff.txt

@ianlancetaylor
Copy link
Contributor

@albertjin For copyright reasons I prefer to look at patches submitted using as a PR or via Gerrit, to ensure that the CLA is signed.

It sounds like you are suggesting handling LDFLAGS that come from an environment variable differently than LDFLAGS listed in the file, and removing duplicates from the environment variable. That seems reasonable.

@tmm1
Copy link
Contributor

tmm1 commented Jul 20, 2019

Is there any way to blacklist or remove certain ldflags from making it to the linker invocation?

For instance, src/runtime/cgo/cgo.go contains #cgo !android,linux LDFLAGS: -lpthread which is causing problems because I'm creating a static binary with some c++ code and it is very particular about where in the link line -lpthread will appear (and it also needs to be in the form of -Wl,--whole-archive -lpthread -Wl,--no-whole-archive). Is my only option to patch cgo.go?

@ianlancetaylor
Copy link
Contributor

Can you simply add the -Wl,--whole-archive ... to #cgo LDFLAGS in your own Go file?

If that doesn't work, the next fallback would be to write a little script that shuffles the arguments as you need, and pass that script as -ldflags=-extld=SCRIPT.

@albertjin
Copy link
Author

Can you simply add the -Wl,--whole-archive ... to #cgo LDFLAGS in your own Go file?

If that doesn't work, the next fallback would be to write a little script that shuffles the arguments as you need, and pass that script as -ldflags=-extld=SCRIPT.

For my situation, which was a generic build and deployment helper, I actually made a wrapper for the native compiler/linker without fixing the go toolchain. The method is the same as done in the previous patch, by adding a prefix mark in the environment variable CGO_LDFLAGS and getting rid of the duplicates later.

I tried to consolidate and submit the previous patch but hesitated. Maybe documenting the limitation of CGO_LDFLAGS is a final fix.

@abarisani
Copy link

I also stumbled upon this while attempting to use:

CGO_LDFLAGS="-marm -static -ffreestanding -specs=nosys.specs"

Linking fails as the final host link gets the flags repeated multiple times:

host link: "arm-none-eabi-gcc" "-marm" "-s" "-o" "/tmp/go-build2721378602/b001/exe/a.out" "-static" "/tmp/go-link-871732654/go.o" "/tmp/go-link-871732654/000000.o" <cut for brevity>" "-marm" "-static" "-ffreestanding" "-specs=nosys.specs" "-marm" "-static" "-ffreestanding" "-specs=nosys.specs" "-marm" "-static" "-ffreestanding" "-specs=nosys.specs" "-marm" "-static" "-ffreestanding" "-specs=nosys.specs" "-specs=nosys.specs"
pkg/tool/linux_amd64/link: running arm-none-eabi-gcc failed: exit status 1
arm-none-eabi-gcc: fatal error: /usr/lib/gcc/arm-none-eabi/11.2.0/../../../../arm-none-eabi/lib/nosys.specs: attempt to rename spec 'link_gcc_c_sequence' to already defined spec 'nosys_link_gcc_c_sequence'

By adding de-duplication for argv in src/cmd/link/internal/ld/lib.go I am able to get past this issue, I cannot find any other easy way to solve it as the cgo I am pulling in is coming from an external import.

The workaround shown in #25930 (comment) also does not work as apparently there is an allowlist of flags (in src/cmd/go/internal/work/security.go) which does not consent the ones I'd like to use.

@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

6 participants