Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/cgo: undefined reference for C function with -buildmode=c-archive #25832

Open
ZolAnder85 opened this issue Jun 11, 2018 · 22 comments · Fixed by bazelbuild/rules_go#3174
Open
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@ZolAnder85
Copy link

What version of Go are you using (go version)?

go version go1.10.3 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/Andari/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/Andari/go"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.10.3/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.10.3/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/lw/5pzww_8n38g1lv_kqqmqfvqr0000gn/T/go-build228325343=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I was experimenting with C interfacing:

// int cadd(int a, int b);
import "C"
...
// using the function somewhere

I built it with go build -buildmode=c-archive GoAdd.go

What did you expect to see?

The build to be successful.

What did you see instead?

Undefined symbols for architecture x86_64:
"_cadd", referenced from:
__cgo_ece3f0761be9_Cfunc_cadd in _x002.o

Why would linking be neccessary, if I am generating an archive which I link together with the cadd implementing object file later?

@ZolAnder85 ZolAnder85 changed the title There should be no need to add cgo ladflags for linking when using -buildmode=c-archive There should be no need to add cgo ldflags for linking when using -buildmode=c-archive Jun 11, 2018
@bcmills
Copy link
Contributor

bcmills commented Jun 11, 2018

CC @ianlancetaylor

@bcmills bcmills changed the title There should be no need to add cgo ldflags for linking when using -buildmode=c-archive cmd/cgo: undefined reference for forward-declared function with -buildmode=c-archive Jun 11, 2018
@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 11, 2018
@bcmills bcmills added this to the Go1.12 milestone Jun 11, 2018
@ianlancetaylor
Copy link
Contributor

The cgo tool works this way because it needs to know the type of the C function cadd. It currently determines it by linking the program against the C libraries and examining the debug info. It would be nice if it could simply parse the declaration in the "C" comment, but, well, that requires a C parser. Examining the debug info seems to work reliably enough.

You are doing something rather complicated: you are building a C archive, which is Go code callable from C. And you want that code, written in Go, to itself call a C function. So you have C calling Go calling C. I agree that in principle this ought to work. But in practice it's kind of complicated.

Which is to say: yes, let's fix this if we can. But no promises on when or if anybody will look into it.

@ianlancetaylor ianlancetaylor modified the milestones: Go1.12, Unplanned Jun 12, 2018
@ianlancetaylor ianlancetaylor changed the title cmd/cgo: undefined reference for forward-declared function with -buildmode=c-archive cmd/cgo: undefined reference for C function with -buildmode=c-archive Jun 12, 2018
@ZolAnder85
Copy link
Author

ZolAnder85 commented Jun 13, 2018

Ok, I understand – I thought, it already parsed the comment abowe the import "C". Note, that I can get it working by compiling-only (clang -c) the C code first and using #cgo LDFLAGS: CAdd.o. The resulted CAdd.o and GoAdd.a can be linked or archived together either with each other or for example a C++ code using extern "C", or an Objective C / Swift code, and possible many other languages. (Actually, my final goal is to write a rendering engine in Go by calling its exported functions from native languages of each platform.)

@ZolAnder85
Copy link
Author

ZolAnder85 commented Jun 13, 2018

Alternative possibility to allow declaring only C function would be something like this:

import "C"

//import cadd
func cadd(a C.int, b C.int) C.int

Like extern "C" in C++ or extern(C) in D.
This way a C parser would be not required.

@ianlancetaylor
Copy link
Contributor

You might be able to do that today using go:linkname. I haven't tried, though.

@ZolAnder85
Copy link
Author

I had no success with go:linkname to get the C function imported.

@NickNaso
Copy link

Hi everyone,
I have the same problem. I have C calling Go calling C and it should be great have a chance to do that without the linker error.

@ZolAnder85
Copy link
Author

Until there is a better option, you can get it working by compiling the C code with -c and use that to compile the Go code and than link together those. For that, you have to have the declarations of the functions exported from the Go source. I did that by hand. I was even lazy enough to simply use int on both sides, which works on most architectures.

I would suggest to create a feature request to be able to use this:

import "C"

//import cadd
func cadd(a C.int, b C.int) C.int

// for use in other packages with Go types
func Cadd(a int, b int) int {
  return int(cadd(C.int(a), C.int(b)))
}

@NickNaso
Copy link

Hi @Andari85 ,
I did as you proposed and all worked fine. Thanks for the advice. It could be good have a better option because in my case compiling with the -c option requested a lot of work. Thanks again.

@ZolAnder85
Copy link
Author

So, is there a specific way to create a feature request, or will this be picked up by someone?

@ianlancetaylor
Copy link
Contributor

@Andari85 This issue is still open, so it is in effect a feature request. However, as I said above, it's kind of complicated, let's fix it if we can, but no promises on when or if anybody will look into it.

Go is an open source project, so: volunteers welcome. I've already marked the issue as "help wanted".

@ZolAnder85
Copy link
Author

ZolAnder85 commented Sep 7, 2018

Okay, thanks. Well, I'm not much of a compiler guru, one day I might try to learn into the Go compiler.

@ianlancetaylor
Copy link
Contributor

For what it's worth, I don't think that any fix to this will involve the compiler itself at all. I think all the work will be in the go tool and/or the cgo tool.

@azsn
Copy link

azsn commented Jan 7, 2020

I was able to achieve the requested result (a Go static library which can be invoked by a C main program and can, in turn, call exposed C functions in the main program) by doing the following:

In the Go code, export functions as normal with the //export directive:

//export myGoFunc
func myGoFunc() {
    C.myCFunc()
}

Import cgo, and in the import declarations, declare the external C functions you want to call from Go.

// extern void myCFunc();
import "C"

Then, the important part: the cgo code needs to be compiled with the linker flag -unresolved-symbols=ignore-all (for Linux GCC, on Mac the equivalent flag is -undefined,dynamic_lookup). You could probably do this at the build command somehow, but its more clear to add it to the cgo declaration like this:

// #cgo linux LDFLAGS: -Wl,-unresolved-symbols=ignore-all
// #cgo darwin LDFLAGS: -Wl,-undefined,dynamic_lookup
// extern void myCFunc();
import "C"

These linker flags allow the Go code to successfully compile without having the external C functions defined anywhere, as they get looked up later when the C program they're in gets linked with the Go library.

Then, finally, build the Go static library with -buildmode=c-archive (specifically, my build command is go build -o mylib.a -buildmode=c-archive src.go) and link it to a C program in which the extern function(s) are defined.

So far this seems to work correctly for me, but I'd like to know from Go experts if there is anything wrong with this solution. Will certain features of Go, like goroutines/scheduling or memory management, not work correctly? They appear to be working, but I don't know enough about the technical details of Go to be sure...

@ianlancetaylor
Copy link
Contributor

@zelbrium I don't think there is anything wrong with your solution. There are some oddball cases where it might fail if you want to use internal linking (-ldflags=-linkmode=internal), but that is not the default, and there is no reason for you to use it.

@NickNaso
Copy link

NickNaso commented Mar 6, 2020

Hi everybody,
how the LDFLAGS need to be set for Windows?

@azsn
Copy link

azsn commented Mar 6, 2020

@NickNaso I don't know for sure -- I'm not very experienced with MSVC -- but perhaps try the /FORCE:UNRESOLVED argument instead of the -unresolved-symbols=ignore-all argument. It appears to do something similar, at least by what the documentation says.

Alternatively, you might be able to use MinGW + GCC on Windows to compile with GCC instead of MSVC. Then the LDFLAGS should be the same as on Linux.

Good luck!

@swq123459
Copy link

omg, finally i foud

// export

is wrong
but the follow is rigth

//export

just don't add space after "//"

@azsn
Copy link

azsn commented Jul 16, 2020

@swq123459 From Go's Hidden Pragmas: "[No space after //] is partly an accident of history, but it also makes it less likely to conflict with a regular comment." But the fact that Go even uses // for pragmas and Cgo directives in the first place is absolutely crazy....

@hajimehoshi
Copy link
Member

Is there any solution for Windows? Apparently -unresolved-symbols=ignore-all doesn't work for MinGW.

@NickNaso
Copy link

@hajimehoshi I had your same problem and I created a fake dll library to link using dlltool.
Es.

 dlltool -d ./your-definition-libray.def -y your-library.a

dlltool is part of MinGW the .def file contains all the symbols you need.
Hope this could help you.

@hajimehoshi
Copy link
Member

hajimehoshi commented May 28, 2022

@NickNaso Thank you! I took a little different approach (creating a dummy DLL from a C file and linking it), and it worked well.

fmeum added a commit to fmeum/rules_go that referenced this issue Jun 4, 2022
Before this commit, if a go_library referenced an unresolved C symbol,
the build would fail with an error about an undefined symbol since
compilepkg.go tries to link all object files into an executable to be
consumed by cgo's -dynimport argument.

Since this library is never used for any purpose other than having its
symbols parsed by the cgo command, retry with additional linker flags
intended to make the linker ignore unresolved symbols. If they remain
missing in the real link step, they will still be reported as unresolved
there.

Fixes golang/go#25832 for rules_go.
fmeum added a commit to fmeum/rules_go that referenced this issue Jun 4, 2022
Before this commit, if a go_library referenced an unresolved C symbol,
the build would fail with an error about an undefined symbol since
compilepkg.go tries to link all object files into an executable to be
consumed by cgo's -dynimport argument.

Since this library is never used for any purpose other than having its
symbols parsed by the cgo command, retry with additional linker flags
intended to make the linker ignore unresolved symbols. If they remain
missing in the real link step, they will still be reported as unresolved
there.

Fixes golang/go#25832 for rules_go.
fmeum added a commit to fmeum/rules_go that referenced this issue Jun 4, 2022
Before this commit, if a go_library referenced an unresolved C symbol,
the build would fail with an error about an undefined symbol since
compilepkg.go tries to link all object files into an executable to be
consumed by cgo's -dynimport argument.

Since this library is never used for any purpose other than having its
symbols parsed by the cgo command, retry with additional linker flags
intended to make the linker ignore unresolved symbols. If they remain
missing in the real link step, they will still be reported as unresolved
there.

Fixes golang/go#25832 for rules_go.
fmeum added a commit to fmeum/rules_go that referenced this issue Jun 4, 2022
Before this commit, if a go_library referenced an unresolved C symbol,
the build would fail with an error about an undefined symbol since
compilepkg.go tries to link all object files into an executable to be
consumed by cgo's -dynimport argument.

Since this library is never used for any purpose other than having its
symbols parsed by the cgo command, retry with additional linker flags
intended to make the linker ignore unresolved symbols. If they remain
missing in the real link step, they will still be reported as unresolved
there.

Fixes golang/go#25832 for rules_go.
linzhp pushed a commit to bazelbuild/rules_go that referenced this issue Jun 4, 2022
…3174)

Before this commit, if a go_library referenced an unresolved C symbol,
the build would fail with an error about an undefined symbol since
compilepkg.go tries to link all object files into an executable to be
consumed by cgo's -dynimport argument.

Since this library is never used for any purpose other than having its
symbols parsed by the cgo command, retry with additional linker flags
intended to make the linker ignore unresolved symbols. If they remain
missing in the real link step, they will still be reported as unresolved
there.

Fixes golang/go#25832 for rules_go.
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants