Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/cgo: _GoStringLen/_GoStringPtr not available with exported functions in c-archive or c-shared modes #48824

Closed
theimpostor opened this issue Oct 6, 2021 · 5 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@theimpostor
Copy link

theimpostor commented Oct 6, 2021

What version of Go are you using (go version)?

$ go version
go version go1.17.1 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/shoda/Library/Caches/go-build"
GOENV="/Users/shoda/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/shoda/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/shoda/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.17.1/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.17.1/libexec/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.17.1"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/ct/x2gct7yn2bxfqs891n8h1dxr0000gn/T/go-build205412794=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Using this program:

package main

/*
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
void dumpGoString(_GoString_ s) {
    size_t len = _GoStringLen(s);
    const char * sp = _GoStringPtr(s); // may not be NULL terminated
    char * str = strndup(sp, len);
    fprintf(stderr, "go string: %s\n", str);
    free(str);
}
*/
import "C"
import "strconv"

//export PrintWithC
func PrintWithC(v int) {
	C.dumpGoString(strconv.Itoa(v))
}

func main() {
	PrintWithC(42)
}

What did you expect to see?

Doing go build -buildmode=c-archive -o gostring.a main.go should produce a library that is callable from C.

What did you see instead?

❯ go build -buildmode=c-archive -o gostring.a main.go
# command-line-arguments
In file included from _cgo_export.c:4:
main.go:8:18: error: implicit declaration of function '_GoStringLen' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
main.go:9:23: error: implicit declaration of function '_GoStringPtr' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
main.go:9:23: note: did you mean '_GoStringLen'?
main.go:8:18: note: '_GoStringLen' declared here
main.go:9:18: warning: incompatible integer to pointer conversion initializing 'const char *' with an expression of type 'int' [-Wint-conversion]

If I remove the line //export PrintWithC, the app runs fine:

❯ go run main-no-export.go
go string: 42

Also go build -buildmode=c-archive -o gostring.a main-no-export.go succeeds but there is no header file so not sure how I would invoke from C.

Ultimately what I would like to do is to be able to declare void dumpGoString(_GoString_ s); in main.go, build main.go as an archive or shared library, and define dumpGoString in a separate C library which is able to call _GoStringLen and _GoStringPtr on the passed in _GoString_ s from golang. I assumed this was possible from reading the cgo doc, but perhaps I am missing something.

@theimpostor
Copy link
Author

Also, if I forward-declare _GoStringLen and _GoStringPtr to resolve the C compiler errors, I get undefined symbol errors later:

package main

/*
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
size_t _GoStringLen(_GoString_ s);
const char *_GoStringPtr(_GoString_ s);
void dumpGoString(_GoString_ s) {
    size_t len = _GoStringLen(s);
    const char * sp = _GoStringPtr(s); // may not be NULL terminated
    char * str = strndup(sp, len);
    fprintf(stderr, "go string: %s\n", str);
    free(str);
}
*/
import "C"
import "strconv"

//export PrintWithC
func PrintWithC(v int) {
	C.dumpGoString(strconv.Itoa(v))
}

func main() {
	PrintWithC(42)
}
❯ go build -buildmode=c-archive -o gostring.a main.go
# command-line-arguments
Undefined symbols for architecture x86_64:
  "__GoStringLen", referenced from:
      _dumpGoString in _x001.o
  "__GoStringPtr", referenced from:
      _dumpGoString in _x001.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

@mknyszek
Copy link
Contributor

mknyszek commented Oct 6, 2021

The cgo docs indicate that these are only available in the C preamble of .go files.

I think that means that no, these symbols will not be available as part of a c-archive build. I'm not sure how you're supposed to make this work, though, or if it's possible at all.

CC @ianlancetaylor @aclements ?

@mknyszek mknyszek changed the title _GoStringLen/_GoStringPtr not available with cgo exported functions cmd/cgo: _GoStringLen/_GoStringPtr not available with exported functions in c-archive or c-shared modes Oct 6, 2021
@mknyszek mknyszek added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Oct 6, 2021
@mknyszek mknyszek added this to the Backlog milestone Oct 6, 2021
@ianlancetaylor
Copy link
Contributor

Quoting https://golang.org/cmd/cgo:

Using //export in a file places a restriction on the preamble: since it is copied into two different C output files, it must not contain any definitions, only declarations. If a file contains both definitions and declarations, then the two output files will produce duplicate symbols and the linker will fail. To avoid this, definitions must be placed in preambles in other files, or in C source files.

You have a definition in your C preamble, and that is not supported when using //export. Sorry.

@theimpostor
Copy link
Author

You have a definition in your C preamble, and that is not supported when using //export. Sorry.

Thanks, you are correct.

Ultimately I think I was able to get this work by:

  1. Following the advice here about C calling Go calling C
  2. Using 'static inline' to be able to define code in the preamble and export at the same time, as demonstrated in the wiki
  3. Manually declaring _GoStringLen and _GoStringPtr in the preamble.

Seems like 2 would be good to mention in the cgo documentation, and 3 should be done automatically?

Here is what I ended up with, which allows me to access a Go string from C with no extra allocations:

main.go:

package main

/*
#cgo linux LDFLAGS: -Wl,-unresolved-symbols=ignore-all
#cgo darwin LDFLAGS: -Wl,-undefined,dynamic_lookup
extern void dumpCString(const char *s, size_t length);
size_t _GoStringLen(_GoString_ s);
const char *_GoStringPtr(_GoString_ s);
static inline void dumpGoString(_GoString_ s)
{
    dumpCString(_GoStringPtr(s), _GoStringLen(s));
}
*/
import "C"
import "strconv"

//export PrintWithC
func PrintWithC(v int) {
	C.dumpGoString(strconv.Itoa(v))
}

func main() {
	PrintWithC(42)
}

main.c:

#include <stdio.h>

#include "libgostring.h"

void dumpCString(const char *s, size_t length)
{
    printf("go string: %.*s\n", (int)length, s);
}

int
main(int argc, char** argv)
{
    PrintWithC(42);
}
❯ go build -buildmode=c-archive -o libgostring.a main.go
❯ clang libgostring.a main.c
❯ ./a.out
go string: 42

@ianlancetaylor
Copy link
Contributor

As far as I can tell _GoStringLen and _GoStringPtr are declared in the preamble--but only if you don't use //export. The static inline trick can often work but there is no intention of officially supporting it.

@golang golang locked and limited conversation to collaborators Oct 6, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

4 participants