Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime,cmd/compile: -buildmode=c-shared and dlopen-ing a shared library #16805

Open
sbinet opened this issue Aug 19, 2016 · 4 comments
Open
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. early-in-cycle A change that should be done early in the 3 month dev cycle. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@sbinet
Copy link
Member

sbinet commented Aug 19, 2016

consider the following GOPATH:

sh> tree .
.
├── mylib.so
└── src
    ├── main.go
    ├── my-cmd
    │   └── main.go
    ├── pkg1
    │   └── pkg.go
    └── pkg2
        └── pkg.go

5 directories, 6 files

with:

// src/pkg1
package pkg1

import "fmt"

var Int = 0
var Map = make(map[string]int)

func init() {
    fmt.Printf("pkg1.Int: %p\n", &Int)
    fmt.Printf("pkg1.Map: %p\n", &Map)
}
// src/pkg2
package pkg2

import "C"
import (
    "fmt"
    "pkg1"
)

//export Load
func Load() {
    fmt.Printf("pkg2.Load...\n")
}

func init() {
    fmt.Printf(">>> pkg2: pkg1.Int: %p\n", &pkg1.Int)
    fmt.Printf(">>> pkg2: pkg1.Map: %p\n", &pkg1.Map)
}
// src/my-cmd/main.go
package main

import "C"

import _ "pkg2"

func main() {}

and:

// src/main.go
package main

// #include <dlfcn.h>
// #cgo LDFLAGS: -ldl
// #include <stdlib.h>
// #include <stdio.h>
//
// void loadPlugin(void *lib) {
//   void (*f)(void) = NULL;
//   char *error = NULL;
//   f = (void (*)(void))(dlsym(lib, "Load"));
//   error = dlerror();
//   if (f == NULL || error != NULL) {
//      fprintf(stderr, "ERROR no such symbol!!! (%s)\n", error);
//      return;
//   }
//   fprintf(stderr, "symbol 'Load' loaded...\n");
//   (*f)();
// }
import "C"

import (
    "fmt"
    "log"
    "pkg1"
    "unsafe"
)

func main() {
    fmt.Printf("main.pkg1.Int: %p\n", &pkg1.Int)
    fmt.Printf("main.pkg1.Map: %p\n", &pkg1.Map)

    fmt.Printf("loading DLL...\n")
    cstr := C.CString("./mylib.so")
    defer C.free(unsafe.Pointer(cstr))
    h := C.dlopen(cstr, C.RTLD_NOW)
    if h == nil {
        log.Fatalf("error loading %s\n", C.GoString(cstr))
    }
    defer C.dlclose(h)

    fmt.Printf("loading plugin...\n")
    C.loadPlugin(h)
    fmt.Printf("loading plugin... [done]\n")

}

running the following command, gives:

sh> go build -buildmode=c-shared -o mylib.so ./src/my-cmd && go run ./src/main.go 
pkg1.Int: 0x728c48
pkg1.Map: 0x70e1e0
main.pkg1.Int: 0x728c48
main.pkg1.Map: 0x70e1e0
loading DLL...
loading plugin...
symbol 'Load' loaded...
pkg1.Int: 0x7f925f7f9a48
pkg1.Map: 0x7f925f7df038
>>> pkg2: pkg1.Int: 0x7f925f7f9a48
>>> pkg2: pkg1.Map: 0x7f925f7df038
pkg2.Load...
loading plugin... [done]

ie: the addresses of pkg1.Int and pkg1.Map are not the same when inspected from the go.main() or pkg1.init and when inspected from the dynamically loaded mylib.so shared library.
also, pkg1.init() is run twice.

here is my environment:

sh> go version
go version go1.7 linux/amd64

sh> go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/binet/work/igo/src/github.com/go-interpreter/example/cshared-bug"
GORACE=""
GOROOT="/usr/lib/go"
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build549233304=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"

from my reading of https://docs.google.com/document/d/1nr-TQHw_er6GOQRsF6T43GGhFDelrAP0NqSS_00RgZQ/edit# this shouldn't happen, even when using the c-shared buildmode instead of the plugin buildmode.

@ianlancetaylor @crawshaw : right ?

@crawshaw
Copy link
Member

I think it's fair to say at this point that what you're doing is not supported yet. Even if the symbols were correctly de-duplicated, the dlopen'ed c-shared library will attempt to initialize the runtime again, which I don't think we have adequate defenses against yet.

(I'll try to get my buildmode=plugin CLs out soon, which take care of this.)

@quentinmit quentinmit added this to the Go1.8Maybe milestone Sep 6, 2016
@quentinmit
Copy link
Contributor

@crawshaw Is this now resolved by saying "use buildmode=plugin"?

@quentinmit quentinmit added the NeedsFix The path to resolution is known, but the work has not been done. label Oct 10, 2016
@crawshaw
Copy link
Member

There's a slight variant of this that deserves fixing (and wouldn't be too much work, but I'm a bit short on time): which is using dlopen from a C program on two separately built Go c-shared libraries.

The second library's global constructor should not re-initialize the runtime, but should do most of the work in plugin_lastmoduleinit.

@rsc
Copy link
Contributor

rsc commented Oct 20, 2016

If buildmode=plugin works, great. Otherwise, this will need to wait for Go 1.9.

@rsc rsc modified the milestones: Go1.9Early, Go1.8Maybe Oct 20, 2016
@bradfitz bradfitz modified the milestones: Go1.10Early, Go1.9Early May 3, 2017
@bradfitz bradfitz added early-in-cycle A change that should be done early in the 3 month dev cycle. and removed early-in-cycle A change that should be done early in the 3 month dev cycle. labels Jun 14, 2017
@bradfitz bradfitz modified the milestones: Go1.10Early, Go1.10 Jun 14, 2017
@rsc rsc modified the milestones: Go1.10, Go1.11 Dec 1, 2017
@bradfitz bradfitz modified the milestones: Go1.11, Go1.12 May 18, 2018
@ianlancetaylor ianlancetaylor added this to the Go1.12 milestone Jun 1, 2018
@bcmills bcmills modified the milestones: Go1.12, Unplanned Oct 24, 2018
@bcmills bcmills added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. compiler/runtime Issues related to the Go compiler and/or runtime. labels Sep 7, 2023
@gopherbot gopherbot removed the NeedsFix The path to resolution is known, but the work has not been done. label Sep 7, 2023
@bcmills bcmills changed the title cmd/go: -buildmode=c-shared and dlopen-ing a shared library runtime,cmd/compile: -buildmode=c-shared and dlopen-ing a shared library Sep 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. early-in-cycle A change that should be done early in the 3 month dev cycle. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

8 participants