Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plugin: loading on machine with different GOPATH fails #26759

Open
suoigwg opened this issue Aug 2, 2018 · 16 comments
Open

plugin: loading on machine with different GOPATH fails #26759

suoigwg opened this issue Aug 2, 2018 · 16 comments
Labels
NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@suoigwg
Copy link

suoigwg commented Aug 2, 2018

What version of Go are you using (go version)?

go version go1.10.3 linux/amd64

Does this issue reproduce with the latest release?

Yes?

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/blablabla/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GORACE=""
GOROOT="/opt/go"
GOTMPDIR=""
GOTOOLDIR="/opt/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build657274199=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I build a go plugin from a docker image and copy it to a server. They have essentially the same go env except for the go path. (One was /go the other /home/username/go)

When I tried to load the plugin from other server, I always get

panic: plugin.Open("./parser"): plugin was built with a different version of package xxxx

It can only be fixed by setting the gopath the same as that in the docker image I used to build the plugin

I don't know if this is a bug or it is just how go plugin works.

Thanks.

@ALTree ALTree changed the title how does go plugin check package version plugin: loading on machine with different GOPATH fails Aug 2, 2018
@ALTree ALTree added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Aug 2, 2018
@ALTree ALTree added this to the Go1.12 milestone Aug 2, 2018
@randall77
Copy link
Contributor

I don't think that's intentional.
Can you run go version on your two different Go installations and report what they print?

The version mismatch is not between the plugin and the Go installation, it is between the Go installation that built the plugin and the Go installation that built the main program. Where was the main binary built? (You don't need the Go installation on the server at all, at least to run Go binaries and plugins.)

I can install two identical go instances at different paths, build the plugin with one and the main program with another, and the plugin loads just fine.

@suoigwg
Copy link
Author

suoigwg commented Aug 8, 2018

@randall77
They run exactly the same version of go(1.10.3). The only difference in go env is the go path.

Yes the main file is build by a different installation of go on a different machine(which only differs in gopath)

Basically what I am doing is building a plugin that references some model.go, and serve the model and plugin to the client so they can write program of their own.

that is

package model

type PublicModel struct{
	Msg string
}
package main

import "go-playground/model"

type api struct {}

func (i api)PublicApi() model.PublicModel {
	md := model.PublicModel{Msg: "Secret implementation"}
	return md
}


var Endpoint api


build a plugin.so and serve the so and model to user, the user may write a program on their machine like

package main

import (
	"plugin"
		"go-playground/model"
)

type endPoint interface {
	PublicApi() model.PublicModel
}
func main()  {
	plgin, err := plugin.Open("./plugin.so")
	if err != nil {
		panic(err)
	}
	symbol, err:= plgin.Lookup("Endpoint")
	if err != nil {
		panic(err)
	}
	b, ok := symbol.(endPoint)
	if !ok{
		panic("Type assertion fail")
	}
	md := b.PublicApi()
	println(md.Msg)
}

Essentially I need to hide the implementation of the PublicApi, I tried to build the plugin in a docker container(go path /go) and test it on a vm (go path /home/username/go). By testing I mean copy copying plugin.so ./model and stub.go to the corresponding directory and of course build the stub.go there, I always get that panic unless I change the go path to the same

@suoigwg
Copy link
Author

suoigwg commented Aug 8, 2018

@randall77
docker file to reproduce
environment 1

FROM ubuntu:16.04

ENV DEBIAN_FRONTEND noninteractive
ENV INITRD No
ENV LANG en_US.UTF-8
ENV GOVERSION 1.10.3
ENV GOROOT /opt/go
ENV GOPATH /go

RUN  apt-get update \
  && apt-get install -y wget \
  && apt-get install -y git \
  && apt-get install -y gcc \
  && rm -rf /var/lib/apt/lists/*

RUN cd /opt && wget https://storage.googleapis.com/golang/go${GOVERSION}.linux-amd64.tar.gz && \
    tar zxf go${GOVERSION}.linux-amd64.tar.gz && rm go${GOVERSION}.linux-amd64.tar.gz && \
    ln -s /opt/go/bin/go /usr/bin/ && \
    mkdir $GOPATH

build plugin

from env:1.0

WORKDIR /go/src/go-playground
COPY  ./ /go/src/go-playground
RUN  go build --buildmode=plugin -o plugin.so

copy the plugin from docker container

environment 2(differ only in gopath)

FROM ubuntu:16.04

ENV DEBIAN_FRONTEND noninteractive
ENV INITRD No
ENV LANG en_US.UTF-8
ENV GOVERSION 1.10.3
ENV GOROOT /opt/go
ENV GOPATH /another-go

RUN  apt-get update \
  && apt-get install -y wget \
  && apt-get install -y git \
  && apt-get install -y gcc \
  && rm -rf /var/lib/apt/lists/*

RUN cd /opt && wget https://storage.googleapis.com/golang/go${GOVERSION}.linux-amd64.tar.gz && \
    tar zxf go${GOVERSION}.linux-amd64.tar.gz && rm go${GOVERSION}.linux-amd64.tar.gz && \
    ln -s /opt/go/bin/go /usr/bin/ && \
    mkdir $GOPATH

use plugin in env2

from env:2.0
WORKDIR /another-go/src/go-playground
COPY ./stub.go ./
COPY ./plugin.so ./
COPY /model/ /another-go/src/go-playground/model

root@808cfd262c8d:/another-go/src/go-playground# go run stub.go
panic: plugin.Open("./plugin"): plugin was built with a different version of package go-playground/model

goroutine 1 [running]:
main.main()
/another-go/src/go-playground/stub.go:14 +0x259
exit status 2

@suoigwg
Copy link
Author

suoigwg commented Aug 8, 2018

It will work by using plugin in env1 in the same way

@randall77
Copy link
Contributor

I still can't reproduce your issue. Your plugin runs fine for me.
Please run go version on all the Go installations involved and report the results. Maybe your symlinks aren't putting the go binary sufficiently in front of a default Go installation in the path?

I've never used Docker, unfortunately, so I don't know what things like "use plugin in env2" mean.

@suoigwg
Copy link
Author

suoigwg commented Aug 8, 2018

go version from machine 1

go version go1.10.3 linux/amd64

go env in case you need more info

GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GORACE=""
GOROOT="/opt/go"
GOTMPDIR=""
GOTOOLDIR="/opt/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build029805510=/tmp/go-build -gno-record-gcc-switches"

go version from machine 2

go version go1.10.3 linux/amd64

go env from machine 2

GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/another-go"
GORACE=""
GOROOT="/opt/go"
GOTMPDIR=""
GOTOOLDIR="/opt/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build463889578=/tmp/go-build -gno-record-gcc-switches"

@randall77
Copy link
Contributor

Ok, my bad. I was thinking GOROOT, not GOPATH. I am now able to reproduce.
With the following directory structure:

./gopathA
./gopathA/src
./gopathA/src/model
./gopathA/src/model/model.go
./gopathB
./gopathB/src
./gopathB/src/model
./gopathB/src/model/model.go
./main.go
./plugin.go

If you do:

$ GOPATH=`pwd`/gopathA go build -buildmode=plugin  -o plugin.so plugin.go
$ GOPATH=`pwd`/gopathB go run main.go
panic: plugin.Open("./plugin"): plugin was built with a different version of package model

The version check is using the hash of some header information in the exported information from the plugin package. That exported information has the full path in it, not just the path from GOPATH. As such, it's different for the two otherwise identical plugin packages.

The full path just appears in the filenames. I'm not sure that's relevant information to be hashing. For source code from GOROOT, the path starts at $GOROOT/src/... but for source from GOPATH the full path is used. I can understand why, as GOPATH can have colon-separated paths and it would not be clear which one was the source of the file in question.

Seems related to #9206 and #16860.

@randall77 randall77 added NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Aug 11, 2018
@juhwany
Copy link

juhwany commented Sep 19, 2018

@randall77
It still happens in Go 1.11. Is there any progress about this issue?

@randall77
Copy link
Contributor

No progress. I think I understand the problem but I have no idea what the fix might be.

@BryceDFisher
Copy link

This is the same problem that I opened with #27062, except in my case the "vendor" directory was being included in the full file path, which messed up the package tracking from the plugin. Should I close my original issue to consolidate here?

@randall77
Copy link
Contributor

I don't think they are quite the same issue. This one is the exact same import path in both the plugin and main build. #27062 has paths differ by the vendor prefix.
Just the issue cross-reference should be fine. They are related, and may be fixed together, but they don't need consolidation.

@rsc
Copy link
Contributor

rsc commented Nov 14, 2018

Once we do #16860, we can make it kick in for plugins automatically. For now, nothing to do for Go 1.12.

@rsc rsc modified the milestones: Go1.12, Go1.13 Nov 14, 2018
@rsc rsc added the NeedsFix The path to resolution is known, but the work has not been done. label Nov 14, 2018
@gopherbot gopherbot removed the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Nov 14, 2018
@juhwany
Copy link

juhwany commented Jun 14, 2019

@rsc

Regarding #16860, It seems that Go 1.13 will support reproducible builds.

Will this loading plugin issue be solved at Go 1.13 ?

@andybons andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019
osdrv pushed a commit to awesome-flow/flow that referenced this issue Jul 29, 2019
…olang/go#26759 for more details

Signed-off-by: Oleg Sidorov <me@whitebox.io>
@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@mytototo
Copy link

Any news regarding this issue? I am struggling distributing plugins to third-party users. I have tried as many build options as possible but still no success.

@henri9813

This comment was marked as spam.

@Furzoom

This comment was marked as spam.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests