Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plugin: build path contributes to types not equal #19233

Closed
zhangxu19830126 opened this issue Feb 22, 2017 · 10 comments
Closed

plugin: build path contributes to types not equal #19233

zhangxu19830126 opened this issue Feb 22, 2017 · 10 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@zhangxu19830126
Copy link

zhangxu19830126 commented Feb 22, 2017

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

1.8

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/opt/lib/go-path"
GORACE=""
GOROOT="/opt/lib/go"
GOTOOLDIR="/opt/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build171911883=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

I want to use custom interface based on go plugin, but i found it's not support.

func newExternalFilter(filterSpec *conf.FilterSpec) (filter.Filter, error) {
        p, err := plugin.Open(filterSpec.ExternalPluginFile)
        if err != nil {
            return nil, err
        }

        s, err := p.Lookup("NewExternalFilter")
        if err != nil {
            return nil, err
        }

        sf := s.(func() (filter.Filter, error))
        return sf()
    }

    // NewExternalFilter create a External filter
    func NewExternalFilter() (filter.Filter, error) {
        return &AccessFilter{}, nil
    }

What did you expect to see?

I expect this operation will success.

What did you see instead?

panic: interface conversion: plugin.Symbol is func() (filter.Filter, error), not func() (filter.Filter, error)

@crawshaw crawshaw self-assigned this Feb 22, 2017
@crawshaw
Copy link
Member

That's an interesting failure. Can you provide a functioning reproduction? (In particular, including the definition of filter.Filter.)

@zhangxu19830126
Copy link
Author

Thanks for your reply, :-)

definition of filter.Filter

package filter

import (
	"net/http"

	"github.com/valyala/fasthttp"
)

// Context filter context
type Context interface {
	SetStartAt(startAt int64)
	SetEndAt(endAt int64)
	GetStartAt() int64
	GetEndAt() int64

	GetProxyServerAddr() string
	GetProxyOuterRequest() *fasthttp.Request
	GetProxyResponse() *fasthttp.Response
	NeedMerge() bool

	GetOriginRequestCtx() *fasthttp.RequestCtx

	GetMaxQPS() int

	ValidateProxyOuterRequest() bool

	InBlacklist(ip string) bool
	InWhitelist(ip string) bool

	IsCircuitOpen() bool
	IsCircuitHalf() bool

	GetOpenToCloseFailureRate() int
	GetHalfTrafficRate() int
	GetHalfToOpenSucceedRate() int
	GetOpenToCloseCollectSeconds() int

	ChangeCircuitStatusToClose()
	ChangeCircuitStatusToOpen()

	RecordMetricsForRequest()
	RecordMetricsForResponse()
	RecordMetricsForFailure()
	RecordMetricsForReject()

	GetRecentlyRequestSuccessedCount(sec int) int
	GetRecentlyRequestCount(sec int) int
	GetRecentlyRequestFailureCount(sec int) int
}

// Filter filter interface
type Filter interface {
	Name() string

	Pre(c Context) (statusCode int, err error)
	Post(c Context) (statusCode int, err error)
	PostErr(c Context)
}

// BaseFilter base filter support default implemention
type BaseFilter struct{}

// Pre execute before proxy
func (f BaseFilter) Pre(c Context) (statusCode int, err error) {
	return http.StatusOK, nil
}

// Post execute after proxy
func (f BaseFilter) Post(c Context) (statusCode int, err error) {
	return http.StatusOK, nil
}

// PostErr execute proxy has errors
func (f BaseFilter) PostErr(c Context) {

}

plugin go file

package main

import (
	"C"
	"strings"
	"time"

	"github.com/CodisLabs/codis/pkg/utils/log"
	"github.com/fagongzi/gateway/pkg/filter"
	"github.com/valyala/fasthttp"
)

// AccessFilter record the http access log
// log format: $remoteip "$method $path" $code "$agent" $svr $cost
type AccessFilter struct {
}

// NewExternalFilter create a External filter
func NewExternalFilter() (filter.Filter, error) {
	return &AccessFilter{}, nil
}

// Name return name of this filter
func (f *AccessFilter) Name() string {
	return "HTTP-ACCESS"
}

// Pre pre process
func (f *AccessFilter) Pre(c filter.Context) (statusCode int, err error) {
	return 200, nil
}

// Post execute after proxy
func (f *AccessFilter) Post(c filter.Context) (statusCode int, err error) {
	cost := (c.GetStartAt() - c.GetEndAt())

	log.Infof("%s %s \"%s\" %d \"%s\" %s %s",
		GetRealClientIP(c.GetOriginRequestCtx()),
		c.GetOriginRequestCtx().Method(),
		c.GetProxyOuterRequest().RequestURI(),
		c.GetProxyResponse().StatusCode(),
		c.GetOriginRequestCtx().UserAgent(),
		c.GetProxyServerAddr(),
		time.Duration(cost))

	return 200, nil
}

// PostErr post error process
func (f *AccessFilter) PostErr(c filter.Context) {

}

// GetRealClientIP get read client ip
func GetRealClientIP(ctx *fasthttp.RequestCtx) string {
	xforward := ctx.Request.Header.Peek("X-Forwarded-For")
	if nil == xforward {
		return strings.SplitN(ctx.RemoteAddr().String(), ":", 2)[0]
	}

	return strings.SplitN(string(xforward), ",", 2)[0]
}

load plugin file

package proxy

import (
	"errors"
	"plugin"
	"strings"

	"github.com/fagongzi/gateway/pkg/conf"
	"github.com/fagongzi/gateway/pkg/filter"
)

var (
	// ErrKnownFilter known filter error
	ErrKnownFilter = errors.New("unknow filter")
)

const (
	// FilterHTTPAccess access log filter
	FilterHTTPAccess = "HTTP-ACCESS"
	// FilterHeader header filter
	FilterHeader = "HEAD" // process header fiter
	// FilterXForward xforward fiter
	FilterXForward = "XFORWARD"
	// FilterBlackList blacklist filter
	FilterBlackList = "BLACKLIST"
	// FilterWhiteList whitelist filter
	FilterWhiteList = "WHITELIST"
	// FilterAnalysis analysis filter
	FilterAnalysis = "ANALYSIS"
	// FilterRateLimiting limit filter
	FilterRateLimiting = "RATE-LIMITING"
	// FilterCircuitBreake circuit breake filter
	FilterCircuitBreake = "CIRCUIT-BREAKE"
	// FilterValidation validation request filter
	FilterValidation = "VALIDATION"
)

func newFilter(filterSpec *conf.FilterSpec) (filter.Filter, error) {
	if filterSpec.External {
		return newExternalFilter(filterSpec)
	}

	input := strings.ToUpper(filterSpec.Name)

	switch input {
	case FilterHTTPAccess:
		return newAccessFilter(), nil
	case FilterHeader:
		return newHeadersFilter(), nil
	case FilterXForward:
		return newXForwardForFilter(), nil
	case FilterAnalysis:
		return newAnalysisFilter(), nil
	case FilterBlackList:
		return newBlackListFilter(), nil
	case FilterWhiteList:
		return newWhiteListFilter(), nil
	case FilterRateLimiting:
		return newRateLimitingFilter(), nil
	case FilterCircuitBreake:
		return newCircuitBreakeFilter(), nil
	case FilterValidation:
		return newValidationFilter(), nil
	default:
		return nil, ErrKnownFilter
	}
}

func newExternalFilter(filterSpec *conf.FilterSpec) (filter.Filter, error) {
	p, err := plugin.Open(filterSpec.ExternalPluginFile)
	if err != nil {
		return nil, err
	}

	s, err := p.Lookup("NewExternalFilter")
	if err != nil {
		return nil, err
	}

	sf := s.(func() (filter.Filter, error))
	return sf()
}

@zhangxu19830126
Copy link
Author

You can find code in this project https://github.com/fagongzi/gateway/tree/go18-plugin-support.

  1. filter.Filter is in pkg/filter package.
  2. load plugin file in proxy/factory.go
  3. plugin go file is in another project.

@edaniels
Copy link
Contributor

edaniels commented Feb 23, 2017

Wouldn't this be similar to #18827? Technically the filter package lives at two distinct paths (gateway project and other project containing plugin) and go doesn't necessarily know the difference. This problem exists for non plugin code where two libraries try to interact with each other using the same type from two different versions of a package (at different paths but equivalent contents). See #12432

@zhangxu19830126
Copy link
Author

zhangxu19830126 commented Feb 27, 2017

Yes, @edaniels . You are right. Thanks very much

@quentinmit quentinmit changed the title go 1.8 plugin use custom interface plugin: build path contributes to types not equal Feb 27, 2017
@quentinmit quentinmit added the NeedsFix The path to resolution is known, but the work has not been done. label Feb 27, 2017
@quentinmit quentinmit added this to the Go1.9 milestone Feb 27, 2017
@sparrc
Copy link

sparrc commented Mar 9, 2017

I am having this same issue, though it manifests a little differently.

We have a project: github.com/influxdata/telegraf, which is meant to run our plugins. Each plugin has one dependency: github.com/influxdata/telegraf

This means that the plugins do not work unless they are built in the exact same GOPATH as the telegraf binary was built in.

ie, if I built the telegraf binary on my workstation from this GOPATH:

/home/sparrc/go/src/github.com/influxdata/telegraf

then build the plugins from a slightly different GOPATH (different home directory) on our CI system:

# telegraf dependency in /home/ubuntu/go/src/github.com/influxdata/telegraf
/home/ubuntu/go/src/github.com/influxdata/telegraf-plugins

I will get the following error:

2017/03/09 12:44:46 error loading [main/main.so]: plugin.Open: plugin was built with a different version of package github.com/influxdata/telegraf

This despite the fact that both telegraf and our plugins were built with the exact same commit hash of the github.com/influxdata/telegraf project. The only difference is the directory that github.com/influxdata/telegraf was located in at build time.

@bradfitz bradfitz modified the milestones: Go1.9, Go1.10 Jun 7, 2017
@easeway
Copy link

easeway commented Jun 10, 2017

The cause is the logic in func genhash (src/cmd/link/internal/ld/lib.go) is problematic: it simply calculate SHA1 of first line of __.PKGDEF plus the export data (between first \n$$ and second \n$$), however, the export data contains strings of full source paths. If a package is shared by the host executable and plugin, but they are built in different environment with different source paths, without change of any code, the hash calculated of this shared package is different in host executable and plugin. That causes the error plugin.Open: plugin was built with a different version of package ...

@crawshaw
Copy link
Member

I believe this was fixed by http://golang.org/cl/63693. Please comment if you still see this at HEAD. (See also, #21373.)

@alperkose
Copy link

alperkose commented May 23, 2018

Late note: I found that the issue is addressed at #18827

A similar issue exists for builds with vendoring support. I've created a sample repository which contains the examples and my observations: https://github.com/alperkose/golangplugins

I've run the example with go version go 1.10.2 darwin/amd64

The issue goes away if vendor folder is not used.

@juhwany
Copy link

juhwany commented Sep 19, 2018

@crawshaw

This issue is reproducible in Go 1.11 with simple plugin usage scenario like the below.

Directory structure

- gos (not GOAPTH)
       \_ sample_plugin (plugin module)
          \_ main.go
          \_ go.mod
       \_ server (server module, main program)
           \_ main.go
           \_ apis
                  \_ do.go
  1. go.mod for plugin
module github.com/xxx/sample_plugin

require (
	github.com/xxx/server v2.0.0
)
  1. plugin source (sample.go)
package main

import (
	"github.com/xxx/server/apis"
)

func Register(t apis.Test) {
       t.DoSomething()
...
}
...
  1. server source (do.go)
package apis 

type Test interface { // interface api which server exposes
        DoSomething()
}
  1. build plugin with go build -buildmode=plugin (not in GOPATH)
  2. build main program (of course, after checking out git tag v2.0.0, and not in GOPATH)
  3. run main program and when it try to load plugin, it failed with plugin was built with a different version of package github.com/x/server/apismessage

There's only one common dependency package(github.com/xxx/server/apis) between plugin and main program and the package belongs to server module as you can see the above directory structure.
I could not understand why it fails even though the referenced package version is same.

@golang golang locked and limited conversation to collaborators Sep 19, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

10 participants