Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plugin: add Windows support #19282

Open
QuestionPython opened this issue Feb 24, 2017 · 80 comments
Open

plugin: add Windows support #19282

QuestionPython opened this issue Feb 24, 2017 · 80 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FeatureRequest NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Windows
Milestone

Comments

@QuestionPython
Copy link

QuestionPython commented Feb 24, 2017

hi
Plugin Pkg Work for Windows!?
i want use this for mac,linux,win,... os.
when(what time) fix this?

https://golang.org/pkg/plugin/

@bradfitz bradfitz changed the title Plugin Pkg for Windows! plugin: add Windows support Feb 24, 2017
@bradfitz bradfitz added this to the Unplanned milestone Feb 24, 2017
@bradfitz
Copy link
Contributor

There is currently nobody working on it, as far as I know.

/cc @alexbrainman @crawshaw

@QuestionPython
Copy link
Author

QuestionPython commented Feb 24, 2017

mean in go 1.8 , plugin pkg work for apple-mac,windows and more ?

@bradfitz
Copy link
Contributor

@QuestionPython, yes, it's even documented in multiple places:

https://golang.org/pkg/plugin/

Currently plugins only work on Linux.

https://golang.org/doc/go1.8#plugin

Plugin support is currently only available on Linux.

@alexbrainman
Copy link
Member

There is currently nobody working on it, as far as I know.

I am not working on it. Sorry.

Alex

@bradfitz
Copy link
Contributor

We delete all "me too" voting comments per https://golang.org/wiki/NoMeToo. Vote with emoji reactions at top instead.

@golang golang locked and limited conversation to collaborators Mar 3, 2017
@bradfitz
Copy link
Contributor

bradfitz commented Mar 3, 2017

The "me too" comments wouldn't stop, so this issue is now locked. If there are updates, they will be made here.

@0xdevalias
Copy link

Out of curiosity, do we know how much effort it would take to implement windows support? Or if there are any blockers to it (and what they are?)

@akavel
Copy link
Contributor

akavel commented Apr 23, 2018

Notably, the recently published Go kernel for Jupyter notebooks is using buildmode=shared, and thus doesn't currently support Windows natively. This is a very cool use case, adding a REPL-like live coding feature to the Go ecosystem, thus it would be really awesome if someone tried to start work on buildmode=shared on Windows to support this use case.

Similar to @0xdevalias , I'm quite interested in some hints as to what is missing for this to work on Windows? I'm especially curious what extra work is needed given that c-shared is already implemented on Windows?

@alexbrainman
Copy link
Member

@0xdevalias and @akavel I don't have any effort estimation or any hints as to what missing here. I have not actually looked at what is involved. I am so much behind at fixing old issues ...

Alex

@akavel
Copy link
Contributor

akavel commented Apr 23, 2018

@alexbrainman Thanks! I'll ask on golang-dev then, maybe someone else can shed some light (edit: link to the thread)

@0xdevalias
Copy link

0xdevalias commented Apr 23, 2018

There doesn't seem to be a huge amount to it in the src: https://github.com/golang/go/tree/master/src/plugin

My completely naive guess would be figuring the windows equivalents to the C-bindings in plugin_dlopen.go.

The main functions I can see there are:

  • https://linux.die.net/man/3/dlopen : The function dlopen() loads the dynamic library file named by the null-terminated string filename and returns an opaque "handle" for the dynamic library.
  • https://linux.die.net/man/3/dlsym : The function dlsym() takes a "handle" of a dynamic library returned by dlopen() and the null-terminated symbol name, returning the address where that symbol is loaded into memory.

Googling for "dlopen equivalent windows" led me to the following:

And "dlsym equivalent windows":

So from that, it sounds like we have the following premise to work from:

  • dlopen in *nix roughly maps to LoadLibrary in windows
  • dlsym in *nix roughly maps to GetProcAddress in windows

The main definitions in FlexDLL don't look too complex.. but there is quite a bit of extra code around those that may be required too:

Hopefully this helps scope out what will be required/start pointing in the right direction :)

@jclc
Copy link

jclc commented Aug 10, 2018

All the posts seem to be concerned with the loading of symbols, but does the compiler support producing a plugin (presumably DLL) on Windows?

@alexbrainman
Copy link
Member

does the compiler support producing a plugin (presumably DLL) on Windows?

It is possible to build Windows DLL from your Go code. You want -buildmode=c-shared 'go build' flag for that. See #26714 for an example. 'go build' command uses gcc under covers to build DLL.

Alex

@jclc
Copy link

jclc commented Aug 12, 2018

I've been hacking on this issue for a while and it seems to be going well for now. I've managed to load a dll built with -buildmode=c-shared and call its init function. The only limitation of this is that the plugin needs to have a main function or it won't compile. I'm developing on Linux using GCC and Wine. Just a few questions if anyone could clarify:

What exactly is going on in this function? The dlopen implementation calls this function and apparently returns the symbols; it doesn't work with Windows's shared objects.
https://github.com/golang/go/blob/master/src/runtime/plugin.go#L10

Secondly, I couldn't find any consistent guidelines for using wide strings with CGO so I ended up depending on unicode/utf16 and unicode/utf8. However, go/build/deps_test.go has pretty strict restrictions on which packages you can import. Is this a problem?

Edit: I guess this isn't so straightforward as I thought. -buildmode=plugin adds some metadata that is needed to find the exported symbols. Reading the PE data (using pev's readpe) doesn't show any of the functions that the plugin is meant to export, only init and main. When loading it, the init function is run implicitly.

@jonathanhecl

This comment has been minimized.

@Indribell
Copy link

This seems invaluable, to me. I work on Windows, primarily. Is this difficult to implement, or is it just not a priority?

Its not a priority seems to be the right answer. There are issues with Windows/Mac support. There is the issue with gopath. The issue with different go versions and they all date back to 2017.

The plugin feature is mostly a tech demo that for some unholy reason got released as a stable feature of the language. When in reality it needed to have been hidden until it was ready. Its not ready and definably not useable as a feature for clients ( we have burned our hands on that also! ). Imagine wanting to offer plugin support for your program using go... They run into the platform issues. The different compile version issues. The gopath issues...

For a language that pride itself in delaying for years, when it comes down to new features ( like generics ) because it needs to be done correctly. And then seeing Plugins in this state 4 years after its release is at best mindboggling.

If you want plugins, go the way slower grpc route, as their are plugins ( more stand alone solution that communicate over grpc ) that work. But at the cost of a 40 a 50 times performance hit, so never run hot code using those.

@edwingeng
Copy link

edwingeng commented Jul 24, 2021

FYI: I made a solution, https://github.com/edwingeng/hotswap, for hot reloading based on the plugin mechanism, with which you can develop and debug a plugin on Windows, and then run the plugin on Linux without changing any code.

P.S. Don't miss the --staticLinking argument.

@godcong

This comment has been minimized.

avelino added a commit to prest/prest that referenced this issue Jan 4, 2022
ref: golang/go#19282

Signed-off-by: Avelino <avelinorun@gmail.com>
avelino added a commit to prest/prest that referenced this issue Jan 5, 2022
ref: golang/go#19282

Signed-off-by: Avelino <avelinorun@gmail.com>
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 7, 2022
@naikrovek

This comment was marked as resolved.

@Linkangyis

This comment was marked as resolved.

@Linkangyis

This comment was marked as off-topic.

@zandercodes
Copy link

zandercodes commented Nov 16, 2022

Hey guys, I have tried to implement plugin for windows. Unfortunately the dynlink in the ASM for windows does not work. R15 can not be used as dynlink because this is a global variable.

Since I am not familiar with ASM it is hard to implement. Or can someone help?

sys_windows_amd64.s:219: when dynamic linking, R15 is clobbered by a global variable access and is used here: 00171

@scottctaylor12
Copy link

@zandercodes I'm not familiar with ASM either, but I would also like to implement plugins for windows. Is there a branch or fork that I can review to see what you've done so far?

@bigBron

This comment was marked as spam.

@Linkangyis
Copy link

https://github.com/Linkangyis/GoLangLinkDLL
我认为这个方案应该可行,只是没办法实现跟plugin一样的共享包使用共享内存空间的效果

@godcong
Copy link

godcong commented Apr 25, 2023

The current Plugin model has the following disadvantages

  1. now the Plugin can not do dynamic loading and destruction.
  2. only support Linux system, can not be used in other systems.

I think to do this, the following ideas can be considered.

  1. when compiling, you need to export the interface to call and compile the file xxx.plugin and xxx_plugin.go, one is a binary file, a go file
  2. In addition, the compilation can be with the system suffix, because the binaries of different systems are different. like xxx.plugin.x86.linux
  3. if the method needs to be called by Plugin, you need to implement the bridge method in the plugin.go file. Usually these should be generated automatically
  4. the automatically generated code may be more complex, you can specify some specifications. For example, the exported code must be commented with //plugin.
  5. loading can be done locally or from a repository if possible, and automatically device your own system configuration. plugin.Load(/mnt/d/xxx.plugin) or plugin.Load(github/golang/go/plugin/xxx.plugin) like this.
    If there is no corresponding system implementation, it can return not found.

If you can follow the above method to achieve. Then go will have its own plug-in system. Instead of relying on c or windows dll or something.

@eh-steve
Copy link

People might be interested in my (still janky) fork of pkujhd's goloader (tested on windows/amd64 as well as linux/amd64, linux/arm64, darwin/amd64, darwin/arm64) and how it differs from plugin.

It doesn't depend on libdl doing the legwork of symbol deduplication - it directly reads Go archive files and loads code and applies relocations.

It doesn't necessarily require -dynlink because it controls the address of the mmap'd segment to ensure PCREL relocs are within 32-bits of the first module's symbols.

@lvyonghuan
Copy link

I'm currently working on an embedded project that requires strong versatility. It has the potential to run on Linux or Windows. The fact that the plugin package only works on Linux is causing me a lot of headaches.

@cchamplin
Copy link

cchamplin commented Nov 1, 2023

@lvyonghuan I think the best path forward here at the moment is to leverage compilation to wasm via either tinygo or go with wasip1, and then load the the wasm 'plugin' in with something like https://github.com/tetratelabs/wazero. You can get pretty far with this approach, it checks a lot of boxes for dynamically loading and executing code.

@RichiCoder1
Copy link

Yah, it's not as clean as the go plugin interface, but https://extism.org/ is a great plugin option at this point.

@lvyonghuan
Copy link

lvyonghuan commented Nov 1, 2023

@lvyonghuan I think the best path forward here at the moment is to leverage compilation to wasm via either tinygo or go with wasip1, and then load the the wasm 'plugin' in with something like https://github.com/tetratelabs/wazero. You can get pretty far with this approach, it checks a lot of boxes for dynamically loading and executing code.

I don't have much knowledge about WebAssembly. Does it typically require a browser environment to function properly? In our environment, there may not be a browser.

@lesomnus
Copy link

lesomnus commented Nov 1, 2023

@lvyonghuan Does it typically require a browser environment to function properly? In our environment, there may not be a browser.

You probably want to look out something like WASM runtime -> https://github.com/appcypher/awesome-wasm-runtimes.
Indeed WASM was developed target to browsers but you can run WASM outside of the browser (with proper runtime).
Anyway, it doesn't seem appropriate to discuss that issue (about WASM) further in this thread.

@Linkangyis
Copy link

I'm currently working on an embedded project that requires strong versatility. It has the potential to run on Linux or Windows. The fact that the plugin package only works on Linux is causing me a lot of headaches.

给你提供一个思路

go->c->go 能跑起来,c做桥,但是在windows下,dll和主程序是几乎完全不同的内存空间,撑死加载一下func,引入相同的包实际内存地址也是完全不一样的

@Linkangyis
Copy link

I'm currently working on an embedded project that requires strong versatility. It has the potential to run on Linux or Windows. The fact that the plugin package only works on Linux is causing me a lot of headaches.

https://github.com/Linkangyis/GoLangLinkDLL

你可以尝试参考这个实现

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FeatureRequest NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Windows
Projects
None yet
Development

No branches or pull requests