Proposal: Basic C API for the runtime #17048
Comments
|
Note that having such an API won't automatically solve #11100,
and it's stil very hard to implement go_runtime_shutdown as
proposed.
Besides, c-shared already allows writing programs where the
main is not written in Go. What's the real benefit of having such
a C API as compared to the current c-shared support?
|
|
There are two advantages. The less important one is that The bigger one is the handle feature. This makes the task of turning arbitrary Go data into a |
|
If you put aside #11100 for the moment (making go_runtime_shutdown work would also make -buildmode=c-shared work, it's the same problem), I believe you can build this interface today on top of c-shared. You can declare CHandle to be an int, allocate them by incrementing a global counter, and keep a map[interface{}]CHandle and map[CHandle]interface{} in a management package. This is roughly how the inside of the gobind tool in the mobile repository works. |
|
I actually don't quite understand why is the CHandle stuff is useful.
You're basically building a layer like JNI where C can't access anything
directly and every access must go through function calls.
It's even worse than what could be achieved with today's c-shared
mode; at least non-pointer data can be shared between Go and C
freely. And Go can access memory allocated by C freely too.
|
|
I believe the handle code can be written using an ordinary Go package. You can write a non-Go program that calls Go code today using either buildmode=c-shared or buildmode=c-archive. I don't understand how to implement |
|
@minux This is not intended to be like JNI at all. There is no analog of the JNI functions for accessing handles. In fact, the direct inspiration is Haskell's excellent Foreign Function Interface, which includes stable pointers (which refer to an arbitrary Haskell object, and which can then be passed back into Haskell). The purpose of a handle is not so that C code can manipulate Go objects. What it is is a way for Go code to wrap a reference to some Go value, such that a later callback from C into Go can use that value. This can be implemented in the runtime more efficiently than in pure Go. For one, using a deleted handle is use-after free. |
|
I don't think it requires any special facilities from the runtime
to be performant.
|
|
W.R.T. the level of indirection / pointer handle implementation the next On Tue, Oct 11, 2016 at 4:12 AM, Minux Ma notifications@github.com wrote:
|
|
Like @ianlancetaylor said, the key insight for cgo was that people writing these kinds of bindings want to write Go code, not C code, so we made it so that wrappers are written in Go. We really don't want to write wrappers in C. C code can call custom cgo-exported Go code, of course. Is there some specific functionality that this approach (write Go code called from C) doesn't work for? |
|
@rsc You are correct that writing stubs in C would not be a good thing. That is why .NET's PInvoke is I can think of two cases for this API. Neither involves writing stubs in C:
Since the latter use case is much more important to me, I am willing to restrict my proposal to it. |
|
It seems unlikely that you can unload a loaded Go plugin from a process.
Even it's possible, the Go runtime will make sure no Go pointer is leaked
when it's unloaded. I'm not sure what that has to do with this proposal.
(because there is no way to forcibly stop a goroutine from outside, the
host application can't simply ask the Go plugin to unload itself through
runtime: the Go plugin must have designed with such a feature.
|
|
@DemiMarie I have no objection in principle to support for shutting down the Go runtime, but I have no idea how to implement it. As @minux says, the Go runtime doesn't even have a way to shut down a stopped goroutine. I think that moving forward on this proposal would require a design that would make that possible. The design would have to not require major changes to the current runtime, because we do not want to rewrite everything to support a minor use case. Unless and until such a design is made, I would have to recommend that we decline this proposal. |
|
@DemiMarie Let's assume the "handle" half of the suggestion is off the table, since, as you agree, it can be done on the Go side. Then what's left is the go_runtime_shutdown. I think it would be plausible - provided it is not an invasive change - to add an exported C function like "bool go_runtime_shutdown(void)" that simply returns false (and does not do any shutdown) if there are any goroutines running. Then it's up to the author of the plugin to arrange that the goroutines are all gone before calling shutdown. If not, the shutdown doesn't shut anything down. As Minux and Ian said, this is the best one could really hope for. I don't understand the need for go_runtime_init. The runtime already initializes itself, as far as I understand. It especially doesn't make sense to me to call it and not know whether the runtime is initialized or not. Again, the caller really needs to understand the state for any of this to work. It can't be stabbing in the dark. |
|
Also, if the use case is Go plugin into host applications, usually, the
plugin interface
of the host application should already provide a method to indicate to the
plugin
that it should shutdown. As you said, the host application might not be
open source,
so even if we have the C API to shutdown a Go plugin, it won't be called by
the
host application directly, otherwise, the host application must have
designed for Go
plugins.
Therefore, the plausible (direct) caller for the proposed shutdown function
will be
the Go plugin itself. And in that case, why do we need a C interface for
that?
a runtime/plugin.Shutdown from the Go side should be enough (provided the
plugin
cleans up all the remaining goroutines first.) And even if a C callable
function is
needed, it's trivial to export that using regular cgo export mechanism.
The design of go_runtime_init and go_runtime_shutdown seems to allow
recursive
init calls, why is that necessary? The Go plugin will initialize itself
when loaded,
and all exported cgo calls will automatically wait until the runtime
initialization
has finished.
|
|
@minux The problem with a Go side shutdown function is that you can't run any (user) Go code after the runtime has shutdown – the runtime Go needs isn't there anymore. It would work only if the function never returned. Expecting the user code to shut down all goroutines is okay. |
|
I don't understand. The hypothetical runtime.Shutdown() will be similar to
os.Exit(), except perhaps the code needs to be sure no other goroutine
exists.
|
|
I wrote:
I think this proposal is essentially on hold until someone does the work to find out whether this is an invasive change or not. It may be necessary to have sysmon shut itself down, but hopefully there isn't more to do. |
Purpose
This is a proposal for a basic C API for the runtime. It allows for non-Go main programs and for using specially written Go libraries from any language that can call C functions, with a few additional features beyond what are currently available by
buildmode=c-shared. This will be a new buildmode.Background
Go can call almost any C function via CGo. However, the ability to have a non-Go main program is much more limited. Go does have
buildmode=c-shared, but it is limited by #11100 and also does not allow for useful features like passingGOMAXPROCSother than as an environment variable.Goals
Non-Goals
The API
Ground rules.
The entire API is provided in a single C header file,
goapi.h.Unless otherwise specified, any function can be called from multiple threads simultaneously. However, a single pointer in the API cannot be passed to API functions in 2 or more threads simultaneously, unless both functions take pointers to
constdata.The API itself
Initializes the Go runtime. Must be called before calling any other Go function. Thread-safe and may be called multiple times.
optionsis a pointer to an array of NUL-terminated strings. Each string corresponds to a matchingvoid*inargumentsthat corresponds to a matching pointer, which points to a flag that can be used to configure the runtime. Currently, the only string inoptionsthat is meaningful isGOMAXPROCS, which replaces theGOMAXPROCSenvironment variable: the correspondingvoid*must point to asize_t*. All other values are reserved and must not be used.If the defaults are OK, both
argumentsandoptionsmay be set to NULL. They are ignored if the runtime is already initialized.Returns 0 on success, a negative number on failure.
*errormsgis set toNULLand*already_initializedis set to the number of times the runtime has been initialized (including this one). It is safe to seterrormsgand/oralready_initializedto NULL, in which case they are not accessed.*errormsgpoints to a NUL-terminated, human-readable error message, and*already_initializedholds a negative number. Again, neither are written to if NULL.This function shuts down the runtime. It must be called once for each call to
go_runtime_init. Only the last such call has any effect.Once the number of calls to
go_runtime_shutdownequals the number of calls togo_runtime_init, this function terminates all active goroutines and resets any signal handlers.The following sentence may not be initially implemented:
Afterwards, it is safe to
dlclosethe shared library containing the runtime, or to restart the runtime withgo_runtime_init.A handle to an arbitrary Go object. Analogous to a Haskell stable pointer. It stays valid even across garbage collections.
A
Go_Handleis not guaranteed to point to valid memory. Dereferencing it invokes undefined behavior.Deletes the handle passed as argument, rendering it invalid and freeing all underlying resources. After this function is called, the object the handle pointed to may be garbage collected if there are no more references to it.
Duplicates the handle passed as argument. The returned handle points to the same object as the original handle, but must be deallocated separately.
Tests if the 2 handles passed are identical. Returns
trueif they point to the same object in memory. Otherwise, returnsfalse.The Go side
These are additional functions exposed from the runtime package.
The type of C handles to Go data. When passed via CGo, becomes a
Go_Handleon the C side.May be accessed by multiple goroutines simultaneously.
Creates a handle to a Go object that can safely be passed to C.
Duplicates the handle.
Tests if the 2 handles passed point to the same object; that is, if modifications of the object pointed to by one will affect the object pointed to by the other.
Deallocates the handle, rendering it invalid.
Dereferences the handle, returning the contained object. If the handle is invalid, invokes undefined behavior.
The text was updated successfully, but these errors were encountered: