Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime/cgo: Handles (small integers) cannot be passed as a C function's void* context #49633

Closed
adonovan opened this issue Nov 17, 2021 · 8 comments

Comments

@adonovan
Copy link
Member

adonovan commented Nov 17, 2021

A coworker recently hit trouble when using cgo.NewHandle to create a handle---an opaque reference to a Go value suitable for transiting C code---because the C function being called expected its context to be a void* value. Of course, C doesn't really care what bit pattern the void* value has, and will happily plumb any integer value, but the cgo-generated code at the Go/C boundary expects all pointer-shaped values to be legal Go pointers, whereas a cgo.Handle is a small integer. Converting the handle to a pointer using unsafe.Pointer(h) satisfies the type checker but causes the cgo-generated pointer validity check to fail nondeterministically, because some of these small integer values look to runtime.cgoCheckUnknownPointer like invalid Go pointers.

A workaround is to write a C wrapper function that performs the Handle-to-void* conversion, but this is tedious and easy to forget. Perhaps the cgo package could provide a variant of NewHandle that returns not an integer but an unsafe.Pointer value that is safe to pass to a C function's void* context argument. The implementation could, for example, use C.malloc(uintptr_t), and store the handle value in the heap cell. This would require a corresponding C.free operation, but the cgo.Handle.Delete method already imposes the necessary discipline.

A possible API could be the addition of methods such as:

type VoidPointer struct { cptr unsafe.Pointer } // pointer to C.malloc(uintptr) cell containing a cgo.Handle
func (Handle).ToVoidPointer() VoidPointer
func HandleFromVoidPointer(VoidPointer) Handle

Alternatively, the cgo package could declare a special type of unsafe.Pointer that is not subject to the usual cgo-generated pointer validity check.

@bcmills
Copy link
Contributor

bcmills commented Nov 17, 2021

Alternatively, the cgo package could declare a special type of unsafe.Pointer that is not subject to the usual cgo-generated pointer validity check.

Compare #46731. (See in particular https://golang.org/cl/345093, which proposed to add a cgo.Incomplete type.)

(But that is for pointers that are known not to point into the Go heap, not for non-pointers.)

@bcmills
Copy link
Contributor

bcmills commented Nov 17, 2021

Of course, C doesn't really care what bit pattern the void* value has, and will happily plumb any integer value,

That may be true of many C environments, but it is not portable. Per the C11 standard, §6.3.2.3, cl. 5 (emphasis mine):

An integer may be converted to any pointer type. Except as previously specified, the result is implementation-defined, might not be correctly aligned, might not point to an entity of the referenced type, and might be a trap representation.

So in general cgo programs must not assume that they can pass integers as type void*. Instead, I suggest passing either a C-allocated *uintptr_t or Go-allocated *cgo.Handle, depending on the lifetime requirements.

@bcmills
Copy link
Contributor

bcmills commented Nov 17, 2021

(A C API that does not care whether its argument is a pointer or an integer should accept a uintptr_t, not a void*.)

@adonovan
Copy link
Member Author

That may be true of many C environments, but it is not portable.

Fair point; not all the world's an Intel.

I suggest passing either a C-allocated *uintptr_t or Go-allocated *cgo.Handle

Ah, so you're allowed to pass a Go pointer so long as the variable it points to doesn't contain Go pointers? That's much simpler. Then consider this Issue a request for an example in the documentation of runtime/cgo.

@bcmills
Copy link
Contributor

bcmills commented Nov 17, 2021

FWIW, the current documentation for the pointer-passing rules is at https://pkg.go.dev/cmd/cgo#hdr-Passing_pointers.

@adonovan
Copy link
Member Author

@ianlancetaylor ianlancetaylor changed the title cgo.Handles (small integers) cannot be passed as a C function's void* context runtime/cgo: Handles (small integers) cannot be passed as a C function's void* context Nov 17, 2021
@gopherbot
Copy link

Change https://golang.org/cl/364774 mentions this issue: runtime/cgo: add example of Handle with void* parameter

@gopherbot
Copy link

Change https://golang.org/cl/366075 mentions this issue: misc/cgo/test: remove erroneous "extern" keyword on forward-declaration

gopherbot pushed a commit that referenced this issue Nov 22, 2021
This test otherwise fails to build on windows/arm64 as of CL 364774
due to a warning (promoted to an error) about a mismatched dllexport
attribute. Fortunately, it seems not to need the forward-declared
function in this file anyway.

Updates #49633
Updates #49721

Change-Id: Ia4698b85077d0718a55d2cc667a7950f1d8e50ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/366075
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
@golang golang locked and limited conversation to collaborators Nov 22, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants