Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unsafe: document valid uses of unsafe.Pointer for non-Go memory #42467

Closed
DemiMarie opened this issue Nov 9, 2020 · 14 comments
Closed

unsafe: document valid uses of unsafe.Pointer for non-Go memory #42467

DemiMarie opened this issue Nov 9, 2020 · 14 comments
Labels
Documentation FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone

Comments

@DemiMarie
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.14.10 linux/amd64

Does this issue reproduce with the latest release?

N/A

What operating system and processor architecture are you using (go env)?

N/A

What did you do?

Read the documentation for unsafe.Pointer

What did you expect to see?

A documented method for accessing memory outside of the garbage-collected heap, such as memory allocated by C’s malloc.

What did you see instead?

No such method.

@randall77
Copy link
Contributor

I'm not sure exactly what you are asking for here. What does "accessing memory" mean here? Do you mean allocating, or just reading & writing?

Reading and writing can be done on non-Go memory just as you would on Go heap memory. Cast the unsafe.Pointer to a concrete pointer type and go at it. (With the large caveat that the non-Go memory cannot contain pointers into the Go heap.)

For allocating, there are many possibilities. Package unsafe probably isn't the place to document those. For instance, sycall.Mmap or C.malloc would be documented in their packages, not in unsafe.

@DemiMarie
Copy link
Author

The problem is that the non-Go memory might contain invalid pointers, which proper C callers will never dereference.

@cagedmantis cagedmantis changed the title Document valid uses of unsafe.Pointer for non-Go memory unsafe: document valid uses of unsafe.Pointer for non-Go memory Nov 9, 2020
@cagedmantis cagedmantis added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Nov 9, 2020
@randall77
Copy link
Contributor

I see. Yes, we should probably say something about pointers-which-are-not-actually-pointers.
The short answer is use uintptr for those. You can't actually dereference them in Go, so there's no benefit to using unsafe.Pointer.

Note that C.malloc never returns an invalid pointer.

@ianlancetaylor
Copy link
Contributor

Note that there is only a potential problem when a value is copied into Go memory one way or another. A pointer value at rest in non-Go memory is never a problem.

@DemiMarie
Copy link
Author

Is it okay to have a pointer in the Go heap, that points to memory not allocated by Go?

@bcmills
Copy link
Contributor

bcmills commented Nov 13, 2020

Yes, as long as the pointer is to a valid address.

(A pointer to an entirely-unmapped address is not ok in general, but a pointer to an address mapped in by C code or by an mmap system call should be fine.)

@DemiMarie
Copy link
Author

I am a bit confused here: non-Go memory could have any value at all, so the Go runtime obviously needs to treat it specially. Why does it access the memory at all if it can’t use its value?

@ianlancetaylor
Copy link
Contributor

The Go runtime does not access non-Go memory. But the Go runtime does make some sanity checks on pointer values, and crashes if it finds a pointer that holds a clearly invalid value--such as a pointer that can't possibly point to any address mapped into memory.

I would encourage you to take a step back with your questions, which seem very specific and potentially confusing. Is there a broader question that you are trying to ask?

@DemiMarie
Copy link
Author

The short answer is that the current behavior makes automatic binding generation much more difficult.

@randall77
Copy link
Contributor

The short answer is that the current behavior makes automatic binding generation much more difficult.

Could you elaborate? What is "automatic binding generation" and how is it made difficult?

@seankhliao seankhliao added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Aug 27, 2022
@seankhliao seankhliao added this to the Unplanned milestone Aug 27, 2022
@gopherbot
Copy link

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)

@DemiMarie
Copy link
Author

The short answer is that the current behavior makes automatic binding generation much more difficult.

Could you elaborate? What is "automatic binding generation" and how is it made difficult?

Tools that generate Go code from C header files do not know if something with a pointer type in C is actually a valid pointer. Some C APIs use pointers to refer to non-pointer data, but this cannot be determined by just parsing C headers.

@bcmills
Copy link
Contributor

bcmills commented Sep 27, 2022

Some C APIs use pointers to refer to non-pointer data

Those APIs are not portable. Per the C11 standard, “the result is implementation-defined, might not be correctly aligned, might not point to an entity of the referenced type, and might be a trap representation.”

When passed to Go as an unsafe.Pointer, it happens to be equivalent to a trap representation.

@DemiMarie
Copy link
Author

Some C APIs use pointers to refer to non-pointer data

Those APIs are not portable. Per the C11 standard, “the result is implementation-defined, might not be correctly aligned, might not point to an entity of the referenced type, and might be a trap representation.”

They are portable to virtually all platforms I know of.

@golang golang locked and limited conversation to collaborators Sep 27, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Documentation FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests

7 participants