Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/sys/unix: Pwritev and Preadv always allocate #57296

Closed
elagergren-spideroak opened this issue Dec 13, 2022 · 3 comments
Closed

x/sys/unix: Pwritev and Preadv always allocate #57296

elagergren-spideroak opened this issue Dec 13, 2022 · 3 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Performance
Milestone
Unreleased

Comments

@elagergren-spideroak
Copy link

elagergren-spideroak commented Dec 13, 2022 •
edited

What version of Go are you using (go version)?

$ go version
go version go1.19.3 darwin/amd64

But for sys/unix:

golang.org/x/sys v0.3.0 h1:w8ZOecv6NaNa/zC8944JTU3vz4u6Lagfk4RPQxv92NQ=
golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=

Does this issue reproduce with the latest release?

n/a

What operating system and processor architecture are you using (go env)?

n/a

What did you do?

Wrote a program that regularly calls unix.Pwritev with a similar number of discontinuous buffers.

What did you expect to see?

A way to call Pwritev without allocating a slice of Iovecs each time.

What did you see instead?

No way to do that.

I agree that using [][]byte is nicer, but unless the Preadv, Pwritev, etc. maintain a cache of Iovecs, each call necessarily allocates.
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Dec 13, 2022
@gopherbot gopherbot added this to the Unreleased milestone Dec 13, 2022
@randall77
Copy link
Contributor

It is possible to stack-allocate a reasonably-sized iovec that would cover a lot of cases.

func Pwritev(fd int, iovs [][]byte, offset int64) (n int, err error) {
	iovecs := make([]Iovec, 0, 8)
	iovecs = appendBytes(iovecs, iovs)

Where appendBytes appends an Iovec for each []byte argument. As long as there are at most 8 entries in iovs, it would not allocate.

(This is assuming pwritev doesn't escape its argument. I'm not sure if that is encoded somewhere or not?)

@thanm thanm added Performance NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Dec 14, 2022
@ericlagergren
Copy link
Contributor

@randall77 that seems to work, sending a CL

@gopherbot
Copy link

Change https://go.dev/cl/457815 mentions this issue: unix: avoid allocations for common uses of Readv, Writev, etc.

@golang golang locked and limited conversation to collaborators Dec 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Performance
Projects
None yet
Milestone
Unreleased
Development

No branches or pull requests
5 participants
@ericlagergren @randall77 @gopherbot @thanm @elagergren-spideroak