Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: bytes: add Buffer.Init #53443

Closed
cwbhhjl opened this issue Jun 18, 2022 · 10 comments
Closed

proposal: bytes: add Buffer.Init #53443

cwbhhjl opened this issue Jun 18, 2022 · 10 comments

Comments

@cwbhhjl
Copy link

cwbhhjl commented Jun 18, 2022

This method adds the ability to replace a Buffer's buf.

Currently bytes.NewBuffer can be used to initialize Buffer with a custom buf, or we can copy a custom buf into Buffer via Write, but in performance-sensitive scenarios, I think it would be useful to avoid heap allocation or memcpy. So I want to add a method to replace a Buffer's buf.

a possible implementation:

func (b *Buffer) Init(buf []byte) {
	b.buf = buf
	b.off = 0
	b.lastRead = opInvalid
}
@seankhliao
Copy link
Member

how is this different from Reset?

@cwbhhjl
Copy link
Author

cwbhhjl commented Jun 18, 2022

how is this different from Reset?

If we get the data and want to wrap it in Buffer, currently we can

  1. bytes.NewBuffer(buf), may need to allocate on the heap
  2. already have a buffer
    buffer.Reset()
    buffer.Grow(len(buf))
    buffer.Write(buf)
    need a copy

@dsnet dsnet changed the title proposal: bytes/buffer: add Buffer.Init([]byte) proposal: bytes: add Buffer.Init Jun 18, 2022
@ianlancetaylor
Copy link
Contributor

Can you show an example where this comes up? Thanks.

(I think you could actually do this today via

    b.Reset(buf)
    b.Grow(len(buf))

but I'm not sure we want to guarantee that.)

@ianlancetaylor ianlancetaylor added this to Incoming in Proposals (old) Jun 19, 2022
@cwbhhjl
Copy link
Author

cwbhhjl commented Jun 19, 2022

@ianlancetaylor

Can you show an example where this comes up? Thanks.

I am using an encoding library similar to protobuf, the encoder is something like

func Encode(Marshaler m) []byte {
    bzSize := m.Size()
    var buf = bytes.NewBuffer(make([]byte, 0, bzSize))
    m.MarshalTo(buf)
    return buf.Bytes()
}

buf will escape to the heap because of the interface, If sync.Pool is used, a copy is required at the end.

func Encode(Marshaler m) (bz []byte) {
    bzSize := m.Size()
    var buf = pool.Get().(*bytes.Buffer)
    buf.Reset()
    buf.Grow(bzSize)
    m.MarshalTo(buf)
    bz = make([]byte, bzSize)
    copy(bz, buf.Bytes())
    pool.Put(buf)
    return
}

It is used in performance sensitive scenarios, so I want to reduce allocation or copy

@DeedleFake
Copy link

Perhaps SetBytes() would be a better name?

@cwbhhjl
Copy link
Author

cwbhhjl commented Jun 22, 2022

Perhaps SetBytes() would be a better name?

@DeedleFake good idea, Buffer already has a Bytes method, I agree that SetBytes is a better name

@rsc
Copy link
Contributor

rsc commented Jun 22, 2022

This doesn't seem like a terribly common operation, and the proposed b.Init(buf) can already be written as *b = bytes.NewBuffer(buf).

@rsc
Copy link
Contributor

rsc commented Jun 22, 2022

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc rsc moved this from Incoming to Active in Proposals (old) Jun 22, 2022
@rsc rsc moved this from Active to Likely Decline in Proposals (old) Jul 1, 2022
@rsc
Copy link
Contributor

rsc commented Jul 1, 2022

Based on the discussion above, this proposal seems like a likely decline.
— rsc for the proposal review group

@rsc rsc moved this from Likely Decline to Declined in Proposals (old) Jul 13, 2022
@rsc
Copy link
Contributor

rsc commented Jul 13, 2022

No change in consensus, so declined.
— rsc for the proposal review group

@rsc rsc closed this as completed Jul 13, 2022
@golang golang locked and limited conversation to collaborators Jul 13, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
No open projects
Development

No branches or pull requests

6 participants