Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/json: Can improve json.Decoder while io.reader is bytes.Bufffer. #9706

Closed
mei-rune opened this issue Jan 28, 2015 · 5 comments
Closed

Comments

@mei-rune
Copy link

Can improve json.Decoder while reader is bytes.Bufffer, use it like this:

    buffer := bytes.NewBuffer(make([]byte, 0, 1024))
    for {
      ....   // read bytes to buffer from network
      dec := json.NewDecoder(buffer)   // buffer is *bytes.Buffer
      dec.Decode(&m)     // it should copy all bytes to internal buf.
      ....  // do some things.
      buffer.Reset()
   }

json.Decoder.Decode() should copy all bytes to internal buf in the readValue(), it is what I want to avoid。I want to reuse buffer and zero copy, Reducing the number of GC. #7709

@adg
Copy link
Contributor

adg commented Jan 28, 2015

What's the context? I'm all for reducing allocations, but I want to make sure that this is worthwhile before we investigate adding special cases for this.

@bradfitz
Copy link
Contributor

Reducing copies and making json.Decoder re-usable are separate issues. I don't think the copies (which don't contribute to GCs, only CPU) will matter as much as a Reset(io.Reader) method on json.Decoder, which is probably worth doing.

@mei-rune
Copy link
Author

I have a bad example, my idea is simply to reduce the memory copy and memory allocation . When the data is too large (size is 1m), will has too many copies and memory allocation in the readValue(), and I hope that we can avoid it.

      buffer := bytes.NewBuffer(make([]byte, 0, 1024))
      ....   // read bytes to buffer from network
      dec := json.NewDecoder(buffer)   // buffer is *bytes.Buffer
      dec.Decode(&m)     // it should copy all bytes to internal buf.
      ....  // do some things.

@mei-rune
Copy link
Author

I hope that we can avoid or improve it while reader is bytes.Buffer or bytes.Reader or strings.Reader.

@adg
Copy link
Contributor

adg commented Jan 29, 2015

I understand the desire to improve things, but we're not going to add special cases if they don't address a real issue.

@adg adg closed this as completed Jan 29, 2015
@golang golang locked and limited conversation to collaborators Jun 25, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants