Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

encoding/gob: Gob encoder's buffering all data in memory may cause oom #8316

Closed
gopherbot opened this issue Jul 2, 2014 · 5 comments
Closed

encoding/gob: Gob encoder's buffering all data in memory may cause oom #8316

gopherbot opened this issue Jul 2, 2014 · 5 comments
Labels
FrozenDueToAge Unfortunate

Comments

@gopherbot
Copy link

by daviddengcn:

My program is saving a big data structure that is about half the current memory. I found
`Encode.EncodeValue` will firstly buffer all the data to the memory (in enc.byteBuf),
then write it as a whole (by calling enc.writeMessage). This consumes almost the same
amout of memory as the value to encode. My program then occasionally panic because of
out of memory.

Can we make it better?

go version go1.3 linux/386
@ianlancetaylor
Copy link
Contributor

Comment 1:

Labels changed: added repo-main, release-none.

@robpike
Copy link
Contributor

robpike commented Jul 2, 2014

Comment 2:

Infeasible.

Status changed to Unfortunate.

@gopherbot
Copy link
Author

Comment 3 by daviddengcn:

Can anybody technically explain why infeasible?

@robpike
Copy link
Contributor

robpike commented Jul 2, 2014

Comment 4:

Messages in the stream are preceded by their byte count for the benefit of the decoder.
Without major restructuring and overhead, there is no way to preserve compatibility
without keeping the encoded message in memory so its length can be transmitted first.

@gopherbot
Copy link
Author

Comment 5 by daviddengcn:

OIC, thanks!

@golang golang locked and limited conversation to collaborators Jun 25, 2016
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge Unfortunate
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ianlancetaylor @robpike @gopherbot