Issue 6392057: code review 6392057: encoding/gob: reduce decoder memory

Keyboard Shortcuts

	File
u :	up to issue
m :	publish + mail comments
M :	edit review message
j / k :	jump to file after / before current file
J / K :	jump to next file with a comment after / before current file
	Side-by-side diff
i :	toggle intra-line diffs
e :	expand all comments
c :	collapse all comments
s :	toggle showing all comments
n / p :	next / previous diff chunk or comment
N / P :	next / previous comment
<Up> / <Down> :	next / previous line
<Enter> :	respond to / edit current comment
d :	mark current comment as done

	Issue
u :	up to list of issues
m :	publish + mail comments
j / k :	jump to patch after / before current patch
o / <Enter> :	open current patch in side-by-side view
i :	open current patch in unified diff view

	Issue List
j / k :	jump to issue after / before current issue
o / <Enter> :	open current issue
# :	close issue

	Comment/message editing
<Ctrl> + s or <Ctrl> + Enter :	save comment
<Esc> :	cancel edit

Issue 6392057: code review 6392057: encoding/gob: reduce decoder memory (Closed)

Can't Edit
Can't Publish+Mail
Start Review

Created:
11 years, 9 months ago by r

Modified:
11 years, 9 months ago

Reviewers:

CC:
dsymonds, nigeltao, golang-dev

Visibility:
Public.

Description

encoding/gob: reduce decoder memory Gob decoding reads a whole message into memory and then copies it into a bytes.Buffer. For large messages this wastes an entire copy of the message. In this CL, we use a staging buffer to avoid the large temporary. Update issue 2539 RSS drops to 775MB from 1GB. Active memory drops to 858317048 from 1027878136, essentially the size of one copy of the input file.

Patch Set 1 #

Patch Set 2 : diff -r 2874979af37a https://code.google.com/p/go/ #

Patch Set 3 : diff -r 2874979af37a https://code.google.com/p/go/ #

Patch Set 4 : diff -r 2874979af37a https://code.google.com/p/go/ #

Total comments: 1

Patch Set 5 : diff -r ed493c4bb3e0 https://code.google.com/p/go/ #

Created: 11 years, 9 months ago

Download [raw] [tar.bz2]

		Unified diffs	Side-by-side diffs	Delta from patch set	Stats (+27 lines, -10 lines)			Patch
	M	src/pkg/encoding/gob/decoder.go	View	1 3	1 chunk	+27 lines, -10 lines	0 comments	Download

Messages

Total messages: 4

Expand All Messages | Collapse All Messages

Hello golang-dev@googlegroups.com, I'd like you to review this change to https://code.google.com/p/go/

11 years, 9 months ago (2012-07-12 04:36:37 UTC) #1

nigeltao

http://codereview.appspot.com/6392057/diff/7001/src/pkg/bytes/buffer.go File src/pkg/bytes/buffer.go (right): http://codereview.appspot.com/6392057/diff/7001/src/pkg/bytes/buffer.go#newcode108 src/pkg/bytes/buffer.go:108: b.buf = b.buf[0 : b.off+m] s/b.off+//? A test (in ...

11 years, 9 months ago (2012-07-12 11:01:46 UTC) #3

11 years, 9 months ago (2012-07-13 03:53:20 UTC) #4

*** Submitted as http://code.google.com/p/go/source/detail?r=10a3b26f9276 ***

encoding/gob: reduce decoder memory
Gob decoding reads a whole message into memory and then
copies it into a bytes.Buffer. For large messages this wastes
an entire copy of the message. In this CL, we use a staging
buffer to avoid the large temporary.

Update issue 2539
RSS drops to 775MB from 1GB.
Active memory drops to 858317048 from 1027878136,
essentially the size of one copy of the input file.

R=dsymonds, nigeltao
CC=golang-dev
http://codereview.appspot.com/6392057

Expand All Messages | Collapse All Messages