Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

encoding/gob: Decoder cannot be reused if a composite types are involved #29766

Closed
myitcv opened this issue Jan 16, 2019 · 4 comments
Closed

encoding/gob: Decoder cannot be reused if a composite types are involved #29766

myitcv opened this issue Jan 16, 2019 · 4 comments
Labels
FrozenDueToAge

Comments

@myitcv
Copy link
Member

myitcv commented Jan 16, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12beta2 linux/amd64

Does this issue reproduce with the latest release?

Testing with beta pre the 1.12 release.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/myitcv/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/myitcv/gostuff"
GOPROXY=""
GORACE=""
GOROOT="/home/myitcv/gos"
GOTMPDIR=""
GOTOOLDIR="/home/myitcv/gos/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/myitcv/gostuff/src/github.com/myitcv/playground/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build266146048=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Reusing a gob.Decoder with a non-composite type works just fine:

func TestGobDecodeInt(t *testing.T) {
	var v, v2 int
	buf := bytes.NewBuffer(nil)
	e := gob.NewEncoder(buf)
	if err := e.Encode(v); err != nil {
		panic(err)
	}
	r := bytes.NewReader(buf.Bytes())
	d := gob.NewDecoder(r)
	fmt.Println("TestGobDecodeInt")
	for i := 0; i < 5; i++ {
		if err := d.Decode(&v2); err != nil {
			panic(err)
		}
		fmt.Printf("  Decoded: %v\n", v2)
		r.Reset(buf.Bytes())
	}
}

gives:

TestGobDecodeInt
  Decoded: 0
  Decoded: 0
  Decoded: 0
  Decoded: 0
  Decoded: 0

However where a composite type is involved, things start to break down:

func TestGobDecodeStruct(t *testing.T) {
	var v, v2 struct{}
	buf := bytes.NewBuffer(nil)
	e := gob.NewEncoder(buf)
	if err := e.Encode(v); err != nil {
		panic(err)
	}
	r := bytes.NewReader(buf.Bytes())
	d := gob.NewDecoder(r)
	fmt.Println("TestGobDecodeStruct")
	for i := 0; i < 5; i++ {
		if err := d.Decode(&v2); err != nil {
			panic(err)
		}
		fmt.Printf("  Decoded: %v\n", v2)
		r.Reset(buf.Bytes())
	}
}

gives:

TestGobDecodeStruct
  Decoded: {}
--- FAIL: TestGobDecodeStruct (0.00s)
panic: extra data in buffer [recovered]
        panic: extra data in buffer

goroutine 8 [running]:
testing.tRunner.func1(0xc000104300)
        /home/myitcv/gos/src/testing/testing.go:827 +0x388
panic(0x567dc0, 0xc000011ac0)
        /home/myitcv/gos/src/runtime/panic.go:522 +0x1b5
github.com/myitcv/playground.TestGobDecodeStruct(0xc000104300)
        /home/myitcv/gostuff/src/github.com/myitcv/playground/main_test.go:49 +0x3be
testing.tRunner(0xc000104300, 0x5a3f48)
        /home/myitcv/gos/src/testing/testing.go:862 +0xc0
created by testing.(*T).Run
        /home/myitcv/gos/src/testing/testing.go:913 +0x357

Same behaviour observed if a slice or map are involved.

What did you expect to see?

I'm unclear whether this is to be expected or not. I don't understand encoding/gob particularly well, so there might be reasons why reusing a Decoder in this way might break.

If this is expected, then I'd suggest documenting the restriction in some way.

Otherwise, I guess this is a bug?

cc @robpike @mvdan
@mvdan
Copy link
Member

mvdan commented Jan 16, 2019

One interesting observation is that gob.Encoder can be reused just fine. Other encoding types, such as json.Encoder and json.Decoder, can be reused too. So I'd expect the gob decoder to be the same, even though this isn't explicitly documented.

@robpike
Copy link
Contributor

robpike commented Jan 16, 2019

This is working as intended. The output from the Encoder is a stream, and is meant to be read, once, as a stream. You're using it wrong.

Mechanically, if the stream contains type information, as it will if there is a non-base type, then as the Decoder runs it will change state to absorb that type. That means the second time you run it with the same data, it will be inconsistent and break.

@myitcv
Copy link
Member Author

myitcv commented Jan 17, 2019

Thanks for confirming, @robpike.

@myitcv myitcv closed this as completed Jan 17, 2019
@mvdan
Copy link
Member

mvdan commented Jan 17, 2019

Thanks for the quick reply, Rob. I wonder if the decoder could give a better error message if it is misused like this. Alternatively, if this happened often enough, perhaps we could add a warning on the Decoder type.

@golang golang locked and limited conversation to collaborators Jan 17, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@myitcv @mvdan @robpike @gopherbot