Skip to content

encoding/gob: non-deterministic breakage encoding map[string]interface{} #3026

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
gopherbot opened this issue Feb 15, 2012 · 4 comments
Closed

encoding/gob: non-deterministic breakage encoding map[string]interface{} #3026

gopherbot opened this issue Feb 15, 2012 · 4 comments
Labels
FrozenDueToAge
Milestone
Go1

Comments

@gopherbot
Copy link
Contributor

by francis@tikitakistudio.com:

I'm very new to Go, so I apologize in advance if I'm missing something obvious here or
if the fault lies in my source...

I've attached a .go source file that demonstrates a strange non-deterministic behavior
with gob decoding map[string]interface{} data. Sometimes I get an incomplete map, other
times I get an error.  Here's an example of command line output after building and
running the attached file a number of times:

FrancisMBA:server francisli$ 6g gob_test.go 
FrancisMBA:server francisli$ 6l gob_test.6 
FrancisMBA:server francisli$ ./6.out 
Data before: map[foo:map[bar:baz] test:testing] 
Unable to deserialize data: gob: name not registered for interface: "ar 
\x06string\f\x05\x00\x03baz\x04test\x06string\f\t\x00\atesting 
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0
0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x
00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\
x00\x00\x00" 
Data after: map[foo:map[]] 
FrancisMBA:server francisli$ ./6.out 
Data before: map[foo:map[bar:baz] test:testing] 
Data after: map[foo:map[] test:testing] 
FrancisMBA:server francisli$ ./6.out 
Data before: map[foo:map[bar:baz] test:testing] 
Data after: map[foo:map[] test:testing] 
FrancisMBA:server francisli$ ./6.out 
Data before: map[foo:map[bar:baz] test:testing] 
Data after: map[test:testing foo:map[]] 
FrancisMBA:server francisli$ ./6.out 
Data before: map[foo:map[bar:baz] test:testing] 
Unable to deserialize data: gob: name not registered for interface: "ar 
\x06string\f\x05\x00\x03baz\x04test\x06string\f\t\x00\atesting 
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0
0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x
00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\
x00\x00\x00" 
Data after: map[foo:map[]] 
FrancisMBA:server francisli$ 

Some details:
Mac OS X 10.7.2, 64-bit Core i5
Running weekly.2012-02-07 as bundled in in appengine distribution:
go_appengine_sdk_darwin_amd64-go1beta2.zip

Discussion thread:
http://groups.google.com/group/google-appengine-go/browse_thread/thread/6cf16da95dd3de1b

Notable excerpt from David Symons:

On Wed, Feb 15, 2012 at 3:04 PM, Francis Li <fran...@tikitakistudio.com> wrote: 
> gob.Register(make(map[string]interface{})) 

You shouldn't need this. 
But this may indeed be a bug in the gob package. File a bug at 
http://code.google.com/p/go/issues/list with this reproduction case. 
Dave.

Attachments:

  1. gob_test.go (678 bytes)
@rsc
Copy link
Contributor

rsc commented Feb 15, 2012

Comment 1:

Confirmed by running program.
This may not be a valid use of interfaces in gobs since you
are not registering them, but then again they only hold 
basic language types.
Regardless, gob should do the same thing on each run of the
program.  While the encoded bytes may depend on iteration 
order, whether the program works at all should not.

Labels changed: added priority-go1, go1-must, removed priority-triage.

Owner changed to @robpike.

Status changed to Accepted.

@gopherbot
Copy link
Contributor Author

Comment 2 by francis@tikitakistudio.com:

I'll just add that I only started experiencing this issue after migrating my AppEngine
Go code from the previous stable release (API 3, r60.3) to the new beta SDK (API
go1beta, weekly.2012-02-07).

@robpike
Copy link
Contributor

robpike commented Feb 16, 2012

Comment 3:

Encode is correct. Decode is wrong and the true error was missed.
type definition {
    map "" id=65
        key id=6
        elem id=8
}
    "foo"
    interface value, type "map[string]interface {}" id=65; valueLength 20
        "bar"
        interface value, type "string" id=6; valueLength 5
        "baz"
    "test"
    interface value, type "string" id=6; valueLength 9
    "testing"
Unable to deserialize data: gob: gob: internal error: inconsistent indirection instr 1
ut 0
I am sending a fix that diagnoses the internal error correctly while I think about the
underlying problem. It's inconsistent because of a bug in how caching works for type
engines, and the order of creation is affected by the order of the data in the map.
I am attaching a pickled version of the bug that is 100% reliable. (you need to enable
encoding/gob/debug.go)

Attachments:

  1. gob.go (1051 bytes)

@robpike
Copy link
Contributor

robpike commented Feb 18, 2012

Comment 4:

This issue was closed by revision 420f713.

Status changed to Fixed.

@rsc rsc added this to the Go1 milestone Apr 10, 2015
@golang golang locked and limited conversation to collaborators Jun 24, 2016
@rsc rsc unassigned robpike Jun 22, 2022
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
Go1
Development

No branches or pull requests
3 participants
@rsc @robpike @gopherbot