Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

math/big: Big.Int inconsistently marshals to JSON #28946

Closed
TACIXAT opened this issue Nov 26, 2018 · 5 comments
Closed

math/big: Big.Int inconsistently marshals to JSON #28946

TACIXAT opened this issue Nov 26, 2018 · 5 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone

Comments

@TACIXAT
Copy link

TACIXAT commented Nov 26, 2018

What version of Go are you using (go version)?

$ go version
go version go1.11 linux/amd64

Does this issue reproduce with the latest release?

Reproduces on go playground.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/dg/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/dg/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build283450947=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Marshal a big.Int to JSON and it doesn't work.

Marshal a big.Int to JSON when inside a struct and it doesn't work.

Marshal a big.Int to JSON when it's inside a slice of structs and it does work.

package main

import (
	"fmt"
	"math/big"
	"encoding/json"
)

type Simple struct {
	I big.Int
}

type List struct {
	S []Simple
}

func main() {
	i := big.Int{}
	b := []byte{
		0x55, 0x55, 0x55, 0x55, 0x55, 
		0x55, 0x55, 0x55, 0x55, 0x55}
		
	i.SetBytes(b)
	
	fmt.Println("b", b)
	fmt.Println("i", i)
	
	// marshal and unmarshal big int
	j, _ := json.Marshal(i)
	// fmt.Println("ji", j)
	
	i2 := big.Int{}
	_ = json.Unmarshal(j, &i2)
	fmt.Println("i2", i2)
	
	// marshal and unmarshal big int in struct
	s := Simple{
		I: i}
		
	j, _ = json.Marshal(s)
	// fmt.Println("js", j)
	
	s2 := Simple{}
	_ = json.Unmarshal(j, &s2)
	fmt.Println("s2", s2)
	
	// marshal and unmarshal big int in struct in list
	l := List{}
	l.S = append(l.S, s)
	
	j, _ = json.Marshal(l)
	// fmt.Println("jl", j)
	
	l2 := List{}
	_ = json.Unmarshal(j, &l2)
	fmt.Println("l2", l2)
}
b [85 85 85 85 85 85 85 85 85 85]
i {false [1431655765 1431655765 21845]}
i2 {false []}
s2 {{false []}}
l2 {[{{false [1431655765 1431655765 21845]}}]}

https://play.golang.org/p/1pI0OCtLQDa

What did you expect to see?

Either it to fail consistently or work consistently.

What did you see instead?

The big.Int sometimes marshals correctly and sometimes does not.

What do you want out of this issue?

I'm mostly just curious why it works sometimes.

@dgryski
Copy link
Contributor

dgryski commented Nov 26, 2018

You're ignoring the errors:

math/big: cannot unmarshal "{}" into a *big.Int

@TACIXAT
Copy link
Author

TACIXAT commented Nov 26, 2018

Yes, that happens because it does not marshal correctly. Why does it marshal correctly when it is in a slice of structs but not alone or simply in the struct?

@TACIXAT
Copy link
Author

TACIXAT commented Nov 26, 2018

For example, neither of these throw errors but fail to encode correctly.

package main

import (
	"fmt"
	"math/big"
	"encoding/json"
)

type Simple struct {
	I big.Int
}

type List struct {
	S []Simple
}

func main() {
	i := big.Int{}
	b := []byte{
		0x55, 0x55, 0x55, 0x55, 0x55, 
		0x55, 0x55, 0x55, 0x55, 0x55}
		
	i.SetBytes(b)
	
	fmt.Println("b", b)
	fmt.Println("i", i)
	
	// marshal and unmarshal big int
	j, err := json.Marshal(i)
	if err != nil {
		fmt.Println(err)
	}
	fmt.Println("ji", j)
	
	// marshal and unmarshal big int in struct
	s := Simple{
		I: i}
		
	j, err = json.Marshal(s)
	if err != nil {
		fmt.Println(err)
	}
	fmt.Println("js", j)
	
	// marshal and unmarshal big int in struct in list
	l := List{}
	l.S = append(l.S, s)
	
	j, _ = json.Marshal(l)
	fmt.Println("jl", j)
	
	l2 := List{}
	_ = json.Unmarshal(j, &l2)
	fmt.Println("l2", l2)
}
b [85 85 85 85 85 85 85 85 85 85]
i {false [1431655765 1431655765 21845]}
ji [123 125]
js [123 34 73 34 58 123 125 125]
jl [123 34 83 34 58 91 123 34 73 34 58 52 48 50 57 55 53 50 55 51 50 48 52 56 55 54 51 57 49 53 54 56 55 50 53 125 93 125]
l2 {[{{false [1431655765 1431655765 21845]}}]}

@AlexRouSg
Copy link
Contributor

https://golang.org/pkg/math/big/#Int.MarshalJSON
func (x *Int) MarshalJSON() ([]byte, error)
Note the pointer receiver.

It is failing to marshal in those situations because the MarshalJSON method has a pointer receiver and you are passing a value and reflect.ValueOf(i).CanAddr() is false so there is no way for the json package to call the method.

See https://golang.org/pkg/reflect/#Value.CanAddr

A value is addressable if it is an element of a slice, an element of an addressable array, a field of an addressable struct, or the result of dereferencing a pointer

@andybons andybons changed the title Big.Int inconsistently marshals to JSON math/big: Big.Int inconsistently marshals to JSON Nov 26, 2018
@andybons andybons added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Nov 26, 2018
@andybons andybons added this to the Unplanned milestone Nov 26, 2018
@andybons andybons added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Nov 26, 2018
@TACIXAT
Copy link
Author

TACIXAT commented Nov 27, 2018

Thanks!

@TACIXAT TACIXAT closed this as completed Nov 27, 2018
ingar added a commit to carbonfive/go-filecoin-rest-api that referenced this issue Oct 8, 2019
Move serialization concerns into the type
big.Int -> *big.Int for JSON marshalling (golang/go#28946 (comment))
ingar added a commit to carbonfive/go-filecoin-rest-api that referenced this issue Oct 8, 2019
Move serialization concerns into the type
big.Int -> *big.Int for JSON marshalling (golang/go#28946 (comment))
@golang golang locked and limited conversation to collaborators Nov 27, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests

5 participants