Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/json: Marshaler/Unmarshaler not stream friendly #12001

Closed
lukescott opened this issue Aug 3, 2015 · 4 comments
Closed

encoding/json: Marshaler/Unmarshaler not stream friendly #12001

lukescott opened this issue Aug 3, 2015 · 4 comments

Comments

@lukescott
Copy link

The Marshaler/Unmarshaler interface deals with whole []byte slices:

type Marshaler interface {
        MarshalJSON() ([]byte, error)
}
type Unmarshaler interface {
        UnmarshalJSON([]byte) error
}

If you're dealing with a type that encodes to an array that has a large number of objects inside you have to encode all of into a single []byte.

The encoding/xml package is, unlike encoding/json, stream friendly:

type Marshaler interface {
        MarshalXML(e *Encoder, start StartElement) error
}
type Unmarshaler interface {
        UnmarshalXML(d *Decoder, start StartElement) error
}

With MarshalXML you can call e.Encode/e.EncodeElement/e.EncodeToken.

Since encoding/json is gaining Token()/EncodeToken() methods it would be really helpful to have a Marshaler/Unmarshaler interface that can take advantage of that. Perhaps something along the lines of:

type Encoder interface {
        EncodeJSON(e *Encoder) error
}
type Decoder interface {
        DecodeJSON(d *Decoder) error
}

Since Marshaler/Unmarshaler can't be changed. Or something like:

type Marshaler2 interface {
        MarshalJSON(e *Encoder) error
}
type Unmarshaler2 interface {
        UnmarshalJSON(d *Decoder) error
}
@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Aug 3, 2015
@mildred
Copy link

mildred commented Oct 30, 2015

This would be really nice to have. I looked a bit into the code and found that it won't be easy to use the Unmarshaler2 interface while just using the json.Unmarshal(...) function, because we don't have a Decoder object.

However, it's more easy to have this implemented when using Decoder.Unmarshal(...)

@JonathanFraser
Copy link

I think it's worth considering looking at the way the yaml package handles custom unmarshallers.

type Unmarshaller interface {
    UnmarshalYAML(unmarshal func(interface{}) error) error
}

type Marshaller interface {
    MarshalYAML() (interface{}, error)
}

While I would normally dislike empty interfaces when it comes to marshalling its kind of unavoidable. What I like about this is it allows the object so simply inject the marshallable type it wishes to use under the hood without exposing a raw byte string. This has a nice side effect of decoupling the encoding from the object. For example, rather than this be a MarshalYAML and UnmarshalYAML interface this could just be Marshal and Unmarshal. This is nice because it breaks dependancies, moreover, the user just needs to specify the underlying marshalling type and it doesn't really care if it's decoded as a stream or a buffer. It's completely agnostic to that.

@smyrman
Copy link

smyrman commented Jun 3, 2020

The linked issue #14750 is not dependent on or directly related to this issue. It just raises another potential use-case for the described interface.

The use-case is relevant for the described Decoder interface (ignoring the naming conflict) in particular. By passing along a (sub) json.Decoder instead of []byte to a DecodeJSON method, you could also pass along options set on the original decoder, such as UseNumber, DissalowUnknownFields etc. If it's indeed a sub-decoder/copy, then it should be possible to override these properties without affecting the parrent. E.g.

const jsonExample = `{"a": "foo", "b": "bar", "c": 42}`

type T struct {
   A string
   B string
   C interface{}
}

func (t *T) DecodeJSON(dec json.Decoder) error {
   // Rely on parent decoder option for DissalowUnknownFields.
    dec.UseNumber() // Override UseNumber for sub-decoder only.
    return dec.Decode(t)
}

@mvdan
Copy link
Member

mvdan commented Nov 24, 2020

For the sake of deduplicating encoding/json issues, I'm going to close this issue in favor of #7872 and #11046. There's also #33714 with a specific design and implementation.

@mvdan mvdan closed this as completed Nov 24, 2020
@golang golang locked and limited conversation to collaborators Nov 24, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

7 participants