How would this be implemented?

func Marshal(v any) ([]byte, error) {
	if seq, ok := v.(iter.Seq[T]); ok {
		// It's a sequence
	}
}

is obviously not possible.

Ok, it's possible with reflection, see #61897 (comment)

I would love for there to be a general-purpose API to dynamically represent types that is semantically equivalent to a Go slice or map, thus container types like linked lists or b-trees don't need custom MarshalJSON and UnmarshalJSON methods.

For this to happen I would expect:

• The API to support both Marshal and Unmarshal. At present, the proposal only works for Marshal, but I'd argue that we need a solution for Unmarshal as well.
• A universal interface for describing ordered values:
  • (for marshal) iterate through values
  • (for unmarshal) ability to reset the sequence and to append to the sequence
• A universal interface for describing unordered key-value pairs:
  • (for marshal) iterate through key-value pairs
  • (for unmarshal) ability to reset the mapping and to get or set arbitrary keys

@gophun, the suggested implementation using reflect.Value.Call would have poor performance. In addition to my concerns about asymmetry between marshal and unmarshal, we would need first-class support or optimizations in "reflect" to make iteration fast.

To encode and decode iter.Seq using JSON, one way is to implement the json.Marshaler and json.Unmarshaler interfaces for iter.Seq.

Here's an example implementation:

func (s Seq[V]) MarshalJSON() ([]byte, error) {
	var buf bytes.Buffer
	buf.WriteByte('[')

	first := true
	for v := range s {
		if !first {
			buf.WriteByte(',')
		}
		first = false

		bytes, err := json.Marshal(v)
		if err != nil {
			return nil, err
		}
		buf.Write(bytes)
	}
	buf.WriteByte(']')

	return buf.Bytes(), nil
}

func (s *Seq[V]) UnmarshalJSON(bytes []byte) error {
	var vs []V
	if err := json.Unmarshal(bytes, &vs); err != nil {
		return err
	}
	*s = Iter(vs)
	return nil
}

This implementation allows iter.Seq to be marshaled into JSON format and unmarshaled from JSON format seamlessly.

https://go.dev/play/p/Jqg81JhEr9_f?v=gotip

To be an appropriate solution, I would expect the following properties:

• Be streaming where no additional memory is needed other than the underlying storage of the backing store for the sequence. In this situation, we're allocating a new []V and ignoring whatever storage was backing the iterator.
• To have no dependency on the "json" package. That is, the idea should work for XML, CBOR, YAML, etc. without forcing iter.Seq to have special knowledge of all of these.

Iterators as we currently (experimentally) have them (i.e. iter.Seq) are single-pass and read-only (what C++ would call an "input iterator"), which is why I only brought up Marshal and not Unmarshal.

To unmarshal, we'd need something equivalent to C++'s Output Iterator, then we could use an equivalent to a back_insert_iterator (we'd probably call it "append" instead of "back insert"). Once we have that, I can envision something similar to json.Decoder.UseNumber() for defining a custom way to unmarshal arrays. Interesting to think about, but out of scope of this ticket, I think.

Then again, making Marshal and Unmarshal asymmetric isn't great, either. For example let's look at the types generated by protoc for grpc/protobuf. When I make a request, I would like to be able to pass a seq.Iter for repeated fields. But when I receive a request, a slice is more convenient, as I can e.g. check its length before iterating through it. Besides, changing that would be a breaking change.

Does that mean protoc would need to generate two different versions of each message for sending and receiving? That does not seem desirable. If I just want to forward (part of) an incoming message to another service, I don't want to have to convert it first.

It is possible in principle to return an iter.Seq that closes over a json.Decoder that consumes the array in a streaming fashion: https://go.dev/play/p/6G4GTrHMNo4?v=gotip

But of course the first round of iterators only supports infallible iterators, so any attempt to delay doing the JSON parsing creates the possibility of errors in a place where they cannot be handled.

In principle it could call json.Valid first to find a subset of possible errors before beginning iteration, at the expense of walking over the tokens twice. Even that isn't really sufficient though, because it's also possible that the syntactically-valid values inside are not compatible with type V. As far as I can tell the only way to deal with that is to a full parsing pass first, at which point buffering the result into memory to return during iteration is probably (but not necessarily) a win vs. parsing it over again during iteration. 🤷‍♂️

To me this just feels like a natural consequence of not supporting fallible iterators: it isn't appropriate to use the iterator abstraction for anything that can dynamically fail, and JSON parsing can dynamically fail.

(I acknowledge that this alternative is also JSON-specific and so fails the goal of being generalizable over multiple serialization formats, but I think the inability to handle errors disqualifies this approach even before considering any formats beyond JSON.)