I am continuing my implementation of #39528, and the incompleteness and mutability of the few reflect.Type created with the proposed new function Named is spreading to all kinds of types, through the functions ArrayOf ChanOf MapOf FuncOf SliceOf and StructOf. It starts looking like a major refactoring of reflect.Type methods.

So I welcome a different API that avoids this problem. Out of curiosity: is there a reason why you did not try to leverage go/types.Type API?

Out of curiosity: is there a reason why you did not try to leverage go/types.Type API?

I briefly considered it but discarded the idea for the simple reason that I'm not terribly familiar with that API, which, at first sight, seemed unnecessarily complex to me. My interest in the reflect package comes from writing generic pieces of code, and while I might be wrong, I suspect it's similar for most users, so I tried to make the TypeDesc API as similar as possible to the familiar Type API. I understand that for people whose primary interest in reflect comes from parsing actual Go code, priorities might be different.

I'm not sure how important that actually is, but another aspect to consider might be the dependencies of the reflect package. I might be misinterpreting this, but it seems to me the implementers went to certain lengths to avoid pulling in "convenience" packages such as fmt. The biggest dependencies currently appear to be Unicode related. go/types with its own dependencies would significantly increase the size of reflect dependencies.

The reflect package can't depend on go/types, because go/types depends on fmt and fmt depends on reflect. It would be quite awkward to rewrite go/types such that it does not depend on reflect in any way.

This is certainly a complex API to design.
I understand the need in a Go interpreter, but it's unclear that the benefit outweighs the very significant API cost.
We should keep thinking about whether there are ways to simplify the cost (API surface, complexity of implementation).

Maybe it would make sense to have a separate package with the same constructors as reflect but that operates only on these incomplete types:

package incomplete // import "reflect/incomplete"

type Type ...

func Complete(list []Type) []reflect.Type
func Of(reflect.Type) Type
func StructOf
PointerTo
ChanOf
MapOf
...

That would at least solve the "Desc everywhere" problem. I'm not sure how much that would end up being a second copy of reflect. And I'm not sure how methods get attached, if anyone wants to try to figure that out.

An external package that allows building recursive reflect.Types could in theory depend on go/types.

Then the most minimal API is probably:

package convert // import "reflect/convert"
import (
    "go/types"
    "reflect"
)
func ToGoType(reflect.Type) *types.Named
func ToReflect([]*types.Named) []reflect.Type

where for simplicity ToGoType only accepts named (i.e. defined) reflect.Types.

To be implementable, it probably also needs this constraints: ToReflect can only convert types.Type which satisfy one of the two conditions:

• either returned by a previous call of ToGoType - for symmetry and invertibility
• or has an underlying type that, when traversed recursively, contains only unnamed types (which are themselves traversed recursively), plus named types returned by a previous call of ToGoType, plus named types passed to the same call of ToReflect - the idea is: you can convert mutually recursive named types A and B by calling ToReflect([]*types.Named{A, B})

This would allow a user to collect the named reflect.Types he wants, convert them to *types.Named, build a new *types.Named (say X) from them with the go/types API - which supports creating new named types and recursive types - then convert such new named type X to a reflect.Type

Clearly, reflect/convert would need to access some a lot of internal features and types of reflect: I'd say at least rtype, arrayType, chanType, mapType, funcType, interfaceType, ptrType, sliceType and structType.

Such API could even (if it's implementable) create new interface types - something that reflect cannot do yet.

And it would not need the proposed reflect.Type.Underlying()

Update: or even

package convert // import "reflect/convert"
import (
    "go/types"
    "reflect"
)
func ToGoType(reflect.Type) types.Type
func ToReflect([]types.Type) []reflect.Type

@rsc Separate package sounds good.

There would certainly be some overlap with existing reflect functionality. It seems unnatural exclude "incidentally" non-recursive types (maybe even unnamed types if there is a Complete API) from the new package, so the *Of functions of reflect would become redundant in the sense that the new API could be used to the same effect (the original *Of functions would still be easier to use for the quick creation of a non-recursive, unnamed type).

@cosmos72 your idea to leverage the go/types package sounds intriguing, especially given it comes with type checking facilities. I've never used it, though, and glancing at it, the API seems quite complex to me, and also not quite geared towards the kind of runtime type creation you'd want in reflect. For example, to create a named recursive type, you'd need a call to NewPackage, NewTypeName and NewNamed, and I don't know what would be sensible arguments for pos and path in a reflectesque context. Also, the type checker only works on an entire file set.

As for implementation complexity, I guess some stuff would probably have to be moved from reflect to some new reflect/internal package, in particular the lookup caches which reflect maintains in order to ensure that for two reflect.Types t1 and t2 we have t1 == t2 if and only if t1 and t2 are identical. The internal package would then be used by both reflect and the new package for incomplete types.

The idea to leverage go/types is useful to understand what's the absolute minimum API needed to create recursive reflect.Types atomically. It has the downside of involving go/types, which:

1. Has a verbose type creation API, slightly more complicated than strictly needed
2. Is a large package - it also contains a lot of type-checking features, which are not used in this case
3. Depends on fmt, which depends on reflect - the implementation of reflect/convert may become tricky, as it cannot depend on reflect

so that's the price to pay for the minimal API: offload the complexity to some other existing package, which does what's needed but also has other features (unused in this case)

Any other API which avoids go/types needs at least:

1. A two-step process to create named and/or recursive types
2. A whole series of functions ArrayOf ChanOf MapOf FuncOf PtrTo SliceOf StructOf to compose (possibly incomplete) types
3. Convenience methods to examine the contents of (possibly incomplete) types: Key Elem ChanDir Field In Out Variadic etc.

thus it cannot be much simpler than your proposal or @rsc one.

Is it not possible to take a middle road, when we still have incomplete types, but only for a defined amount of time. Something like:

NamedOf(name Name, k Kind, m []Method, fn func(t Type) (underlying Type)) Type

This allows the incomplete type to only exist in the closure (as t) but it is an actual type of kind k and will panic if the underlying returned is not the same kind as k. It could be documented that functions like Elem on t would panic, and this no longer needs to be guarded. This seems simple enough to fit in the reflect package and does not cause any duplication of methods.

If you build on go/types you'd also have to build some kind of bridge between the types already built into the binary and the go/types API. Those are really meant to be separate. It doesn't make sense to cross-connect them just for reflection. Let's focus on non-go/types solutions.

Like @cosmos72 said above, it really does seem like the full API (as in #39717 (comment)) is needed, and we just want to get it as lightweight as possible.

Maybe someone would be interested to try to build the API sketched in that comment and see how well it works?

Edit: updated according to remarks by @cosmos72.

@rsc

// Package incomplete implements run-time reflection for incomplete types.
// Its purpose is the creation of recursive types.
//
// Invariants:
//
//     Complete([]Type{Of(t)}, nil)[0] == t
//     ArrayOf(n, Of(t)) == Of(reflect.ArrayOf(n, t))
//     ChanOf(d, Of(t)) == Of(reflect.ChanOf(d, t))
//     FuncOf([]Type{Of(in[0]), …, Of(in[len(in)-1])}, []Type{Of(out[0]), …, Of(out[len(out)-1]}, v) == Of(reflect.FuncOf(in, out, v))
//     MapOf(Of(k), Of(e)) == Of(reflect.MapOf(k, e))
//     PtrTo(Of(e)) == Of(reflect.PtrTo(e))
//     SliceOf(Of(e)) == Of(reflect.SliceOf(e))
//     StructOf([]StructField{
//       StructField{ Name: "Foo", Type: Of(ft) },
//       …,
//     }) == Of(reflect.StructOf([]reflect.StructField{
//       reflect.StructField{ Name: "Foo", Type: ft },
//       …,
//     })
package incomplete

import "reflect"

// Type represents an incomplete type, or part of an incomplete composite type.
// It is a safe way to define the layout of (possibly recursive) types
// with the Of, NamedOf, ArrayOf, ChanOf, FuncOf, InterfaceOf, MapOf, PtrTo,
// SliceOf, StructOf, and SetUnderlying functions before the actual types are
// created with Complete.
type Type interface {
	// interface shared with reflect.Type
	ChanDir() reflect.ChanDir
	IsVariadic() bool
	Len() int
	Name() string
	NumField() int
	NumIn() int
	NumMethod() int
	NumOut() int
	PkgPath() string

	// reflect.Type analogous methods
	Elem() Type
	Field(int) StructField
	FieldByName(string) (StructField, bool)
	Key() Type
	In(int) Type
	Method(int) Method
	MethodByName(string) (Method, bool)
	Out(int) Type

	// incomplete.Type-specific methods

	// Kind returns the specific kind of this type. It returns reflect.Invalid
	// if this type was returned by NamedOf and it has not been defined yet.
	Kind() reflect.Kind

	// Define defines a named type as the given type. It panics if the type's
	// kind is not reflect.Invalid, if the defining type's kind is
	// reflect.Invalid, or if the result would contain an invalid recursion.
	Define(Type)

	// AddMethod adds the given method to this type. It panics if there is a
	// method name clash, or if methods with distinct, non-empty PkgPath strings
	// are added. Furthermore, one of the following cases must apply:
	//
	// Case 1: this type was created with InterfaceOf.
	//
	// Case 2: this type was created with NamedOf and defined to a non-pointer,
	// non-interface type.
	//
	// Case 3: this type was created with PtrTo, with an element type which
	// Case 2 applies to.
	AddMethod(Method)

	// Underlying returns a type's underlying type.
	//
	// The underlying type of an interface type, an unnamed type, or a type with
	// a predeclared name is the type itself. For a user-defined non-interface
	// named type, the underlying type is the first unnamed or predeclared type
	// in its definition chain. For example, if the type represents A in
	//
	//     type A B
	//     type B byte
	//
	// then its underlying type is the type which represents byte (i. e., uint8).
	//
	// If a named type has not been defined yet, Underlying returns nil.
	Underlying() Type
}

// StructField is analogous to reflect.StructField, minus the Offset field.
type StructField struct {
	Name, PkgPath string
	Type          Type
	Tag           reflect.StructTag
	Index         []int
	Anonymous     bool
}

// Method represents an incomplete method.
// Unlike in reflect.Method, the implementing Func is not part of this
// structure.
type Method struct {
	Name, PkgPath string
	Type          Type // receiver = first arg, except for interface methods
}

// Of returns a Type representing the given complete reflect.Type.
func Of(reflect.Type) Type

// NamedOf creates the incomplete type with the specified package path and name.
// The name can be bound to an underlying type with the Define method.
func NamedOf(pkgPath, name string) Type

// ArrayOf creates an incomplete array type with the given count and
// element type described by elem.
func ArrayOf(count int, elem Type) Type

// ChanOf is analogous to reflect.ChanOf.
func ChanOf(dir reflect.ChanDir, elem Type) Type

// FuncOf is analogous to reflect.FuncOf.
func FuncOf(in, out []Type, variadic bool) Type

// InterfaceOf returns an incomplete interface type with the given list of
// named interface types. InterfaceOf panics if one of the given embedded types
// is unnamed or its kind is not reflect.Interface. It also panics if types
// with distinct, non-empty package paths are embedded.
//
// Explicit methods can be added with AddMethod.
func InterfaceOf(embedded []Type) Type

// MapOf creates an incomplete map type with the given key and element types.
func MapOf(key, elem Type) Type

// PtrTo is analogous to reflect.PtrTo.
func PtrTo(t Type) Type

// SliceOf is analogous to reflect.SliceOf.
func SliceOf(t Type) Type

// StructOf is analogous to reflect.StructOf.
func StructOf(fields []StructField) Type

// Complete completes the incomplete types in in, transforming them to a list
// of reflect.Type types. The function method is called once for each method
// added with AddMethod and should return an implementation of that method:
// a function whose first argument is the receiver.
// The list out contains the fully usable resulting types, except that methods
// can be called on them only after Complete has returned. The index indicates
// which type will be the method receiver, and stub indicates the method.
func Complete(
	in []Type,
	method func(out []reflect.Type, index int, stub Method) interface{},
) []reflect.Type

I've used @nrwiersma's idea to add method implementations in the Complete step, when the resulting types are completely constructed and usable, except for calling methods on their values.

This API is slightly more restrictive compared to what you can do at compile-time, but only as regards order of declarations. For example, given the run-time equivalent of the fragment

type A B
func (A) Print() { println("Foo") }
type B int

the API would not allow the declaration of Print before the declaration of B (because if B were bound to, say, *int instead, the method Print should not exist in the first place).

Thanks for the effort :)
The 'invariants' part is useful, and the rest of the API looks quite complete.

Some quick comments:

	// Define defines a named type as the given type. It panics if the type's
	// kind is not reflect.Invalid, if the defining type's kind is
	// reflect.Invalid, or if the result would contain an invalid recursion.
	Define(Type)

what about SetUnderlying(Type)? It seems more intuitive to me

	Underlying() Type

Underlying may or may not be necessary, depending on the exact semantic of Define or SetUnderlying above.

// StructField is analogous to reflect.StructField, minus the Offset and Index
// fields.
type StructField struct {
	Name, PkgPath string
	Type          Type
	Tag           reflect.StructTag
	Anonymous     bool
}

Please keep the Index field, it costs basically nothing and makes programmers' life easier.

// MapOf creates an incomplete map type with the given key and element types.
// It panics if key's kind is reflect.Invalid.
func MapOf(key, elem Type) Type

MapOf also needs to accept key's kind = reflect.Invalid.
Otherwise you cannot create this type:

type Foo struct {
  Map *map[Foo]string
}

The check on whether key Type is acceptable as map key, and in general all non-recursion checks, must be delayed until Define or SetUnderlying gets invoked.

// StructOf is analogous to reflect.StructOf.
// It panics if one of the fields' type's kind is invalid.
func StructOf(fields []StructField) Type

Also, StructOf needs to accept fields whose type is incomplete, i.e. their kind is reflect.Invalid.
Otherwise you cannot create these mutually recursive types:

type Foo struct {
  Bar Bar
}
type Bar *struct { // notice the '*'
  Foo Foo
}

Thanks for checking my post!

@cosmos72 wrote:

	// Define defines a named type as the given type. It panics if the type's
	// kind is not reflect.Invalid, if the defining type's kind is
	// reflect.Invalid, or if the result would contain an invalid recursion.
	Define(Type)

what about SetUnderlying(Type)? It seems more intuitive to me

I've decided against SetUnderlying and Bind because you can define a type A to a type B as in

type A B
type B int

but the underlying type of A is then int, and A is bound to that underlying type.

	Underlying() Type

Underlying may or may not be necessary, depending on the exact semantic of Define or SetUnderlying above.

Underlying may not return the same thing that was set with Define, so it may be useful.

// StructField is analogous to reflect.StructField, minus the Offset and Index
// fields.
type StructField struct {
	Name, PkgPath string
	Type          Type
	Tag           reflect.StructTag
	Anonymous     bool
}

Please keep the Index field, it costs basically nothing and makes programmers' life easier.

OK.

// MapOf creates an incomplete map type with the given key and element types.
// It panics if key's kind is reflect.Invalid.
func MapOf(key, elem Type) Type

MapOf also needs to accept key's kind = reflect.Invalid.
Otherwise you cannot create this type:

type Foo struct {
  Map *map[Foo]string
}

The check on whether key Type is acceptable as map key, and in general all non-recursion checks, must be delayed until Define or SetUnderlying gets invoked.

// StructOf is analogous to reflect.StructOf.
// It panics if one of the fields' type's kind is invalid.
func StructOf(fields []StructField) Type

Also, StructOf needs to accept fields whose type is incomplete, i.e. their kind is reflect.Invalid.
Otherwise you cannot create these mutually recursive types:

type Foo struct {
  Bar Bar
}
type Bar *struct { // notice the '*'
  Foo Foo
}

Yup, you're correct. I also removed the restriction that you cannot add methods to types which have already been embedded because, e. g., this here is legal:

type T struct {
  A
}
type A []T
func (A) Foo() {
  println("foo")
}
…
T{}.Foo()

So yes, we have to delay these checks.

Can the restriction not to declare methods on an t.Kind() == reflect.Invalid type remain? I think so, at the cost of the implementation of Type.Method* for interfaces and structs with embedded fields becoming somewhat more complex.

I don't see an obvious reason to replicate reflect.Type in incomplete.Type. The program that constructs the incomplete type presumably does not need to interrogate it.

@ianlancetaylor wrote:

I don't see an obvious reason to replicate reflect.Type in incomplete.Type. The program that constructs the incomplete type presumably does not need to interrogate it.

I guess you're talking about most methods of incomplete.Type in @TheCount #39717 (comment)
and my remark

3. Convenience methods to examine the contents of (possibly incomplete) types: Key Elem ChanDir Field In Out Variadic etc.

I intentionally named them "convenience" methods: it's certainly possible to make incomplete.Types opaque (i.e. no way to inspect them) by removing most of their methods.

But using them it will not be pleasant: complicated code will need lots of them, and may even choose to keep them around, to avoid re-creating them every time. I think that making them opaque would often force to maintain the same information externally - duplicating it and risking discrepancies.

I've decided against SetUnderlying and Bind because you can define a type A to a type B as in

type A B
type B int

but the underlying type of A is then int, and A is bound to that underlying type.

There are two alternatives here:

1. the Type u passed to Type.Define(u Type) must be its own underlying type, i.e. u.Underlying() == u (or however you compare two incomplete.Types).
   The code type A B; type B int is then equivalent to:

import "reflect/incomplete"

var A = incomplete.NamedOf("A", "mypackage")
var B = incomplete.NamedOf("B", "mypackage")
B.Define(/*type of int*/) 
A.Define(B.Underlying()) // non-trivial dependency: A.Define() must be *after* B.Define()

This also requires Type.Underlying.

2. the Type u passed to Type.Define(u Type) will not be used exactly - its underlying type will be used instead.
   The code type A B; type B int is then equivalent to:

import "reflect/incomplete"

var A = incomplete.NamedOf("A", "mypackage")
var B = incomplete.NamedOf("B", "mypackage")
A.Define(B)               // any order will do: A.Define() can be before or after B.Define()
B.Define(/*type of int*/) 

This does not strictly require Type.Underlying.

@ianlancetaylor wrote:

I don't see an obvious reason to replicate reflect.Type in incomplete.Type. The program that constructs the incomplete type presumably does not need to interrogate it.

I don't think it's strictly necessary. A program which has from the beginning a clear plan from in its mind probably wouldn't interrogate the incomplete types. For something like a Go interpreter, where types may be constructed on the go, it might be convenient. In any case, it looks like a separable problem, so we could start with a minimal API and add interrogation methods later on.

package incomplete

import "reflect"

// Type represents an incomplete type, or part of an incomplete composite type.
// It is a safe way to define the layout of (possibly recursive) types
// with the Of, NamedOf, ArrayOf, ChanOf, FuncOf, InterfaceOf, MapOf, PtrTo,
// SliceOf, and StructOf functions before the actual types are created with
// Complete.
type Type interface {
	// Define defines a named type as the given type. It panics if the type
	// is not a named type, the type has already been defined, or if the result
	// would contain an invalid recursion.
	Define(Type)

	// AddMethod adds the given method to this type. The Index field of the given
	// method is ignored. It panics if there is a method name clash, or if
	// methods with distinct, non-empty PkgPath strings are added. Furthermore,
	// one of the following cases must apply:
	//
	// Case 1: this type was created with InterfaceOf.
	//
	// Case 2: this type was created with NamedOf and defined to a non-pointer,
	// non-interface type.
	//
	// Case 3: this type was created with PtrTo, with an element type which
	// Case 2 applies to.
	AddMethod(Method)
}

// StructField is analogous to reflect.StructField, minus the Index and Offset
// fields.
type StructField struct {
	Name, PkgPath string
	Type          Type
	Tag           reflect.StructTag
	Anonymous     bool
}

// Method represents an incomplete method.
// Unlike in reflect.Method, the implementing Func is not part of this
// structure.
type Method struct {
	Name, PkgPath string
	Type          Type // receiver = first arg, except for interface methods
	Index         int
}

// Of returns a Type representing the given complete reflect.Type.
func Of(reflect.Type) Type

// NamedOf creates the incomplete type with the specified name and package path.
// The name can be bound to an underlying type with the Define method.
func NamedOf(name, pkgPath string) Type

// ArrayOf creates an incomplete array type with the given count and
// element type described by elem.
func ArrayOf(count int, elem Type) Type

// ChanOf is analogous to reflect.ChanOf.
func ChanOf(dir reflect.ChanDir, elem Type) Type

// FuncOf is analogous to reflect.FuncOf.
func FuncOf(in, out []Type, variadic bool) Type

// InterfaceOf returns an incomplete interface type with the given list of
// named interface types. InterfaceOf panics if one of the given embedded types
// is unnamed or its kind is not reflect.Interface. It also panics if types
// with distinct, non-empty package paths are embedded.
//
// Explicit methods can be added with AddMethod.
func InterfaceOf(embedded []Type) Type

// MapOf creates an incomplete map type with the given key and element types.
func MapOf(key, elem Type) Type

// PtrTo is analogous to reflect.PtrTo.
func PtrTo(t Type) Type

// SliceOf is analogous to reflect.SliceOf.
func SliceOf(t Type) Type

// StructOf is analogous to reflect.StructOf.
func StructOf(fields []StructField) Type

// Complete completes the incomplete types in in, transforming them to a list
// of reflect.Type types. The function method is called once for each method
// added with AddMethod and should return an implementation of that method:
// a function whose first argument is the receiver.
// The list out contains the fully usable resulting types, except that methods
// can be called on them only after Complete has returned. The index indicates
// which type will be the method receiver, and stub indicates the method.
func Complete(
	in []Type,
	method func(out []reflect.Type, index int, stub Method) interface{},
) []reflect.Type

I've also swapped the order of arguments to NamedOf, so name and pkgPath match the order in StructField and Method (rather than the order in go/types).

@cosmos72: regarding type definitions, I would lean towards your 2., just so the behaviour is not too surprising for users of the API.

But using them it will not be pleasant: complicated code will need lots of them,

Why would code need them?
Those methods are for understanding an unknown reflect.Type that is passed to general-purpose code.
It's hard to see why you'd need them halfway through constructing a new type.
Wouldn't the code building the type know about the type?

It seems like it would be enough to just do a sentinel interface like:

type Type interface {
    isType() 
}

Unless you have a specific example of code where those extra methods would be needed?

Otherwise the sketch looks about right. Would you like to try implementing it and see if it satisfies your use case?

@TheCount and/or @cosmos72, any interest in implementing the package sketched above and see if it serves your use case?

My use case is a Go interpreter.

For this particular case, it can be modeled as a library that takes a slice of arbitrary go/types.TypeSpec as input, and uses reflect and reflect/incomplete to create a corresponding slice of reflect.Type.

I tried a mental experiment, and the proposed reflect/incomplete API seems powerful enough for such task, which can be summarized as "a library that creates reflect.Types from arbitrary source code".

About implementing reflect/incomplete: it's definitely non-trivial, in part because it needs access to reflect internal types, starting from *rtype

Hello, as developer of yaegi, also a go interpreter, I'm very interested in having 2 missing features in reflect:

1. the ability to create recursive types (this is today poorly emulated in the interpreter),
2. the ability to add methods to a created type, so it implements one or several interfaces (this is impossible to get it right without improving reflect).

To me, a problem is that the pre-compiled code must be able to check that an object instantiated from a such created type implements an existing interface. It seems not trivial to achieve.

Also the need to create a new interface type (not existing in the pre-compiled code) is less obvious to me, as only the interpreted code uses it, it can be handled internally in the interpreter (and is already, at least in gomacro and yaegi).

I understand that the proposed reflect/incomplete package is useful primarily to isolate the effort involved in the creation of a recursive type. It's not clear to me yet how it helps for the point 2. May be through the NamedOf ?

I agree also that this package is for reflect type creation only, not interrogation, as an interpreter has already its own type representation and spec.

Once successfully implemented, I don't get yet the interest of keeping this package exposed. It seems to rather belong to internal, and having the interesting functions accessible through reflect only.

But yes definitely interested to participate and help if I can, and put efforts on it. and for info, @nrwiersma an I work in the same team on yaegi.

Question:
is it acceptable to move to a new package reflect/internal the types and functions from reflect that are also needed by this proposal?
Doing so would simplify the implementation and reduce duplicated code/types.

The types to move would be at least: rtype and its wrapper types: arrayType chanType funcType interfaceType mapType ptrType sliceType structType, plus structField and possibly others.

@cosmos72 Perhaps you can use the already existing package internal/reflectlite. But yes, it would be OK to create a new internal package if required.

@mvertes

Once successfully implemented, I don't get yet the interest of keeping this package exposed. It seems to rather belong to internal, and having the interesting functions accessible through reflect only.

The reason to have the package exposed is that it factors the API out of reflect, where we can use the package name to help shorten the other names. You're right that it would be quite tied up with reflect in the implementation, but it helps keep the concept of incomplete types out of the exported reflect API.

As far as methods, the proposed reflect.NamedOf in #16522 has the benefit that it creates a newly named type with methods in one step - by creating and attaching methods in the same operation, it avoids the potential confusion of having a type that does not have any methods at one moment and then does have methods the next. But of course that wouldn't work with recursive types: creating the named type is necessary to finish the recursion. So attaching methods may need to be in this new reflect/incomplete package as well.

And of course reflect/incomplete could be used to create non-recursive types as well as recursive types, so it might be that adding methods should be done here. A type that doesn't have methods yet (but will) is incomplete in a different way. I would certainly support exploring that in the API as well.

It sounds like in general we think this new package is the right approach, but the devil is in the details.
Putting on hold until there is an implementation we can look at.

I am continuing my implementation at https://github.com/cosmos72/go/tree/issue-39717/src/reflect/incomplete
Legal: it is a fork of https://github.com/golang/go and has the same authors&license, but if needed I can also submit a CL

Status: it's not complete yet, but the main skeleton and most details are there.
The idea is:

• *incomplete.itype is the only implementation of incomplete.Type and it contains all available information about an incomplete type. It should be canonical i.e. only one instance per type.
• it has a field itype.incomplete *rtype which is filled incrementally when the user calls the various API functions/methods.
• when such *rtype is complete, it is copied to the field itype.complete reflect.Type which can be returned by Complete()

Comments/suggestions are welcome.