New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
proposal: spec: variadic type parameters #66651
Comments
Aside from the addition of constraints on the count of type parameters, this seems to be a more precise statement of #56462, for some additional background. (cc @DeedleFake) |
Presumably |
@zephyrtronium Thanks, I entirely forgot about #56462.
|
If there were tuple types that looks like it would satisfy the MetricN case and simplify variadic fields/values since you could just wrap any type list in a tuple and not have to define additional machinery. It seems unfortunate to be limited to a single variadic type as that means you couldn't write, for example, a decorator for any function |
Would a |
Another question: func F[T... any](T) {
F(T, T) // legal?
} |
@Merovius Presumably the rules would need to be tightened to not allow calls that generate infinite shapes. The following could also generate an infinite series of func shapes.
Though the word "compatible" might be doing some heavy lifting to disallow these example. |
@jimmyfrasche A major goal here is to avoid requiring both @Merovius I think that is already prohibited by the rule saying that a generic function may only refer to itself using identical type parameters. |
@ianlancetaylor I meant that if you also had tuples you could offload the handling of values of type-vectors onto them. However it occurs to me you'd still need some mechanism for writing out the parameters to a closure. |
This comment has been minimized.
This comment has been minimized.
If it is used as a struct field, what type should it be, tuple? Or some other unknown type? |
in addition, why isn't this the case here? @ianlancetaylor type Seq[E... 1 2 any] = func(yield func(E) bool) |
This comment was marked as resolved.
This comment was marked as resolved.
@ianlancetaylor I can't find that rule in the spec? I can find one for generic types, but nothing equivalent for functions. |
What does |
If a struct field has a variadic type, then it is actually a list of struct fields. The types of the listed fields are the list of type arguments. |
This comment was marked as resolved.
This comment was marked as resolved.
@Merovius Apologies, I'm probably misremembering it. But it seems to me that we need that rule. Otherwise we can write func F[T any]() string {
var v T
s := fmt.Sprintf("%T", v)
if len(s) >= 1000 {
return s
}
return F[struct{f T}]()
} And indeed that currently fails to compile:
|
The same thing as |
Is this list mutable? Does it support iterate and subscript access? There is a scenario where variable generics are used as function parameters and are expected to be accessible through subscript. |
I'm guess that these:
the first one is never allowed and the second one is only allowed if called with a single type? It seems weird to me that this introduces variables that aren't real variables and struct fields that aren't real struct fields. |
@leaxoy The proposal intentionally does not propose allowing that. There are undeniably useful abilities, but they are out of scope here.
|
@ianlancetaylor I'm a bit worried about the ability to specify bounds on the number of type-parameters, as it seems to me to have the potential to introduce corner cases allowing computation. For that reason, it would be useful to have a relatively precise phrasing what those rules are. |
|
I'm not sure what that syntax should mean, so I'd say no. Note that
I don't think so (assuming you meant
No. |
func F[T... any]() (x struct{ F T }) {
return x
}
func main() {
var y struct { F0 int }
fmt.Println(y == F[int]())
} Note that this has no defined types. So I tend to agree with @jimmyfrasche that this should reasonably print |
At least for iterators and decorators, it's interesting that the use of variadic type parameters is at function input or output parameters. This seems sympathetic with type unification - if a variadic variable In type parameter lists: (Limiting variadic type parameters to function inputs and outputs is not helpful w/r/t generic tuples. I think the metrics examples are still possible but more complicated.) edit: Defining tuples in terms of functions is maybe a bit plausible. There would be no access to component terms of the tuple in-flight, so it's less similar to anonymous structs or tuples with accessible fields, and more like a this proposal's existing notion of a variadic variables. Except that, the function offers a self-evident way to take the tuple out of flight.
|
Not according to the wording of the proposal which says: "Direct references to these fields in Go code are not permitted" with no qualification as to whether that Go code is inside or outside the variadic code. |
Sorry if this has already been mentioned. When I write generic code at times I'd like to write wrappers around functions that may return a variable number of arguments and
And that could be called as:
or:
or:
And would type inference be able to infer the |
As far as I understand, that should work. See #66651 (comment) for a related case, too. The inference is the real trick. My guess is that your case is easier to infer than the one that I linked. |
In my comment above (#66651 (comment)), I clearly omitted one crucial issue: the fact that in the original proposal, using a variadic-typed value produces a list of values, not a single value. There is no way to do something similar when variadic values are treated as a tuple instead. However, it seems to me that there is a natural way to support that, while also addressing some people's concern (e.g. here and here) that the list expansion is too implicit. We start by defining a tuple type as a first class object. A tuple is defined by a parenthesized list of types, for example:
A tuple literal consists of a parenthesized list of expressions:
A tuple type can be "spread" into a list of values by using the
Note that there is no ambiguity with respect to the slice A tuple value is comparable if all its component types are comparable.
A variadic type list can be declared as specified in the proposal:
However, in contrast with the original proposal, if the variadic type is used directly, it is exactly the tuple type containing its component types. It can be turned back into a list by using the As with the original proposal, such a list can be used anywhere a list of types is allowed:
When a variadic type list is used as an argument list for a function or method, the argument value
When For example:
Here are the examples from the original proposal rewritten to use this style.
From a readability point of view, ISTM that the main subtlety here is the distinction between Note that there is no direct way to turn a tuple back into its component type parameters, although
Of course, as @ianlancetaylor points out, this is a larger change than the original proposal, as it would require the runtime changes to support tuples, but I think that it would still be worthwhile nonetheless. |
Sorry, but tuples for Go have already been rejected repeatedly , last in #64457 mainly because they are like struct with anonymous members. So all this discussion about them with regards to this proposal is off topic. I think this feature needs to be implemented without any need for tuples. |
@bjorndm I totally agree. I wish we could focus this issue on the original proposal and its close derivatives. I don't mind people opening a new issue with their related tuple proposals, referring to this as needed, but right now I feel totally overwhelmed by the tuple noise. |
@bjorndm FWIW #64457 was rejected because "there don't seem to be convincingly strong use cases (where a struct wouldn't be a better choice) that justify the extra language complexity.". I'm daring to suggest that this particular use case is strong enough that it can justify that extra complexity, as in my view it makes variadic type parameters fit much more naturally into the language. |
@rogpeppe Hmm, I would rather say that if we need all this complexity that tuples bring to be able to implement this feature, maybe we should not do this proposal either. All this complexity seems to much cost for the benefit of a relatively small feature of generics. |
I think tuples aren't buying you anything in your proposal, you could use structs and just have the
etcætera. Your syntax in particular is also ambiguous:
is this returning a tuple of int and error or an int and an error? |
I think that the discussion about tuples is simply a discussion about ways that one would try to implement multiple variadic type parameters. Basically, that would either keep tuple types as a purely generic programming construct (as variadic type parameters, implicitly (un)packed where needed) or inscribe them in the spec. Tuples as a reified concept is some sort of sugar around anonymous structs. If it is necessary, then the other question is whether the proposed implementation of variadic type parameters is not so limiting that people start to want tuples still. I guess, if people are to try and generate code, the 255 limit might be too limiting, although that's very unlikely to be the norm. That limit could potentially be removed one day anyway. Or having multiple variadic type parameters, they could just be juxtaposed to increase the limit. |
Yes, that is definitely a possibility. Tuples are, after all, "just" structs with unnamed members. Still, on balance, I think they'd be worthwhile:
I should point out that the semantics I am suggesting for tuples are (almost?) identical to those proposed in #64457.
|
A possible alternate approach to variadic templates using tuples: Start with tuples as in #64457. Adjust range-over-func to unpack tuples:
Add an additional assignability rule: A function value taking and/or returning some set of parameters is assignable to a variable of a func type that takes and/or returns a single tuple containing the same parameters.
(This implies that the calling convention for a func taking and/or returning a single tuple is the same as that for a func taking and/or returning the flattened contents of the tuple. I think that should be straightforward, but I'm not a compiler person.) Given this, we write
This is almost exactly the current definition of In the two-element case, this is parameterized as
|
This is slightly off-topic and I'd rather not continue this discussion here but I think I remember one issue with tuples as types being: how to unpack unexported fields. Within the package the tuple type would be defined in, it should be fine. Should the unexported field be unpacked when outside of the package the tuple was defined in? Also, it seems to be a bit complex, if manageable, to have tuple typed values and multiple values. I am a bit concerned by what would happen when someone has to think about how to unpack those. Variadic type parameters which really are some kind of tuples of type parameters seem to strike a good balance and would integrate in a smoother way with the existing language, I personally find. |
Actually as defined in this proposal variable type arguments are not tuples but type lists, which is different, we should not conflate them. Tuples are one way in which this type list could be made usable but there are other, simpler alternatives. |
Tuples have no field names, so no "unexported fields". This is an issue with the "struct-unpacking operator" proposal, not tuples. |
This is to be taken slightly more in the mathematical sense since type parameters are not types themselves anyway.
Indeed, that's true. Forgot that if tuple types are implemented as very specific kinds of structs, this problem goes away. So this point notwithstanding. |
Edit: this possibility was already suggested by @ianlancetaylor here. I've been wondering about the constraints on the number of type arguments. More substantively, I wonder whether it's necessary to have
It is indeed the case that range can return at most two values, but I'm wondering if it might actually be OK to change the language to
That would allow both a zero-element Given the ability, with variadic type parameters, to write code that Then there would be no need for the numeric constraints at all. AFAICS
AFAICS that "zero" should actually be "one. |
One another aspect of the proposal, I tend to agree with some other replies Being able to represent a full function signature is a very useful It becomes easy to write many useful wrapper functions that are
Although this can't directly represent functions with variadic
Another, more abstract, reason for allowing multiple variadic type So I suggest that multiple variadic type parameters should be allowed, For example:
To my mind allowing multiple variadic type parameters |
To summarise my thoughts from my various posts above, I propose the following:
FWIW things started falling into place for me when I realised that a To my mind this direct analogy makes variadic type parameters |
Just to see how it looked on the page, I tried to write everything in the iteration adapters proposal (not just Zip The tuple-fied implementation of the The original implementation of Merge In contrast, there's an interesting detail with
The original 1-ary implementation uses a comparison function
I was sort of puzzled by how to write the same relationship with tuples. How can we express that |
That's a great exercise! Here's my stab at it: https://go.dev/play/p/CpQpxovY9VP The fact that the results of Zip are always tuplified is arguably not a great UX. There are some other questions over what API works best too, but in general
Yes, this isn't something the original proposal could do. This is a concrete example of the "it isn't possible to make a generic type or function that wraps more than one variadic type without that capability" reasoning from this comment.
Something like this perhaps?
Although it hasn't been explicitly stated yet, I think the rules for arity of Thinking further, a similar rule must apply to all values: although the generic code can For example:
|
In a tuple world, I'd say that
This is a bit different from the API in #61898: You can't |
I think it's worth keeping in mind that this is blocking the iterators proposal. Rightly so, in my opinion, because the If tuples would be the way to go, I think it should first be considered to be allowed inside variadic type parameter functions only, to keep changes at a minimum. Also, general support for tuples would be a completely different propsal, IMO. With that in mind, the original proposal seems to be almost exactly that, although not named as tuples. It does constrain it to only be used expanded, not as a single value. The original proposal doesn't use ellipsis, but it could be added to allow for more generic tuples later. There seems to be a general agreement that a single variadic type list is too limiting, I agree as well. Parentheses have been suggested as a workaround, which seems reasonable.
I suppose if I want a reference to this function I would then write
Perhaps this restriction could be lifted, or the restriction for unique field names?
This syntax wouldn't be allowed, which is why the original takes K only. This also shows another reason why general tuples need to be discussed separately.
I prefer this syntax for size constraints. |
I am worried that this discussion is losing clear direction, and therefore failing to progress, due to the many different proposals/variations being thrown into the mix via comments. Perhaps we should create a separate issue for such discussion? Or pull some of the proposed changes into separate proposals so they can go through a more clear and distinct vetting process? |
One thing I would worry about is that without explicit sigils, free-form arguments might block some preferred syntax for a future enhancement to the language that has yet to be proposed or even conceived. But I have no examples of what that might be, so it is merely a worry I have. |
Proposal Details
Background
There are various algorithms that are not easy to write in Go with generics because they are naturally expressed using an unknown number of type parameters. For example, the metrics package suggested in the generics proposal document is forced to define types,
Metric1
,Metric2
, and so forth, based on the number of different fields required for the metric. For a different example, the iterator adapter proposal (https://go.dev/issue/61898) proposes two-element variants of most functions, such asFilter2
,Concat2
,Equal2
, and so forth.Languages like C++ use variadic templates to avoid this requirement. C++ has powerful facilities to, in effect, loop over the variadic type arguments. We do not propose introducing such facilities into Go, as that leads to template metaprogramming, which we do not want to support. In this proposal, Go variadic type parameters can only be used in limited ways.
Proposal
A generic type or function declaration is permitted to use a
...
following the last type parameter of a type parameter list (as inT...
for a type parameterT
) to indicate that an instantiation may use zero or more trailing type arguments. We useT... constraint
rather thanT ...constraint
(that is, gofmt will put the space after the...
, not before) becauseT
is a list of types. It's not quite like a variadic function, in which the final argument is effectively a slice. HereT
is a list, not a slice.We permit an optional pair of integers after the
...
to indicate the minimum and maximum number of type arguments permitted. If the maximum is0
, there is no upper limit. Omitting the numbers is the same as listing0 0
.(We will permit implementations to restrict the maximum number of type arguments permitted. Implementations must support at least 255 type arguments. This is a limit on the number of types that are passed as type arguments, so 255 is very generous for readable code.)
With this notation
V
becomes a variadic type parameter.A variadic type parameter is a list of types. In general a variadic type parameter may be used wherever a list of types is permitted:
func SliceOf[T... any](v T) []any
func PrintZeroes[T... any]() { var z T; fmt.Println(z) }
type Keys[T... any] struct { keys T }
A variadic variable or field may be used wherever a list of values is permitted.
[...]T
syntax (hereT
is an ordinary type or type parameter, not a variadic type parameter).Note that a variadic type parameter with a minimum of
0
may be used with no type arguments at all, in which case a variadic variable or field of that type parameter will wind up being an empty list with no values.Note that in an instantiation of any generic function that uses a variadic type parameter, the number of type arguments is known, as are the exact type arguments themselves.
Variadic type parameters can be used with iterators.
In a struct type that uses a variadic field, as in
struct { f T }
whereT
is a variadic type parameter, the field must have a name. Embedding a variadic type parameter is not permitted. Thereflect.Type
information for an instantiated struct will use integer suffixes for the field names, producingf0
,f1
, and so forth. Direct references to these fields in Go code are not permitted, but the reflect package needs to have a field name. A type that uses a potentially conflicting field, such asstruct { f0 int; f T }
or evenstruct { f1000 int; f T }
, is invalid.Constraining the number of type arguments
The
Filter
example shows why we permit specifying the maximum number of type arguments. If we didn't do that, we wouldn't know whether the range clause was permitted, as range can return at most two values. We don't want to permit adding a range clause to a generic function to break existing calls, so the range clause can only be used if the maximum number of type arguments permits.The minimum number is set mainly because we have to permit setting the minimum number.
Another approach would be to permit range over a function that takes a yield function with an arbitrary number of arguments, and for that case alone to permit range to return more than two values. Then
Filter
would work as written without need to specify a maximum number of type arguments.Work required
If this proposal is accepted, at least the following things would have to be done.
Ellipsis
in the constraint of the last type parameter.The text was updated successfully, but these errors were encountered: