New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
proposal: Go 2: generics: use operators for meta typing #41449
Comments
One place where people want to use generics is in containers. For a simple example, people want to be able to write a linked list package that can store values of any type without unnecessary overhead. With the current generics design draft, that might look like https://go.googlesource.com/go/+/refs/heads/dev.go2go/src/cmd/go2go/testdata/go2path/src/list/list.go2 . How could that be done with this approach? |
Instead of
For most generic types the constraint will be either |
In my opinion the syntax is a bit cryptic.
This is not quite true - the |
Is there a way to write something like the |
Dunno why some downvoted without giving their reasons. I upvoted for 2 reasons: the proposer gave his honest (and I believe valid) take even if eventually his proposal would be rejected; the proposed syntax is clean albeit with some drawbacks - perhaps we could use a combination of operators for default and interfaces for others? |
You are right about the . "operator"; I will remove the example. Regarding the |
Note that my proposal is to explicitly not support this as Go provides interfaces already for this:
Note that this version is more general than the generic Stringify that you linked to: here the items in list can be of different underlying types while in the generic version they necessarily are of the same type. So in this case, allowing generics to specify methods would lead to a less general implementation (although possibly more performant) than what you can already do with interfaces. Other arguments are in the original post. Of course, a generic solution with a helper function is still also possible:
|
A slice of type I agree that a helper function would work, but it's awkward. That seems to me that it might be a significant drawback to this proposal. Perhaps I am mistaken. |
I may also be mistaken of course, I just hope to add to the discussion and understanding of what generics in Go should do. What I meant is that the abstract functionality provided by interfaces is inherently more powerful than that of a generic list []T:
Interfaces are more powerful than generics in most cases because they can operate on heterogeneous types. Therefore I think the interface version There are exceptions where using []T is better, however a solution with a helper function is not awkward but quite natural (and quite common):
This more succinct and more direct than:
Which requires defining an extra interface type and also requires the addition of Less as a method on T. Other examples are Map/Reduce. Of course the first example can be written in the current proposal; the point is that you don't need an interface meta type here. |
Just to be clear, it seems to me that the current generics design draft permits writing any kind of code that is permitted by this proposal, albeit with a different syntax. The current design draft also permits writing other kinds of code, that cannot be written with this proposal. Would you agree with that? |
How well would this support something like a
There doesn't seem to be a good way to explain the relationship between TNode and TEdge in the Graph interface from what I can tell. As for some feedback, I think this proposal offers a solution which really is trying to accomplish adding "operator interfaces" (allowing interfaces to describe operations on types), but doing it by tightly coupling it with the idea of type parameterization, when they are actually two entirely separate concepts. Because this proposal couples these ideas together, it would make it very hard to add more features to either of these concepts individually. The current generics draft introduces these two concepts as two separate ideas. "Operator interfaces" are instead introduced as type lists, and type parameterization is introduced as such. For instance, consider the function |
Regarding the graph structure, that's not supported (by intention), because you can use functions or interface already. I think a simpler approaches that provide the same functionality is better because it is so much shorter, something like:
Regarding your feedback. I am not trying to accomplish "operator interfaces" I think (I am not entirely sure want you mean with it). I started out by looking at what the minimal possible thing is that is needed to support T in any type: thats assignability ('any' currently) and comparison (on equality, for map keys)). With those you can express any generic type (except functions and methods). These are expressed in Go using = and == which are also short and so are the most direct way of expressing want you need. The next thing is Min, Max functions, etc. These can also easily be supported by allowing any operator instead of just = and ==, and also a list of operators. This is all still orthogonal to current expressibility in Go. If you want to be able to call methods on T, there's interfaces and functions already. Yes, this excludes examples like the graph example, but I couldn't find any convincing example for which it would be needed and for which there's no alternative with helper functions or generics. Maybe that's indeed incorrect, I'd love to see a convincing country example (that does not resemble C++ Boost or STL implementation). Using operators also does not need to make it very hard to add more features. For example you could add syntax to the language that give a name to the meta type constraint:
I am not saying that we should; it is not part of my proposal. Just an example of how it could be extended. If we want you can extend that and allow interfaces as well:
Although I think this is not a good idea, for reasons explained earlier. But it's not hard to extend the syntax. In fact, I think that if you want to allow to call methods on T, you should also be able to access fields etc. |
Almost, but this proposal does not introduce 'any', 'comparable', type lists in interfaces and a multitude of additional rules on interfaces (can't call method expressed type list types only, comparable on being able to use as normal interface type, etc etc). It also does not convolute the concept of interface. Almost, because I think my proposal would allow to make functions that operate on untyped constants which could be useful:
That's not possible with the current generics design without additional syntax. I also think that allowing parametrized methods are easier to allow, but I am not sure.
Yes, and vice versa (see above). I am not sure it is fruitful to compare proposals on syntax that they are being able to generate though. On a type safety level they can express the same functionality, although indeed the current generics proposal can write a succincter Stringify. |
That is exactly what I am questioning. It seems to me that the current generics design draft permits expressing functionality that this proposal does not permit. This is not a question of syntax, it's a question of semantics. Specifically I am thinking about examples like The untyped |
Well, your basic point is true of course but it's a bit of a moot point because I stated that exactly in my proposal:
You may disagree with that of course (and to be honest I would be very interested in opinions/convincing examples from anyone, I could be very wrong). You are also absolutely right that you can't do Of course I might be very wrong and the use case for the
This example convinces me because writing getter/setters accessor for every field seems silly if used in generic code especially combined with perfomance arguments. Maybe this proposal is not a good idea at all. It could also be that's its a matter of taste and people just like to write generic code instead of using interfaces. I hope you and others will consider my arguments against overloading interfaces with applications to generics. I am a bit worried for fancy coding syndrome wrt to generics and the extra design choice you would have to make when writing a functions like Thank you for your notes on the Max wrt untyped constants. I am quite out of my depth on that area, but I agree inlining is indeed looks like to only way to realistically do it. |
I would agree with a slightly different statement: there is no reason to use generics to do something that you can already do with interfaces. But you seem to be saying that it is important that it should not be possible to use generics do things that can already be done with interfaces. I don't agree with that as a goal. There is no reason that generics should permit code that can already be written with interfaces. But I see no reason to go out of our way to ensure that generics cannot be used to write code that can already be written with interfaces. Among the important things about generics is that they are clear and straightforward, and that they are as orthogonal as possible to other language constructs. Ensuring that they cannot be used where interfaces can be used is not a priority.
This is close to a circular argument. The Go standard library was written in a language that doesn't have generics, so naturally it doesn't use generics. When we present a use case in which the Go standard library could use generics, it doesn't make a lot of sense to say that it doesn't fit the Go standard library because the Go standard library doesn't use generics. The whole point is to change the Go standard library to use generics. Now, of course, it may be a bad idea for other reasons. But this reason isn't one of them.
You can already call free functions on variables of type T. Perhaps I misunderstand your point. Methods are a general concept in Go that apply to all types, unlike struct fields. I'm not opposed to a generics proposal that permits accessing fields with known names and types. I agree that the current design draft doesn't support that. I agree that your |
Using your proposal, how would you express a generic function that increments a generic numeric integer argument by a given constant value, say 1234, and then returns that incremented value? Specifically, how would you express that you can't provide a byte argument because 1234 overflows byte? This is one of the problems one will need to address when constraining generic functions with operators, at least for Go. Along similar lines, if one writes a generic function that uses |
Maybe I am misinterpreting your question though: the code above expresses your requirement of "not being able to pass 1234 as a byte const", however it did not need to use type constraints for it (nor couldn't). It's already checked by Go on the call site. (Note: I am assuming here that
No, it would require an api change. If the api was Edit: Regarding the first example, I think you meant this:
There is indeed no way in my proposal to specify that T cannot be a type that cannot store 1234. I also don't see an obvious solution, expect for disallowing constant integers in generic functions that are outside the range of the smallest int (byte). That might not be entirely unreasonable though. |
I agree, that's the reason I wrote this proposal. Would you agree that the current generic proposal is not as orthogonal as possible to other language constructs, especially with respect to interfaces? |
I am still confused about how the current generics draft is not orthogonal to interfaces. The draft integrates with interfaces, but any features that are provided by interfaces are provided by interfaces, not parametric types. |
@griesemer I am so sorry that I somehow missed your message directly above mine. Thanks for pointing out. But I do have some further questions:
Why operators become an issue in type-checking when they are treated as method names eventually (type
A generic implementation can only use methods from a constraint. Why shouldn't we need an API change? Could you maybe give some more detailed examples of the problem?
"possibly needed operators" seems equivalent to "possibly needed methods," why are they be treated differently? |
The issue is not about Regarding the
and later decide that
even though the effect of the function didn't change. Sometimes such changes come about from refactoring. We don't want to need the API to change in such cases (and possibly break client code). The current generics design draft doesn't have any of these problems because no operators are specified, only types (in an interface's type list). Given
one can use any operation defined on an |
PS: Worse, what if the generic function requires |
Indeed this is an issue using operators for meta typing but still seems fine in operator methods because all applicable types are declared in the constraint.
This seems natural, if the implementer decides to use
OK. This is a fair point because it will leads to the discussion that Go has no function overloading and SFINAE support eventually. |
In a programming language orthogonality doesn't mean that there is only one way to do something. It means that there are no restrictions on how different concepts can be combined. See https://en.wikipedia.org/wiki/Orthogonality_(programming) . That said, Go is not a perfectly orthogonal language. And the fact that in the current design draft interface types with type lists can only be used as type constraints is a failure in orthogonality. But the fact that interface types in general can be used as type constraints is, I think, reasonably orthogonal. We are adding another layer of meaning to interface types, but it is a layer consistent with what interface types already mean. |
Let me add something about the larger point here. In Go, interface types are already a form of generics. I mentioned that https://blog.golang.org/why-generics, and I explained why I don't think that is sufficient. Perhaps that will help explain why I think that the facts that you are pointing out are not a significant problem with the generics design draft. Thanks. |
I fully understand the concern. Let me try to address them. To use numerical constants, you must require at least Regarding the 1024/1000000 example: what I meant was even under
I don't think this is entirely unreasonable: many functions say something like 'if value is not a pointer the function panics`, etc. (For floats there is no issue I think (not sure) because That being said, another solution is of course to allow both (or either) operators and basic types as constraints. Please note that I don't know all the details of Go, although I try to, and so there may be more concerns that I am no aware of. Thanks for the explanations. |
Thanks for the links and the explanations. My wording and use of "orthogonality" maybe didn't help. Please don't think I am just trying to bash the current proposal. I kindly ask you to help me with the following worry that I have. If you have addressed it already somewhere, feel free to just provide a link.
I agree and it was the basis for me to write my proposal. My main worry is this:
I honestly don't know which would be better or how to chose which version to write. Yet, I write these kinds of functions quite often. So it adds a design question (that I don't know the answer to) to my daily work. I didn't have this question before. Now I have it, and I don't know the answer. I will default to the interface version, but many others may uses the generic version. That would make their code harder to read for me and vice versa. The question is: can you explain me which one to chose here? (And why?). That would certainly take away my concerns. I have some other concerns, like the Graph example being too clever, that might be unjustified, but the above is my main concern. |
@markusheukelom This is a great question. We absolutely need to develop good conventions and "best practices" around the use of generics. I'll try to answer for your specific example: I believe in this case
On the other hand, there is a cost of having a generic version for each type of (Note, this specific situation may be a case where a compiler might choose to implement the generic version like the non-generic version.) In summary, if the generic version doesn't improve the code along the axes of static type safety, memory use, and performance, choose the non-generic version. Here's another way of looking at this: Generics are essentially a "glorified" version of macros. Would you use a macro in this case (because it might give you a significant benefit)? I think the answer would be a clear "no". |
Thanks for this. I believe the arguments you provide would actually slightly favor the generics
This argument seems void as it can be used for both versions.
This actually says that the generic version has possibly (although unlikely) better memory layout/use, and has zero packing costs (because there is no packing), while the interface version would always has non-zero packing costs (even if minimal). Plus, as you say lower in your post, the compiler can always choose to implement with the non-generic version if it wants. So this seems to favor the use of the generics version.
This actually says that the generic version has potential improvement in performance (although unlikely), and the non-generic version is potentially (although not likely) slower. But hey it doesn't matter: the compiler can always choose to implement with the non-generic version if deemed better. So the programmer should just always use the generic version. I would personally add the argument that the non-generic version is far more readable and therefore outweighs the unlikely added costs. Maybe that tips the scale towards the interface version, at least for some people, I don't know. In either case, it's very likely that a schism will develop under go programmers. (Btw I do believe I have a possibly good argument for why |
I honestly don't know if I would. I don't think so, no. Well maybe I would because the compiler might in the future be better able to optimise my code. At least I won't lock out that opportunity. I don't know really. At least currently I don't have to think about this or make a decision because luckily Go doesn't have macros. |
Generic features tend to add type complexity - generally, generic code is significantly more complicated to understand due to the extra abstraction. If you doubt that I encourage you to decipher some of the generic pieces of code submitted with some of the issues against the prototype. Because of that, generic features should be used when all else fails; i.e., it's either not possible to write the code without it, it's massively slower, or it's significantly less type safe. I fail to see how that is the case with your specific question. You're of course free to code as you please, but personally, I'd go with the simpler solution that does the same thing. |
I agree. And I certainly agree with you in case of the I was voicing what I expect people will use as "excuse" to still use generics, as the arguments I gave have been used for the use of "template for everything" in for example libraries for C++. (Sorry for referencing other languages, I have tried not to for as long as possible). Keep in mind that there are many programmers who use Go on an intermittent basis (other microservices might be written in Kotlin or Rust), or are learning Go on the side coming from Java/C/C++. They will know templates/macros, but the dynamic interface concept requires some more effort to fully understand (think of nil and type assertion syntax etc). Therefore I believe they will be inclined to use the generics version, especially because it leaves open the opportunities for optimisation by the compiler. To them the interface version might be harder to understand "with everything that's going on at runtime". In the end, the results are fixed so both work sort of equally well. Of course if you think this will certainly not be an issue, I'll follow your lead. Just to be absolutely sure, I do think generics are very useful (even in the current proposal). It would allow the containers packages to be type safe and allows for many other very useful stuff. The only thing that concerns me is that I personally probably don't ever need a way to call methods on T, because I have runtime interfaces. Therefore I believed a simpler version like the operator list in this proposal is worth exploring. That has be done by now, of course. Thanks a lot for taking the time and effort to listen to the community. |
Thanks for the clarification and bringing in the perspective of people that might be new to Go. That's certainly something to consider, especially if we want to establish a "best practices" document. I agree that where interfaces work great now, generics are not needed. It's really situations where interfaces would be extremely cumbersome (because they need to model operators, or because we need extra type assertions) where generics should be considered. When you look at the examples in the design draft, it is noteworthy that most examples actually use constraints with type lists because we want to use operators in a generic way. But I think it would be too restrictive to drop the ability to require methods in constraints, which is what your proposal essentially does. (FWIW, the shortcomings I brought up earlier about problems with constants etc. could be addressed by making the constraint for a type parameter a type list. Or in other words, if we removed the ability to specify methods on constraints and only permitted type lists - and adjusted the syntax adequately - we'd arrive at the same place where you are with this proposal.) |
Yes, that's true, although you'd still need the special Regarding the current proposal, would it be worth considering using just |
Once you accept that a type constraint is an interface type, then to me it makes sense for the two special cases to also be interface types. So your suggestion is sort of like saying that |
Ha, yes I agree that it would be certainly weird to say Btw, I see the point that |
Out of curiosity,
I understood
So this probably means that not all interfaces can be used as function argument types? |
In the current generics design draft If we did permit that then your |
I see. So the I understand now that you see "comparable" as a property of a type instead of an operator between typed values (although it is defined like that for an interface). That might make sense indeed for a strictly typed language. It feels a bit weird to me that both these function would compile without problem:
In while in contrast, these don't:
Maybe it just something to get used to as a special case. |
There is a difference between |
@markusheukelom Presumably you meant |
Thanks for the suggestion. We're going to move forward with the current generics design draft, at least for now (https://go.googlesource.com/proposal/+/refs/heads/master/design/go2draft-type-parameters.md). This is a likely decline. We can revisit that decision if the current design draft is not adopted. Leaving open for four weeks for final comments. |
No further comments. |
The current proposal for generics is based on interfaces with added type lists. In this proposal I'd like to explore the possibility of using operators instead.
Rationale: compactness and orthogonality with interfaces.
My main argument is that generics should not allow you to do something you can already do with interfaces. For example, if interfaces can be used for meta typing you then have to decide whether to take io.Reader as a parametric type or as an argument type when implementing some function that reads data. In almost all cases it is hard to decide which option is objectivily better (if any), however the question would arise for a lot of functions. Therefore it is better not to provide the option.
Furthermore, interfaces are complicated enough as they are and deal mostly with dynamic (runtime) behavior. Overloading the interface concept to also express generic type constraints does not align very well with the Go adagium of "orthogonality" and can become confusing. I think it is therefore better to not use interfaces for generic type constrains.
Instead of using interfaces, we could use operators directly. This is compact and allows to write functions and types that can currently not be specified with intefaces (and vica versa). The drawback of this approach is that no methods can be called directly on variables of parametric types. However, that can always be alleviated by using a helper (generic) function or (generic) interface.
Proposed syntax:
The text was updated successfully, but these errors were encountered: