Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: Go 2: permit conversion from []chan int to []<-chan int #41695

Closed
scorsi opened this issue Sep 29, 2020 · 25 comments
Closed

proposal: Go 2: permit conversion from []chan int to []<-chan int #41695

scorsi opened this issue Sep 29, 2020 · 25 comments
Labels
Milestone

Comments

@scorsi
Copy link

scorsi commented Sep 29, 2020

I would like to find a solution to #40010.

To remind the problem is:

func myFunc() []<-chan int {
	var res []chan int
	return res
}

Which result on a Compile error: cannot use res (type []chan x) as type []<-chan x in return argument.

The only workaround is either

  • To return a slice of channels without receive-only or send-only type qualifier.
    But it's not dev-friendly, what the dev have to do with a simple chan ? read or write ?.. This is what I choose actually.
  • Or to append the channels to a new slice with the right type.
func myFunc() []<-chan int {
	var chans []chan int
        // do your stuff to chans
        var res []<-chan int
        for c, _ := range chans {
                res = append(res, c)
        }
	return res
}

Which is very compute-time consuming and not the right solution too.

@gopherbot gopherbot added this to the Proposal milestone Sep 29, 2020
@beoran
Copy link

beoran commented Sep 29, 2020

What you want is covariance, where if X and Y are assignable, []X and []Y also become assignable ti each other. https://en.m.wikipedia.org/wiki/Covariance_and_contravariance_(computer_science)

In the past the Go authors have repeatedly stated that they want to keep the type system simple, and don't want to introduce covariance.

Generics will help a bit here, though, it will likely be possible to have a generic function like that does the copying conversion.

@ianlancetaylor ianlancetaylor changed the title Proposal: automatically cast / inference slice of receive-only or send-only channels proposal: Go 2: permit conversion from []chan int to []<-chan int Sep 30, 2020
@ianlancetaylor ianlancetaylor added v2 A language change or incompatible library change LanguageChange labels Sep 30, 2020
@ianlancetaylor
Copy link
Contributor

It's true that Go has so far rejected covariance, but this does seem to be a special case: a case of adding (or, presumably, removing) type qualifiers, rather than changing types in a more general way. In particular there is no question here of changing the method set, which is one of the reasons for rejecting covariance (https://golang.org/doc/faq#convert_slice_with_same_underlying_type).

@beoran
Copy link

beoran commented Sep 30, 2020

Well, if you are willing to consider this special case, then simply allowing a type cast ([]<-chan int)(chans), and presumably also (<-chan int)(ch) would be a backwards compatible feature.

@ianlancetaylor
Copy link
Contributor

Note that (<-chan int)(ch) is already permitted.

@beoran
Copy link

beoran commented Sep 30, 2020

Yes, that's correct.

However, I am not really convinced that the argument about changing the method set is very convincing. Already, Go allows certain casts where the method set changes, but not for others with the built in collections of these types.

https://play.golang.org/p/8Pfni3CMXvS

package main

import (
	"fmt"
)

type MyInt int
type MyIntChan chan MyInt
type ROMyIntChan <-chan MyInt

type IntChan chan int
type ROIntChan <-chan int
type ArrIntChan []chan int
type ArrROIntChan []<-chan int
type ArrMyIntChan []chan MyInt

func main() {
	var ch1 chan int
	ch2 := (<-chan int)(ch1)    // This does not change the method set.
	ch3 := IntChan(ch1)         // This changes the method set.
	ch4 := ROIntChan(ch1)       // This changes the method set.
	// ch5 := MyIntChan(ch1)    // Not allowed
	
	fmt.Printf("%v, %v, %v, %v\n", ch1, ch2, ch3, ch4)

	var i1 int
	i2 := MyInt(i1)             // This changes the method set.		
	fmt.Printf("%d, %d\n", i1, i2)	
	
	var a1 []int	
	// a2 := ([]MyInt)(a1)	    // Not allowed.
	fmt.Printf("%v\n", a1)
	
	var cha1 []chan int
	//cha2 := ([]<-chan int)(cha1)// Not allowed
	cha3 := ArrIntChan(cha1)      // This changes the method set.
	//cha4 := ArrROIntChan(cha1)  // Not allowed
        //cha5 := ArrMyIntChan(cha1)  // Not allowed

	fmt.Printf("%v\n", cha3)	
}

Probably similar for maps. Casts that would change the method set of the element of a array, slice, map, or channel are not allowed, but changing the method set for the whole collection, as well as adding a channel restriction to make it read only or write only are allowed.

Perhaps some of these other restrictions could be loosened, especially if no change in the underlying type occurs? As a chan MyInt has the same layout in memory as a chan int, and similarly for maps, slices and arrays of MyInt versus those with int?

@ianlancetaylor
Copy link
Contributor

Let's please not sidetrack this specific proposal into a general one about converting slices of elements with the same underlying type. Thanks.

I'm just pointing out that this specific proposal is not quite like the general case, because it involves only changing type qualifiers in the slice elements. Go has exactly one type qualifier, channel direction, and perhaps it is OK to permit conversions of slices that only change the type qualifier in the slice element type. Or perhaps not, I'm not sure. My only point is that this is different from the general case, in a way that may be relevant.

@beoran
Copy link

beoran commented Sep 30, 2020

OK, I agree that a more general proposal would be a separate issue.

However, the point I was trying to make was that Go, as it is now, is consistent in that casts between collections of types that have the same underlying type are not allowed. That's why I thing that an exception just for read only channels would make Go harder to learn with limited benefits.

@scorsi
Copy link
Author

scorsi commented Sep 30, 2020

The debate about ([]MyInt)(ints) is another one, I'm agreed. Here it's not about underlying type but only about underlying type qualifier.

I'm not sure this is an exception about the casting of underlying type, for me chan int and <-chan int is the same type just it allow or not different things (read-only, write-only or read-write). What I think is that even in read-only or write-only, the channel stay a channel of the same type.

About the consistency, what I suggests is to make the same behaviour for all type qualifier (even if as @ianlancetaylor noticed, it only exists channel direction actually).

This is a subject to debate with the community. Because, of course, people will think that underlying type casting should be accepted if that specific case here is accepted.

@randall77
Copy link
Contributor

This proposal suggests allowing

var x []chan int
y := ([]<-chan int)(x)

But what about

var x []<-chan int
y := ([]chan int)(x)

I think we don't want to allow the latter. So it isn't just about changing type qualifiers. We need the additional restriction that the cast is changing qualifiers in the direction that restricts the allowed operations.

As a separate thought, this proposal assumes that chan int and <-chan int have the same representation. That's true today, but need not be true in some other implementation. I could imagine channels implemented as endpoints, where bidirectional channels are pairs of pointers, one to a read endpoint and one to a write endpoint, and the direction-qualified channels are single pointers.

@bcmills
Copy link
Contributor

bcmills commented Sep 30, 2020

In the other direction: if we are going to assume that chan int and <-chan int have the same representation, would that also imply allowing (say) a func() chan int to be converted to a func() <-chan int?

@scorsi
Copy link
Author

scorsi commented Sep 30, 2020

I think the actual type qualifier casting behaviour must stay the same, to remind (where T is a type like int and never change) :

  • chan T to <-chan T ✔️ OK
  • chan T to chan<- T ✔️ OK
  • <-chan T to chan T ❌ NOT OK
  • chan<- T to chan T ❌ NOT OK
  • <-chan T to chan<- T ❌ NOT OK
  • chan<- T to <-chan T ❌ NOT OK

To resume: we can add a type qualifier but we can't change it or remove it.

We can add this behaviour to different things, but now, we can talk about underlying type qualifier casting.

Slices

I assume we can add type qualifier casting to slices (not included options are not allowed) :

  • []chan T to []<-chan T
  • []chan T to []chan<- T

Maps

I assume we can add type qualifier casting to maps (not included options are not allowed) :

  • map[T]chan T to map[T]<-chan T
  • map[T]chan T to map[T]chan<- T

Weird but maybe:

  • map[chan T]T to map[<-chan T]T
  • map[chan T]T to map[chan<- T]T

Functions

I assume we can also add type qualifier casting to functions (not included options are not allowed) :

  • func() chan T to func() <-chan T
  • func() chan T to func() chan<- T

Not totally sure for functions to be honest 🤔 It's pretty weird here, no ?

Channels

I assume we can also add type qualifier casting to channels (not included options are not allowed) :

  • chan chan T to chan <-chan T
  • chan chan T to chan chan<- T

Pointers

I assume we can also add type qualifier casting to pointers (not included options are not allowed) :

  • *chan T to *<-chan T
  • *chan T to *chan<- T

@beoran
Copy link

beoran commented Sep 30, 2020

I think from the above we can see this is getting complicated, with limited benefits.

As an aside, it also makes me wonder if we should perhaps consider allowing type qualifiers on all types, <-T for a read only T and T<- for a write only T. But I digress...

@scorsi
Copy link
Author

scorsi commented Oct 1, 2020

This is getting complicated... yes and no, because it's for all the possible cases, not an exception to slices. ^^
I'm not sure there is limited benefits. We gain to have that automatic casting, and if you don't need it, nothing will change for you. Of course casting func() chan T to func() <-chan T may be weird, but for pointers, slices and maps it make sense.

This doesn't not apply to all types. Only "containers" to the underlying type qualifier.
About the <-T or T<-, I didn't see the benefits and it's out of this context... Also, there is already var and const and it's enough I think.

Btw, this was an idea... It require to be validated by the community. ;)

@deanveloper
Copy link

deanveloper commented Oct 3, 2020

This proposal is impossible without introducing runtime panics, as adding covariance to a collection in OOP will restrict the operations which you can do on that collection. There is no way of doing this in Go.

The following functions seem innocent:

func SetPointer(assign *<-chan int, value <-chan int) {
    *assign = value
}
func FillSlice(slice []<-chan int, value <-chan int) {
    for i := range slice {
        slice[i] = value
    }
}

Until we realize that they make this possible:

func main() {
    ch := make(chan int)
    readCh := (<-chan int)(make(chan int))

    // we assign ch, type `chan int`, a value of type `<-chan int`.
    // this cannot be allowed.
    SetPointer(&ch, readCh)

    // and same issue with slices:
    chSlice := make([]chan int, 10)
    FillSlice(chSlice, readCh)
}

In languages like Java, adding covariance to a class's generic type parameter will limit the methods which you can use with it. It adds a ton of complications, and gets very confusing. We should really avoid adding this to Go in my opinion.

@deanveloper
Copy link

deanveloper commented Oct 3, 2020

I'd also like to point out that allowing []<-chan int to []chan int conversions is safe in the circumstances listed above, however of course it isn't safe in the circumstances listed below:

func ReadFirst(slice []chan int) chan int {
    return slice[0]
}

Because, of course, passing in a slice of []<-chan int to that function will return a chan int.

@deanveloper
Copy link

deanveloper commented Oct 3, 2020

To give an idea of how this issue is solved in Java:

// in the following examples, E is the element type of the list, and V is the type of some
// variable (whatever is being assigned to/from the list)

// safe: we can pass any List<E> such that E extends V,
// meaning that E is always assignable to V
public static <E extends V, V> V getFirst(List<E> list) {
    return list.get(0)
}

// safe: we can pass any List<E> such that E is a superclass of V,
// so we know V is always assignable to E
public static <E super V, V> void setFirst(List<E> list, V v) {
    return list.set(0, v)
}

// compile error, as E is a superclass of V, so E is not
// assignable to V. (aka Gopher g = animalList.get(0); is not valid)
public static <E super V, V> V getFirst(List<E> list) {
    return list.get(0) 
}

// compile error, as E is a subclass of V, so V is not
// assignable to E. (aka gopherList.set(0, animal); is not valid)
public static <E extends V, V> void setFirst(List<E> list, V v) {
    return list.set(0, v)
}

@scorsi
Copy link
Author

scorsi commented Oct 5, 2020

Hello @deanveloper

This proposal is impossible without introducing runtime panics, as adding covariance to a collection in OOP will restrict the operations which you can do on that collection. There is no way of doing this in Go.

FIrst, we don't talk about adding covariance. We're not talking either about adding runtime panics because it's not really in the mindset of Go. And finally Go is not OOP.

In languages like Java, adding covariance to a class's generic type parameter will limit the methods which you can use with it. It adds a ton of complications, and gets very confusing. We should really avoid adding this to Go in my opinion.

We absolutely not talking about adding covariance... Furthermore, types are not type qualifiers (chan int and <-chan int are of the same type, just one has a type qualifier where the other one hasn't).

I'd also like to point out that allowing []<-chan int to []chan int conversions is safe in the circumstances listed above, however of course it isn't safe in the circumstances listed below:

func ReadFirst(slice []chan int) chan int {
    return slice[0]
}

Because, of course, passing in a slice of []<-chan int to that function will return a chan int.

I never talk about allowing <-chan T to chan T. Actually, we can't change <-chan T to chan T and we don't talk about modifying that here.

Finally, I don't understand why you talk about Java. Go and Java are two totally different languages. Your last snippet show that you don't understand that type qualifiers are not types, it's a Go specific feature which I don't see in any other languages.


Anyway, you pointed out a really good example where the type qualifier casting might create issues:

The following functions seem innocent:

func SetPointer(assign *<-chan int, value <-chan int) {
    *assign = value
}

Here, the function looks good, but the automatic casting from chan T to *<-chan T is not good at all since the code change the pointed variable. 🤔
There is the same issue with *[]chan T and *[]<-chan T which might not be allowed, but chan *T to <-chan *T is not a problem. So we can't allow underlying type qualifier casting bellow a pointer.
This might be the only one exception at least if anyone found another exception ?

@scorsi
Copy link
Author

scorsi commented Oct 5, 2020

Hugh... It seems that there is some another BIG exceptions...

package main

import (
	"fmt"
)

func SetSlice(assign []<-chan int, value <-chan int) {
	assign[0] = value
}

func main() {
	ch := []chan int{make(chan int)}
        readCh := (<-chan int)(make(chan int))

        SetSlice(ch, readCh)
}

Then come another one idea: (my concrete example why we should add that)

func f() []<-chan int {
	chans := []chan int{make(chan int)}
	go func(){
		time.Sleep(1000)
		chans[0]<-42 // Error: chans[0] is now of type `<-chan int`
	}()
	return chans
}

func main() {
	chans := f()
        ch := make(<-chan int)
	chans[0] = ch // Error: chans[0] is of type `chan int` in the function f()
	time.Sleep(1500)
}

This example show that in term of types, both main and f functions are correct. But the code is not.

Soooo... Unfortunately, we can't add underlying type qualifier casting to Go because it may add more and more issues.

At least if someone else has an idea to counter that both issues. Adding "runtime panics" may fix that but it's absolutely not in the Go philosophy. Adding const/unmodifiable variables may resolves issues but it's another subject..

@bcmills
Copy link
Contributor

bcmills commented Oct 5, 2020

@scorsi, @deanveloper is referring to Java because Java has covariant arrays, and as a result has a run-time ArrayStoreException. The analogous behavior in Go would be a panic when setting a slice element.

And I agree with @deanveloper that this conversion cannot be allowed safely. (You can already do it unsafely using unsafe.Pointer, of course: https://play.golang.org/p/LxhUGK7U3qQ.)

@scorsi
Copy link
Author

scorsi commented Oct 5, 2020

@scorsi, @deanveloper is referring to Java because Java has covariant arrays, and as a result has a run-time ArrayStoreException. The analogous behavior in Go would be a panic when setting a slice element.

Ok now I see the relationship with Java. But Go can't add panics when settings a slice element because a slice is a type and there is no differences between []int and []string in term of slice possibilities/functionalities.

And I agree with @deanveloper that this conversion cannot be allowed safely. (You can already do it unsafely using unsafe.Pointer, of course: https://play.golang.org/p/LxhUGK7U3qQ.)

@bcmills which conversion cannot be allowed safely ? chan T to <-chan T should never be considered as safe at all.


Btw, there are a lot of deep potentials issues when allowing underlying type qualifier casting.
I think that this issue and proposal can be closed except if someone has an idea about the issues @deanveloper and I pointed out ?

@bcmills
Copy link
Contributor

bcmills commented Oct 5, 2020

We could, however, allow the copy and append built-ins to copy the elements from a slice of undirected channels to a slice of directed channels.

Today they fail with: arguments to copy have different element types []<-chan int and []chan int: https://play.golang.org/p/4GVt96_08fp

However, there is no fundamental reason why they need to do so, given that it is possible for user code to implement exactly the same copying and appending behavior: https://play.golang.org/p/DCNKeILHQEV

Perhaps I will file that as a separate proposal.

@bcmills
Copy link
Contributor

bcmills commented Oct 5, 2020

@scorsi: we cannot safely allow any conversion from a slice of (or pointer to) chan T to a slice of (or pointer to) <-chan T or chan<- T.

Pointers and slices cannot be covariant, because the type itself cannot distinguish between inputs (writes) and outputs (reads). In contrast, functions can (at least theoretically) be covariant, because the inputs (arguments) and outputs (return-values) in the function signature are distinct.

@scorsi
Copy link
Author

scorsi commented Oct 5, 2020

We could, however, allow the copy and append built-ins to copy the elements from a slice of undirected channels to a slice of directed channels.

Yes ! This is a really good idea here. Allowing those both functions might do the trick. If we edit the copied slice we don't change the original slice but we can read (or write) from (or to) the original channel.

Pointers and slices cannot be covariant, because the type itself cannot distinguish between inputs (writes) and outputs (reads). In contrast, functions can (at least theoretically) be covariant, because the inputs (arguments) and outputs (return-values) in the function signature are distinct.

You can't append or set a chan<- int to a []<-chan int for example so today slices, pointers and maps are covariant (ref: https://play.golang.org/p/h4idwaL2eGp).
The issue you pointed out is that if we allow the conversion from []chan T to []<-chan T (same apply for pointers and maps) so this is no longer covariant because set and append must only allow variables of type chan T and not <-chan T and so it's by addition no longer safe.
I agree !

@ianlancetaylor
Copy link
Contributor

Based on the discussion above, this is a likely decline. Leaving open for four weeks for final comments.

@ianlancetaylor
Copy link
Contributor

No further comments.

@golang golang locked and limited conversation to collaborators Nov 10, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

8 participants
@beoran @ianlancetaylor @deanveloper @bcmills @randall77 @scorsi @gopherbot and others