Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: spec: infer argument types when a type set only represents its core type #52272

Open
changkun opened this issue Apr 11, 2022 · 22 comments
Labels
generics Issue is related to generics Proposal Proposal-Hold TypeInference Issue is related to generic type inference
Milestone

Comments

@changkun
Copy link
Member

What version of Go are you using (go version)?

$ go version
1.18

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="auto"
GOARCH="arm64"
GOBIN=""
GOCACHE="/Users/changkun/Library/Caches/go-build"
GOENV="/Users/changkun/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/changkun/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/changkun/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/changkun/goes/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/changkun/goes/go/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.18"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/changkun/dev/poly.red/polyred/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/g2/6fmr1qzx0ns3shq74zrp6bd40000gn/T/go-build1836918707=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

https://go.dev/play/p/bW8xjgG0x6Z

In this example, When we use Option1[T] (constrainted by A or B) as
an argument of NewA[T], there is only one possibility to infer the concrete
type of Option1[T], because NewA only accept A as type parameter.

package main

import "fmt"

type A struct{ val int }
type B struct{ val int }

type Option[T A | B] func(v *T)

func NewA[T A](opts ...Option[T]) *T {
	a := new(T)
	for _, opt := range opts {
		opt(a)
	}

	return a
}

func NewB[T B](opts ...Option[T]) *T {
	b := new(T)
	for _, opt := range opts {
		opt(b)
	}

	return b
}

func Option1[T A | B](val int) Option[T] {
	return func(v *T) {
		switch vv := any(v).(type) {
		case *A:
			vv.val = val
		case *B:
			vv.val = val
		}
	}
}

func main() {
	u := NewA(Option1[A](42)) // OK
	v := NewB(Option1[B](42)) // OK

	// NewA(Option1(42))    // ERROR: cannot infer T
	// NewB(Option1(42))    // ERROR: cannot infer T

	fmt.Println(u, v) // &{42} &{42}
}

What did you expect to see?

It is possible to write code without specifying type parameter in this case:

NewA(Option1(42))

What did you see instead?

Compile error: cannot infer T

More reasons

This is a quite simplification when the options are distributed in a different package, say pkgname:

package pkgname

import "fmt"

type A struct{ val int }
type B struct{ val int }

type Option[T A | B] func(v *T)

func NewA[T A](opts ...Option[T]) *T {
	a := new(T)
	for _, opt := range opts {
		opt(a)
	}

	return a
}

func NewB[T B](opts ...Option[T]) *T {
	b := new(T)
	for _, opt := range opts {
		opt(b)
	}

	return b
}

func Option1[T A | B](val int) Option[T] {
	return func(v *T) {
		switch vv := any(v).(type) {
		case *A:
			vv.val = val
		case *B:
			vv.val = val
		}
	}
}

---

package main

func main() {
	pkgname.NewA(pkgname.Option1[pkgname.A](42)) // Feels stutter
	pkgname.NewB(pkgname.Option1[pkgname.B](42))

	// pkgname.NewA(pkgname.Option1(42)) // Would be much simpler in terms of use and readability
	// pkgname.NewB(pkgname.Option1(42))
}
@ianlancetaylor
Copy link
Contributor

This looks like a fairly complicated type inference rule.

  • Look at the parameter list for NewA, see Option[T]
  • Look at the constraint for T, see that it must be A
  • Therefore the parameter has to be Option[A]
  • Based on that, infer the type argument to Option in the call

The example is sufficiently abstract that it's hard to understand why anybody would write code this way. Why not write

func NewA(opts ...Option[A]) *A {

Making type inference work for that simpler case is something like #47868.

I want to stress that arranging for the compiler to make every type inference that a human can make is not a goal. The goal is to only make very simple and obvious type inferences, ones that can be clearly described and understood by everybody.

In any case we're not to change type inference soon, putting this on hold for now.

@ianlancetaylor ianlancetaylor added this to the Proposal milestone Apr 12, 2022
@ianlancetaylor ianlancetaylor changed the title cmd/compile: infer argument types when a type set only represents its core type proposal: spec: infer argument types when a type set only represents its core type Apr 12, 2022
@changkun
Copy link
Member Author

The example is sufficiently abstract that it's hard to understand why anybody would write code this way.

I can think of an initial reason. Putting it in type parameter could allow more types for future changes (despite after instatiation, it is drifted from original purpose, and the same as writing New(...Option[A]) *A or New(...Option[Alike]) *Alike then fall back to #47868):

func NewA[T A | Alike | Alike2](opts ...Option[T]) *T {
	t := new(T)
	for _, opt := range opts {
		opt(t)
	}
	return t
}
New[A](Option1(42))
New(Option1[A](42))

New[B](Option1(42))
New(Option1[B](42))

Making type inference work

Even we write code like this:

func NewA(opts ...Option[A]) *A

We still need provide the type parameter for Option1 today (https://go.dev/play/p/Ja26sqyex71):

u := NewA(Option1[A](42)) // OK
v := NewA(Option1(42))    // ERROR: cannot infer T

@changkun
Copy link
Member Author

In any case, we're not to change type inference soon, putting this on hold for now.

I don't know if the mitigated form (as you suggested) still fits the current proposal. But it's pretty upset to hear that we are not planning on changing type inference soon. Here is some actual code:

https://github.com/polyred/polyred/blob/7007de39046da4ead38eb214598c42b490fc472e/material/material.go#L37

https://github.com/polyred/polyred/blob/7007de39046da4ead38eb214598c42b490fc472e/material/options.go#L17-L30

Using generics can improve the safety of using an option to work for different types. Specifically, assume an option is defined as type Option func(SomeInterface), and an option internally use type switch to do different configuration for different concrete types. In that case, the user must know the implementation detail of such an option to entire it is safe to use the option for different types.

make very simple and obvious type inferences, ones that can be clearly described and understood by everybody

I don't know how to evaluate this goal. But from an honest user perspective, without type inference in this case, it is quite unpleasing experience to ask API users provide an obvious type, and I'd argue this could be something called "very simple and obvious type inferences".

But without this type of inference, really makes the code much more verbose. E.g.:

m := material.NewBlinnPhong(
	// require type parameter is really uncessary I think
	material.Texture[material.BlinnPhong](buffer.NewTexture()), 
	material.Diffuse(color.FromValue(0.6, 0.6, 0.6, 1.0)),
	material.Specular(color.FromValue(0.6, 0.6, 0.6, 1.0)),
	material.Shininess(25),
)
opts := []material.Option[material.BlinnPhong]{
	material.Diffuse(mat.Diffuse),
	material.Specular(mat.Specular),
	material.Shininess(mat.Shininess),
	// require type parameter is really uncessary I think
	material.FlatShading[material.BlinnPhong](false), 
	material.AmbientOcclusion[material.BlinnPhong](false),
	material.ReceiveShadow[material.BlinnPhong](false),
}

@ianlancetaylor
Copy link
Contributor

I understand that it is annoying to have to write an explicit type argument. But we know from long experience with C++ that type inference is a very subtle and confusing property. We think it's extremely important that people reading the code always be able to easily understand exactly what type argument will be inferred for any given call.

This is not unlike the way that people writing C or C++ can get confused about the arithmetic properties of operations involving different types. Go avoided that confusion by simply saying that you must always use the same type with an operation, and requiring explicit conversions for anything else. Those explicit conversions can be very obvious, but requiring them was a big step toward making Go easier to understand.

Once we add a form of type inference to the language, we can never remove it due to backward compatibility concerns. So we are being very careful about type inference rules, to make sure that we don't tip over the edge into code that is hard for people to read.

@sammy-hughes
Copy link

sammy-hughes commented Apr 13, 2022

@changkun, how different would your experience be if you reorganized your approach to receive, not an untyped value, but an instance either of A or B? It's not clear what work your snippet modelled, but I think this is roughly your example: https://go.dev/play/p/2FuvAagvasE Do you agree?

You'll notice it compiles and runs, but at least how I interpreted it, your use-case looks much more like that of an interface. The thing that tipped me off is, well, you converted it to an interface. I suppose you're intending to mutate values of arbitrary type, according to an established protocol, as a constructor or an iterator.

I think the guideline is that generic functions are routers (switches, multiplexers, observers, cursors, etc) and interfaces define an inspective/mutative API. Generally, other than primitives, if you need to know what you have, you should be using an interface.

You may notice my example breaks down a little, but your apparent API clashes with my pure functions, so I had to adjust my example in clumsy ways.
https://go.dev/blog/when-generics

@changkun
Copy link
Member Author

This is not unlike the way that people writing C or C++ can get confused about the arithmetic properties of operations involving different types. Go avoided that confusion by simply saying that you must always use the same type with an operation, and requiring explicit conversions for anything else.

I can align myself with the general goal here. But I think this answer is too fuzzy and needs proper specialization to fit into our current context.

Is there really a confusion conversion in this specific case? Especially when there is only a single possible inference result. If yes, how might it look like?

Those explicit conversions can be very obvious, but requiring them was a big step toward making Go easier to understand.

We probably need to be more specific to define the metrics regarding what it means to be "easier to understand." Otherwise, the discussion cannot proceed and could only stay on personal preferences.

As elaborated before, I would propose an (ad-hoc) metric that quantitatively measures: the "Go programmer's understanding/reading difficulty" by using the number of inference ambiguity. For instance, given the following example:

type Float interface {
	float32 | float64
}

type Vec3[T Float] struct {
	X, Y, Z T
}

func NewVec3[T Float](x, y, z T) Vec3[T] {
	return Vec3[T]{x, y, z}
}

On the caller/user side, the following code:

p := NewVec3(1.0, 2.0, 3.0)

has two inference ambiguity: NewVec3[float32] or NewVec3[float64] because 1.0 is untyped. In this case, they might expect NewVec3[float32] other than NewVec3[float64] but actually the other way around. Hence, the Go programmer's understanding/reading difficulty is 2.

However, when writing code like this:

var x, y, z float32 = 1.0, 2.0, 3.0
p := NewVec3(x, y, z)

The inference ambiguity of NewVec3 collapses into the only possibility: NewVec3[float32]. In this case, the Go programmer's understanding/reading difficulty is 1.

Comparably, the second example is better than the first. Therefore, to improve the code readability and reduce the "Go programmer's understanding/reading difficulty", a Go programmer could specify a type parameter to reduce the reading difficulty from 2 to 1: p := NewVec3[float32](1.0, 2.0, 3.0).

The above is not a perfect definition, but it holds potential insights to avoid preference arguments.

Back to the option example on top of this thread:

type A struct { v int }
type B struct { v int }

type Option[T A | B] func(*T)

func V[T A | B](v int) Option[T] {
    return func(t *T) {
        ...
    }
}

func New[T A](o Option[T]) *T { ... }
func NewA(o Option[A]) *A { ... }

On the caller/reader side:

New(V(42))
NewA(V(42))

The Go programmer's understanding/reading difficulty for both cases is 1 because, for function V, only one possible inference is V[A], and for the function New, the only possible inference is New[A]

@changkun
Copy link
Member Author

I think this is roughly your example: https://go.dev/play/p/2FuvAagvasE Do you agree?

From what I parsed from the code:

func Option[T SuccessCode | FailureCode](i T) OptionFunc[T]

I think it aligns with the similar definition of Option in my posted example in this thread, but I am don't think your example reveals the benefits of using type inference because your code that uses Option does not indicate the benefits of type inference:

fnOk := Option(SuccessCode(200 << shiftOk))
var okCode SuccessCode
fnOk(&okCode)// both fnOK and okCode are explicit typed before using SuccessCode

Your example cannot be simplified to:

fnOk := Option(200 << shiftOk) // fnOK has inference ambiguity and cannot be inferenced
var okCode SuccessCode
fnOk(&okCode)

if you need to know what you have, you should be using an interface.

I think we could clearly say the interface is not a win over generics. An article elaborates why to interface may not do a good job and why using generics might make things better: https://golang.design/research/generic-option/#using-interfaces.

@sammy-hughes
Copy link

Speaking as someone that abuses APIs for sport, I think the example given concerning the "heavy cost of interfaces" explicitly bypasses the protection of an interface, then complains that interfaces aren't safe enough.

@ianlancetaylor
Copy link
Contributor

We probably need to be more specific to define the metrics regarding what it means to be "easier to understand." Otherwise, the discussion cannot proceed and could only stay on personal preferences.

Obviously this is just my own personal preference, but I think that it would be a serious mistake to look for objective metrics for Go language design. There is no good metric for "easy to use" or "easy to understand." Those are subjective measurements. When the language was first being designed, features only went in if all three of the original designers agreed on them. That approach has served Go well so far.

@changkun

This comment was marked as off-topic.

@ianlancetaylor

This comment was marked as off-topic.

@changkun

This comment was marked as off-topic.

@ianlancetaylor

This comment was marked as off-topic.

@changkun

This comment was marked as off-topic.

@ianlancetaylor
Copy link
Contributor

But as I mentioned earlier, I don't believe in the notion of a metric for this in the first place. So before we agree to improve the metric, we need to agree that a metric is the appropriate way to judge a change like this.

Stepping back, I doubt this thread of the conversation will be useful. To restate what I wrote above in #52272 (comment), the type inference you are looking for here is an inter-weaving of the parameter type and the type constraint. If I understand it correctly, this new kind of type inference only applies when the type constraint only has a single type. I think you are suggesting when a type parameter has a type constraint with a single type, then we should copy that type constraint into uses of the type parameter in the regular parameter list.

That seems like a complicated rule to write down. It also seems like something that will approximately never happen. I don't understand why we should spend compilation time looking for this case. I understand that you have a case for it, but I am very skeptical that will be a common case in Go programs.

I will repeat that it is absolutely not a goal for the compiler to make all type inferences that a human could make.

@changkun
Copy link
Member Author

Stepping back, I doubt this thread of the conversation will be useful. To restate what I wrote above in #52272 (comment), the type inference you are looking for here is an inter-weaving of the parameter type and the type constraint. If I understand it correctly, this new kind of type inference only applies when the type constraint only has a single type. I think you are suggesting when a type parameter has a type constraint with a single type, then we should copy that type constraint into the uses of the type parameter in the regular parameter list.

At the point, while I was filling this issue, I wasn't expecting this to become a language change, but at this point, the previous comments regarding why not write

func NewA[T A](opts ...Option[T]) *T

to

func NewA(opts ...Option[A]) *A {

Need an update.

The reason is we also need the ability to instantiate functions to a defined type (usually a composite structural type):

func NewA[T ~A](opts ...Option[T]) *T

which is currently in a different proposal #52318.

The proposed rule here then becomes much more general rather than "the type constraint only has a single type". Instead, it is "the type constraint has a core type".

That seems like a complicated rule to write down.

I can try to prototype a spec change if necessary.

It also seems like something that will approximately never happen. I don't understand why we should spend compilation time looking for this case. I understand that you have a case for it,

I am not sure how to calibrate this prior. Despite the posted example only from mine, I think the level of abstraction should give a sense of how it might happen.

But if you like, I can also try to prototype a change to measure the increased compilation time and the total keystroke time for a programmer to fill these type parameters (such as typing 100 type parameters in a ...Option[T] context).

but I am very skeptical that will be a common case in Go programs.

Perhaps Kubernetes? Aren't they have vast enough functional option patterns? What about similar software that need much of configurations? Although their codebase is stable, for future Go programs, if the ability of expressiveness is not there, programs won't be able to write. Sure, it will never be a common case because it can never be written down. We could agree that Go lacks its user group in some domains outside the cloud.

I will repeat that it is absolutely not a goal for the compiler to make all type inferences that a human could make.

I completely agree with this vision regarding "all", but I also think the vision is less practical because it does not give enough guidance regarding what rule should not fall into the "all".

@ianlancetaylor

This comment was marked as off-topic.

@changkun

This comment was marked as off-topic.

@ianlancetaylor

This comment was marked as off-topic.

@changkun

This comment was marked as off-topic.

@ianlancetaylor ianlancetaylor added TypeInference Issue is related to generic type inference generics Issue is related to generics labels Apr 18, 2022
@mdempsky
Copy link
Member

mdempsky commented Apr 27, 2022

Do I understand correctly that the crux of this proposal is that given:

type A int
type B int
func f[T A](T) {}
func g[T A | B]() T { return *new(T) }

the type checker should infer f(g()) as f[A](g[A]())?

@changkun
Copy link
Member Author

The simplest form, I think, yes.

In general, g may be constrained by n different types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
generics Issue is related to generics Proposal Proposal-Hold TypeInference Issue is related to generic type inference
Projects
None yet
Development

No branches or pull requests

4 participants