Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/go/ssa: method calls sometimes have different count of arguments because of closure #37253

Closed
Matts966 opened this issue Feb 16, 2020 · 8 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Tools This label describes issues relating to any tools in the x/tools repository.
Milestone

Comments

@Matts966
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.13.8 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN="~/go/bin"
GOCACHE="~/Library/Caches/go-build"
GOENV="~/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="~/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13.8/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13.8/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="~/go/src/golang.org/x/net/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/_q/1mwpxmn91wj5fs_c_f29mz3h0000gn/T/go-build604211852=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

analysis code

package main

import (
	"fmt"

	"golang.org/x/tools/go/analysis/singlechecker"

	"golang.org/x/tools/go/analysis"
	"golang.org/x/tools/go/analysis/passes/buildssa"
	"golang.org/x/tools/go/ssa"
)

func main() { singlechecker.Main(listargs) }

var listargs = &analysis.Analyzer{
	Name:     "listargs",
	Doc:      "List up all the calls and their arguments",
	Run:      run,
	Requires: []*analysis.Analyzer{buildssa.Analyzer},
}

func run(pass *analysis.Pass) (interface{}, error) {
	ssainput := pass.ResultOf[buildssa.Analyzer].(*buildssa.SSA)
	for _, fn := range ssainput.SrcFuncs {
		for _, b := range fn.Blocks {
			for _, i := range b.Instrs {
				c, ok := i.(ssa.CallInstruction)
				if ok {
					// fmt.Println("********************")
					fmt.Println(c.Common().StaticCallee().Object())
					for j, a := range c.Common().Args {
						fmt.Println("arg", j, a)
					}
					// fmt.Println("********************")
				}
			}
		}
	}
	return nil, nil
}

target code

package main

type X struct{}

func (X) method() {}

func x() {
        var x X
        x.method()
        m := x.method
        m()
}

What did you expect to see?

func (_~/listargs/test/m.X).method()
arg 0 *t0
func (_~/listargs/test/m.X).method()
arg 0 *t0

What did you see instead?

func (_~/listargs/test/m.X).method()
arg 0 *t0
func (_~/listargs/test/m.X).method()

Created closure does not have arguments though its object is the same as (X).method. When developing analysis tools, the consistency of the count of arguments is important for programmers so I think m is better with a receiver argument.
Is this behavior intended?

@odeke-em odeke-em changed the title Method calls sometimes have different count of arguments because of closure x/tools/analysis/singlechecker: method calls sometimes have different count of arguments because of closure Feb 17, 2020
@gopherbot gopherbot added this to the Unreleased milestone Feb 17, 2020
@gopherbot gopherbot added the Tools This label describes issues relating to any tools in the x/tools repository. label Feb 17, 2020
@toothrot toothrot added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Feb 18, 2020
@toothrot
Copy link
Contributor

/cc @ianthehat

@matloob
Copy link
Contributor

matloob commented Feb 18, 2020

cc @findleyr

If I understand this comment, the problem is that the arguments first function call, x.method() include the receiver argument, while the arguments for m() don't. This looks consistent to me: the first function has a reciever, while the second is niladic. It seems like this behavior is intended.

@Matts966
Copy link
Author

Thank you for your answer. I also confirmed that this behavior is correct and the receiver argument is in the FreeVar of the function m, closure functions.

However, for more simplification for users of the SSA interface, could the first argument of closure functions be the receiver and could the difference of arguments between methods and its closure functions be removed?

For now, users of the SSA interface should concern about the difference of arguments even the functions object is the same. This difference causes potential SEGMENTATION FAULT in analyzers assuming that the arguments are the same in the same function object.

The closure functions of methods are simply the curried functions with the partial application of only the first argument. So I think it is not problematic to unify their arguments.

Luckily, the SSA interface is noted that THIS INTERFACE IS EXPERIMENTAL AND IS LIKELY TO CHANGE. and users can deal with this change only by removing the checks and operations of ObjectFact about curried functions.

This change also solves the problem of uncertainty of the receiver argument because we can't choose the actual receiver from FreeVar easily for now, and the problem of redundant checks dealing with these differences.

@Matts966 Matts966 changed the title x/tools/analysis/singlechecker: method calls sometimes have different count of arguments because of closure x/tools/go/ssa: method calls sometimes have different count of arguments because of closure Feb 23, 2020
@matloob
Copy link
Contributor

matloob commented Feb 24, 2020

I'm leaning against making this change, especially because I don't want to break anyone, but I'll let @findleyr be the final decider.

@findleyr
Copy link
Contributor

I'm also leaning against this change. Yes, go/ssa is labeled as experimental, but it is being used and such a breaking changes carry a high cost/risk. I'm also not sure that this change would be philosophically correct: I don't think it makes sense for the receiver value to be both a bound FreeVar and an Arg. It seems to me that this is working as intended. This is consistent with the description of a bound closure as a synthetic wrapper, here:
https://github.com/golang/tools/blob/a0ec867d517c00ba65f6b4d0fc1de72414152fc7/go/ssa/wrappers.go#L169

@Matts966 could you expand on the problems this is causing? You say it can cause a segmentation fault: how specifically does that arise?

@Matts966
Copy link
Author

Matts966 commented Feb 28, 2020

@findleyr
Sorry, I should say that there is potential index out of range errors instead of segmentation faults.

Potential index out of range errors are like this.

type argumentsFact []ssa.Value

// Export facts about function arguments here.
var fact argumentsFact = callOfF.Args
pass.ExportObjectFact(f, &fact)
...
// Import facts about function arguments in other place.
pass.ImportObjectFact(f.Object(), &fact)
for i, arg := range fact {
    anotherCallOfF[i] // access arguments in the same index as in `callOfF`
    // This can cause index out of range errors because the count of arguments
    // can be different if the anotherCallOfF's function is method closure, whose
    // receiver is not in arguments but in free variables.
}

In order to handle these cases correctly, we, the users of ssa interface, for instance, writers of some linters, should always check the count of arguments and fetch the actual receivers from free variables, which is sometimes difficult to fetch because they are in the closure declaration, not in calls to closure.

As we already regard methods as functions and receivers as the first argument, it's easier if the method closures and their source functions are dealt with in the same way. So I thought that the bound receivers should be in arguments not in free variables at first, but I admit that this can break somethings.

@findleyr
Copy link
Contributor

findleyr commented Mar 3, 2020

How are you determining that callOfF and anotherCallOfF are the same? By associating Common().StaticCallee().Object()? I think the lesson here is that this is not sufficient.

For example, consider the following package:

package curry

type X struct{}

func (X) method() {}

func x() {
  var x X
  x.method()
  m := x.method
  m()
  n := X.method
  n(x)
}

When I run your analyzer, all of these invocations look the same:

> go run ./tmp/analyze ./tmp/curry
func (golang.org/x/tools/tmp/curry.X).method()
arg 0 *t0
func (golang.org/x/tools/tmp/curry.X).method()
func (golang.org/x/tools/tmp/curry.X).method()
arg 0 *t0

But if I instead print Common().StaticCallee(), it's clear that they are different:

> go run ./tmp/analyze ./tmp/curry
(golang.org/x/tools/tmp/curry.X).method
arg 0 *t0
(golang.org/x/tools/tmp/curry.X).method$bound
(golang.org/x/tools/tmp/curry.X).method$thunk
arg 0 *t0

I don't think we can change the meaning of Args to include a bound Freevars if that Freevar happens to be a receiver. I understand the API may be difficult to work with, but don't think this is the correct solution to that problem. Does that make sense?

@Matts966
Copy link
Author

Matts966 commented Mar 6, 2020

@findleyr
Thank you for your kind explanation. I will close this issue for I also understood that we should handle those differences correctly.

@Matts966 Matts966 closed this as completed Mar 6, 2020
@golang golang locked and limited conversation to collaborators Mar 6, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Development

No branches or pull requests

5 participants