Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: cmd/vet: add new flag to support stdmethod checks for specific methods #52445

Open
fishy opened this issue Apr 19, 2022 · 17 comments
Open
Labels
Analysis Issues related to static analysis (vet, x/tools/go/analysis) Proposal
Milestone

Comments

@fishy
Copy link

fishy commented Apr 19, 2022

What version of Go are you using (go version)?

$ go version
go version go1.18.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/fishy/.cache/go-build"
GOENV="/home/fishy/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/fishy/.gopath/pkg/mod"
GONOPROXY="redacted"
GONOSUMDB="redacted"
GOOS="linux"
GOPATH="/home/fishy/.gopath"
GOPRIVATE="redacted"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.18"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.18/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.18.1"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/fishy/work/thrift/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2627081732=/tmp/go-build -gno-record-gcc-switches"

What did you do?

In Apache Thrift go library, the interface TProtocol has
ReadByte and WriteByte that's different from the expectation of stdmethods:

type TProtocol interface {
  ...
  WriteByte(ctx context.Context, value int8) error
  ...
  ReadByte(ctx context.Context) (value int8, err error)
  ...
}

This causes go vet to complain about the implementations, and we have to do extra work to disable stdmethods check for packages containing the implementation, for example: https://github.com/reddit/baseplate.go/blob/205ac6852ad82ad5508106a6cb7156e2e73d3b3a/scripts/linters.sh#L19-L31

IMHO the stdmethods check should auto skip "violations" that explicitly implement a different interface (e.g. this line of var _ thrift.TProtocol = (*tDuplicateToProtocol)(nil) should be enough to silent stdmethods on (*tDuplicateToProtocol).ReadByte and (*tDuplicateToProtocol).WriteByte), as an explicit interface with different function signature should be enough signal that the difference is intentional, otherwise the interface should embed io.ByteReader/io.ByteWriter instead.

What did you expect to see?

What did you see instead?

$ go vet .
# github.com/apache/thrift/lib/go/thrift
./binary_protocol.go:215:27: method WriteByte(ctx context.Context, value int8) error should have signature WriteByte(byte) error
./binary_protocol.go:423:27: method ReadByte(ctx context.Context) (int8, error) should have signature ReadByte() (byte, error)
...
@timothy-king timothy-king changed the title cmd/vet: False negative/exception in stdmethods check cmd/vet: False positive on ReadByte/WriteByte in stdmethods Apr 19, 2022
@timothy-king timothy-king added Analysis Issues related to static analysis (vet, x/tools/go/analysis) NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. labels Apr 19, 2022
@timothy-king
Copy link
Contributor

See also #36970. This is roughly the same but a different function.

An alternative is to rename the methods. This is not ideal, but it does work.

IMHO the stdmethods check should auto skip "violations" that explicitly implement a different interface (e.g. this line of var _ thrift.TProtocol = (*tDuplicateToProtocol)(nil) should be enough to silent stdmethods on (*tDuplicateToProtocol).ReadByte and (*tDuplicateToProtocol).WriteByte), as an explicit interface with different function signature should be enough signal that the difference is intentional

This is a reasonable mechanism for suppression. There would be a requirement that the assignment appears in the declaring package. There would also need to be a suppression mechanism for the warning on the interface definition. A possible mechanism could again be an assignment:

var _ interface{WriteByte(ctx context.Context, value int8) error} = TProtocol(nil)

IMO changing the interface of stdmethods to allow for suppression [mechanism aside] goes against my intuition of 'if it passes cmd/vet [stdmethods], then if I see name a method named M, M has signature S [and implements standard interface I]'. I think this is a sufficiently large interface change to cmd/vet to require a proposal. Instructions on the proposal process: https://github.com/golang/proposal#the-proposal-process.

otherwise the interface should embed io.ByteReader/io.ByteWriter instead.

Would this effectively require that interfaces embed standard interfaces for interface definitions to be checked by stdmethods? That would be turning off checking for pre-existing interfaces if they contain the same function but do not embed the interface. This would also be true going forward. (Is this different than just type-checking at this point?)

@timothy-king timothy-king added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. and removed NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. labels Apr 19, 2022
@fishy
Copy link
Author

fishy commented Apr 19, 2022

An alternative is to rename the methods. This is not ideal, but it does work.

thrift.TProtocol dated back to 0.7.0/2011 (the ctx args were added in 0.14.0/2021). Renaming the methods will be a big breaking change to all users. Also all other thrift language libraries call the methods readByte/writeByte (see: java, c++, py, etc.), so renaming is really infeasible in this case.

Would this effectively require that interfaces embed standard interfaces for interface definitions to be checked by stdmethods?

No. What I'm suggesting is by default it still checks everything (the current behavior), just allow explicit suppression.

@timothy-king
Copy link
Contributor

Thank you for the clarification. If I now understand correctly, this can be read as 'If not suppressed, keep the pre-existing check'?

@fishy
Copy link
Author

fishy commented Apr 20, 2022

Thank you for the clarification. If I now understand correctly, this can be read as 'If not suppressed, keep the pre-existing check'?

correct

@timothy-king
Copy link
Contributor

timothy-king commented Apr 20, 2022

Thinking out loud about yet more options:

  1. An option is to post-process the vet output. (Probably as annoying as excluding the package in the shell script.)
  2. An option is to add a flag that allows for suppression. Some precedent for this, but precedent is not flexible enough to turn this off for some packages or some types.
  3. An option is to remove these [edit: ReadByte/WriteByte] from stdmethods. (IMO this would also need a proposal).

@robpike
Copy link
Contributor

robpike commented Apr 20, 2022

You can run vet by hand with -stdmethods=false. Run go tool vet help for more information.

@fishy
Copy link
Author

fishy commented Apr 20, 2022

We are already using -stdmethods=false, the problem is that this also disables stdmethods check for things we do not want to suppress (for example, if there's a bug in Read, the bug will not be caught by go vet -stdmethods=false).

I also disagree with the title change. This is a false positive with any stdmethods functions, not just ReadByte/WriteByte. It's just that I have this example that's ReadByte/WriteByte.

@timothy-king timothy-king changed the title cmd/vet: False positive on ReadByte/WriteByte in stdmethods cmd/vet: False positive on stdmethods check Apr 20, 2022
@timothy-king
Copy link
Contributor

I have partially reverted the title.

If none of the suggested paths forward is appealing, I think the next step is to put forward a proposal for a suppression mechanism for this checker. A community contribution would be welcomed here.

@guodongli-google
Copy link

As @timothy-king mentioned, the suggested mechanism may be insufficient since the interface declaration alone

type TProtocol interface {
	WriteByte(ctx context.Context, value int8) error
	ReadByte(ctx context.Context) (value int8, err error)
}

will trigger the warnings:

./t1.go:11:2: method WriteByte(ctx context.Context, value int8) error should have signature WriteByte(byte) error
./t1.go:12:2: method ReadByte(ctx context.Context) (value int8, err error) should have signature ReadByte() (byte, error)

Basically checker stdmethods disallows reusing these method names with different signatures. On one hand, it is too strong since it uses only the method names to do matching. On the other hand, it is guessing that people are using these methods in a standard manner, and it is right in most cases.

Instead of introducing a complicated suppression mechanism, I would suggest a simple one that allows people to define the subset of method names that should be checked or skipped checking, as checker printf does.

@fishy
Copy link
Author

fishy commented Apr 25, 2022

I think the anonymous interface way as proposed by @timothy-king should work? e.g.

var _ interface{WriteByte(ctx context.Context, value int8) error} = TProtocol(nil)

but adding a flag to stdmethods as similar to the printf one allowing removing some of the names from the check would also be good enough for our case. I do prefer it to be a denylist rather than allowlist (as the printf case) though, as this way if we have other names added to stdmethods in the future we don't need to add that to the overridden list.

@zpavlinovic
Copy link
Contributor

I think the anonymous interface way as proposed by @timothy-king should work? e.g.

var _ interface{WriteByte(ctx context.Context, value int8) error} = TProtocol(nil)

I guess @guodongli-google point is that the current implementation of the checker would still complain as there is a definition of interface TProtocol and the checker analyzes such definitions too.

but adding a flag to stdmethods as similar to the printf one allowing removing some of the names from the check would also be good enough for our case. I do prefer it to be a denylist rather than allowlist (as the printf case) though, as this way if we have other names added to stdmethods in the future we don't need to add that to the overridden list.

I agree a deny list seems a better fit here as the checker already has a predefined set of methods+signatures it looks at. Either way, this will likely require a proposal IMO.

@fishy
Copy link
Author

fishy commented May 13, 2022

yea I will write a proposal when I get time 😅

@seankhliao seankhliao added this to the Unplanned milestone Aug 20, 2022
@fishy
Copy link
Author

fishy commented Jan 24, 2023

Following the guidance of https://go.dev/s/proposal-process, I'm adding proposal label to this issue to turn it into a proposal.

From the discussions above, the proposal is to:

Add a command line flag to go vet with stdmethods check, let's call it stdmethods.suppress for now (open to better names, obviously), which should work as a denylist for stdmethods to ignore.

As a concrete example, for Apache Thrift go library, we should be able to run go vet -stdmethods.suppress="ReadByte,WriteByte" ./... to still do all other stdmethods checks except ReadByte and WriteByte.

A few (half) open questions/rationales:

  1. denylist vs. allowlist

I think a denylist works much better here, as it allows users to suppress certain functions they know stdmethods will complain about otherwise (e.g. they know they are implementing some other interfaces with different expectations). If we use an allowlist instead, users are subject to add new methods to the allowlist when future version of go vet adds checks for more methods.

  1. command line flag vs. suppression from the code

I initially tried to write this proposal with suppression from the code approach, as suggested in #52445 (comment), but as I wrote that proposal I realized that there are too many corner cases and potential breaking changes in that approach to make it far less feasible than the flag approach. For example, currently stdmethods will also complain unexpected function signatures in anonymous interfaces, so this needs to be changed (at least we need to make stdmethods to stop complaining on anonymous interfaces when the anonymous interface is used in a blank variable declaration case).

@fishy
Copy link
Author

fishy commented Jan 24, 2023

oh wait actually I don't have the permissions to add proposal label 🤦

@ianlancetaylor ianlancetaylor changed the title cmd/vet: False positive on stdmethods check proposal: cmd/vet: add new flag to support stdmethod checks for specific methods Jan 24, 2023
@ianlancetaylor ianlancetaylor added Proposal and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jan 24, 2023
@ianlancetaylor ianlancetaylor modified the milestones: Unplanned, Proposal Jan 24, 2023
@ianlancetaylor
Copy link
Contributor

I turned this issue into a proposal. Thanks.

@adonovan
Copy link
Member

adonovan commented Mar 15, 2023

In the case of vet's printf checker we have been able to avoid the need for external configuration files by using Go declarations themselves to enable or disable the checker (with a comment to make the intent clear). This has several advantages:

  • it keeps the actual logic and the checker configuration in the same place;
  • it causes the checker configuration to be subject to type-checking by the compiler;
  • it localizes the effect of changes on the dependency graph (by contrast, using an external configuration file would demand a complete "vet rebuild" after any change to it). This is important for performance of 'go vet' and gopls, which are both structured as incremental build systems.

I wonder whether a similar approach could be made to work here. In other words, is there some Go syntax we could add to the declaration of TProtocol that would have no effect at runtime but would indicate to the stdmethods analyzer (suitably modified) that TProtocol is exempt from the check. Here's a (very weak) starting point for illustration:

// This declaration tells vet's stdmethods checker to disable
// its usual checks for each of the parameter types.
func stdmethodsSuppress(TProtocol, SomeOtherType) {}

I imagine a number of other vet checks might want similar ways of expressing a set of types, or functions, or whatever, so we should examine them holistically before committing to any particular syntax.

See also #58340 (comment)

@gopherbot
Copy link

Change https://go.dev/cl/489835 mentions this issue: go/analysis/passes/unusedresult: support annotations

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Analysis Issues related to static analysis (vet, x/tools/go/analysis) Proposal
Projects
Status: Incoming
Development

No branches or pull requests

9 participants