Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools: go/analysis/passes/printf flags race condition #66887

Closed
Antonboom opened this issue Apr 18, 2024 · 5 comments
Closed

x/tools: go/analysis/passes/printf flags race condition #66887

Antonboom opened this issue Apr 18, 2024 · 5 comments
Labels
Tools This label describes issues relating to any tools in the x/tools repository.
Milestone

Comments

@Antonboom
Copy link

Go version

go version go1.22.0 darwin/arm64

Output of go env in your module/workspace:

GO111MODULE='on'
GOARCH='arm64'
GOBIN='/Users/anthony/golang_workspace/bin'
GOCACHE='/Users/anthony/Library/Caches/go-build'
GOENV='/Users/anthony/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/anthony/golang_workspace/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/anthony/golang_workspace'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.22.0'
GCCGO='gccgo'
AR='ar'
CC='clang'
CXX='clang++'
CGO_ENABLED='1'
GOMOD='/Users/anthony/go-analysis-printf-race/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/6t/v80c8sfs5zqf38b2yhzq592h0000gn/T/go-build3485128818=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

I want to run passes.printf analyzer and reuse his diagnostics in my own analyzer and merge it with other my own diagnostics related to specific library.

What did you see happen?

But I faced with race condition, more details here:
https://github.com/Antonboom/go-analysis-printf-race

What did you expect to see?

I guess I'm using the analyzer in a way it wasn't intended. And in fact the problem is deeper than the race condition around the flags. But I hope you can help me find a solution, thanks! 🙏

P.S. Technically, the analyzers are started one after another (but in a nested form), so it’s not clear what race detector doesn’t like

@gopherbot gopherbot added the Tools This label describes issues relating to any tools in the x/tools repository. label Apr 18, 2024
@gopherbot gopherbot added this to the Unreleased milestone Apr 18, 2024
@timothy-king
Copy link
Contributor

Unlike many projects, the Go project does not use GitHub Issues for general discussion or asking questions. GitHub Issues are used for tracking bugs and proposals only.

For questions please refer to https://github.com/golang/go/wiki/Questions

More details:

I guess I'm using the analyzer in a way it wasn't intended. And in fact the problem is deeper than the race condition around the flags. But I hope you can help me find a solution, thanks! 🙏

https://github.com/Antonboom/go-analysis-printf-race is going outside of what is supported. I think gophers.slack.com is a better venue for this question. #tools and #static-analysis are both good.

P.S. Technically, the analyzers are started one after another (but in a nested form), so it’s not clear what race detector doesn’t like

This analyzer has facts so singlechecker.Main runs the analyzer on transitive imports too. Any package with realistic imports is not totally ordered and creates actions that can run in parallel. Example: package a transitively imports b and c where b and c do not transitively import each other. Then passes (go-analysis-printf-race/analyzer.Analyzer, b) and (go-analysis-printf-race/analyzer.Analyzer, c) can be scheduled in parallel. anlzr.Flags.Set is just setting a global (and setting it too late) within parallel Run cases. Hence the race.

@Antonboom
Copy link
Author

@timothy-king thank you for quick reply

I made an issue because wanted to highlight some area for improvements.

And now I try to understand – I need to find a workaround (if it is technically possible at all) on my own or I can hope for some changes in printf / Run.

I think the feature like "run analyzer inside analyzer" is useful and see two ways:

  1. Implement ad-hoc patch for my case (e.g. make stringSet is thread safe). I can do it.
  2. Open proposal for "analyzer inside analyzer" feature.

?

@timothy-king
Copy link
Contributor

Open proposal for "analyzer inside analyzer" feature.

You can file this proposal, but keep in mind that to be accepted it needs to (1) work with unitchecker (e.g. a process per package) (2) communicate facts correctly to its driver, and (3) not be redundant with communicating via Results.

My recommendation is to take a step back think about the graph of Passes and how a driver of the analysis framework is required to execute these. See
https://pkg.go.dev/golang.org/x/tools/go/analysis#hdr-Pass and
https://pkg.go.dev/golang.org/x/tools/go/analysis#hdr-Modular_analysis_with_Facts .

IMO what you are hoping for is a dependency in this graph that is a leaf and all passes of an analyzer have as a dependency. Setting flags is the supported path for such leaves at the moment. It would be nice to have better alternatives for this and proposals for this are welcomed.

Implement ad-hoc patch for my case (e.g. make stringSet is thread safe). I can do it.

You can, but setting a global immediately before each call is kinda hacky. A moderate improvement is to do this once func init() { if err := anlzr.Flags.Set("funcs", "logger.Printf,logger.Fprintf"); err != nil { panic(err) } }. Also hacky. It is not terribly compatible with another user of printf.Analyzer nor adjusting the flags via go vet -vettool=.... Maybe a few other things.

You can also file a proposal to add a function that returns a new *Analyzer object for printf based on a given configuration. That will be easier to get correct.

@gopherbot
Copy link

Change https://go.dev/cl/580555 mentions this issue: analysis/passes/printf: introduce analyzer constructor

@Antonboom
Copy link
Author

Antonboom commented Apr 20, 2024

You can also file a proposal to add a function that returns a new *Analyzer object for printf based on a given configuration. That will be easier to get correct.

This is exactly what I do in my analyzers but completely forgot about it, affected by singleton nature of std passes 🤦
PR raised. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Development

No branches or pull requests

3 participants