Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: x/tools/go/analysis: export internal/analysisflags and internal/checker #53336

Open
Antonboom opened this issue Jun 12, 2022 · 11 comments

Comments

@Antonboom
Copy link

Antonboom commented Jun 12, 2022

Hello!

I spent a couple of days, but I did not find an elegant way to implement this:

package main

import (
    "flag"
    "log"
    "os"

    "golang.org/x/tools/go/analysis/singlechecker"

    "github.com/Antonboom/myanalyzer/pkg/analyzer"
    "github.com/Antonboom/myanalyzer/pkg/config"
)

var (
    configPath = flag.String("config", "", "path to config file (yml)")
    dumpCfg    = flag.Bool("dump-config", false, "dump config example (yml) in stdout")
)

func main() {
    flag.Parse()

    if *dumpCfg {
        mustNil(config.Dump(config.Default, os.Stdout))
        return
    }

    cfg := config.Default
    if *configPath != "" {
        var err error
        cfg, err = config.ParseFromFile(*configPath)
        mustNil(err)
        mustNil(config.Validate(cfg))
    }

    singlechecker.Main(analyzer.New(cfg)) // <-- Own flags logic inside via internal packages.
}

Inputs:

  1. It is assumed that the analyzer is used both as a module and as a binary.
  2. I want the linter to be configured using a config file (or via config struct if used as module).
  3. I don't want to lose the set of flags and functionality from the golang.org/x/tools/go/analysis/singlechecker.

I found a workaround using analysis.Analyzer.Flags and flag.Value implementation interface for config.Config, but this leads to singletons, package-level hacks, etc.

Now I see that a lot of linters that are configured through the configuration file refuse to use the golang.org/x/tools/go/analysis/singlechecker(multichecker).

@gopherbot gopherbot added this to the Proposal milestone Jun 12, 2022
@ianlancetaylor
Copy link
Contributor

CC @taking @adonovan

@ianlancetaylor ianlancetaylor added this to Incoming in Proposals (old) Jun 12, 2022
@timothy-king
Copy link
Contributor

@Antonboom I do not yet understand quite a bit of the proposal. Is what is being proposed the title, "export internal/analysisflags and internal/checker"? To cut to the chase, we are unlikely to expose these as is. We may expose specific APIs. The proposal should say exactly what the exposed API should be to be accepted.

I also do not understand the problem you are experiencing very well. What I do understand is that: 1) you want to create a *Analyzer from a config object. 2) The config file should be loadable via a file.

I do now know what these mean or what you are referring to:

I don't want to lose the set of flags and functionality from the golang.org/x/tools/go/analysis/singlechecker.

I found a workaround using analysis.Analyzer.Flags and flag.Value implementation interface for config.Config, but this leads to singletons, package-level hacks, etc.

Now I see that a lot of linters that are configured through the configuration file refuse to use the golang.org/x/tools/go/analysis/singlechecker(multichecker).

Can you please provide more context? Thanks.

@Antonboom
Copy link
Author

  1. singlechecker uses internal packages for own flags registration and parsing.
  2. Logic of singlechecker flags is assumed that the analyzer can be constructed without the use of flags.
  3. You can't add your "top level" flags and use it before singlechecker.Main without conflicting with singlechecker.
singlechecker.Main
package singlechecker

import (
    // ...
    "golang.org/x/tools/go/analysis"
    "golang.org/x/tools/go/analysis/internal/analysisflags" // INTERNAL
    "golang.org/x/tools/go/analysis/internal/checker"       // INTERNAL
    "golang.org/x/tools/go/analysis/unitchecker"
)

// Main is the main function for a checker command for a single analysis.
func Main(a *analysis.Analyzer) {
    // ...
    if err := analysis.Validate(analyzers); err != nil {
        log.Fatal(err)
    }

    checker.RegisterFlags() // <- Useful logic.

    flag.Usage = func() {
        // ...
    }

    analyzers = analysisflags.Parse(analyzers, false) // <- Useful logic.

    args := flag.Args() // Not for all linters is useful.
    if len(args) == 0 {
        flag.Usage()
        os.Exit(1)
    }

    if len(args) == 1 && strings.HasSuffix(args[0], ".cfg") { // Not for all linters is useful.
        unitchecker.Run(args[0], analyzers)
        panic("unreachable")
    }

    os.Exit(checker.Run(args, analyzers))
}

How it may be in my case:

package main

import (
    // ...
    "golang.org/x/tools/go/analysis"
    "golang.org/x/tools/go/analysis/checker" // NOT INTERNAL
    "golang.org/x/tools/go/analysis/singlechecker"

    "github.com/Antonboom/myanalyzer/pkg/analyzer"
    "github.com/Antonboom/myanalyzer/pkg/config"
)

var (
    configPath = flag.String("config", "", "path to config file (yml)")
    dumpCfg    = flag.Bool("dump-config", false, "dump config example (yml) in stdout")
)

func main() {
    checker.RegisterFlags()
    flag.Parse()

    if *dumpCfg {
        mustNil(config.Dump(config.Default, os.Stdout))
        return
    }

    cfg := config.Default
    if *configPath != "" {
        var err error
        cfg, err = config.ParseFromFile(*configPath)
        mustNil(err)
        mustNil(config.Validate(cfg))
    }

    a := analyzer.New(cfg)
    analyzers := []*analysis.Analyzer{a}

    err := analysis.Validate(analyzers)
    mustNil(err)
    
    os.Exit(checker.Run(flag.Args(), analyzers))
}

@timothy-king
Copy link
Contributor

IIUC you want to be able to process flags before calling singlechecker.Main(...) in order to create an analyzer based on the flags. This is doable via package level variables at the moment, but IIUC you find this a bit unwieldy. This seems fairly reasonable to me as a feature request.

Important details still need to be worked out in the proposal. It is not yet clear how this would interact with flags from the checkers. It is also not yet clear how such flags will be forwarded to unitchecker. Some level of glue code seems necessary to register and forward these.

@timothy-king timothy-king added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jun 13, 2022
@adonovan
Copy link
Member

This issue is closely related to #53215.

I think it would be fine for single- and multi-checker to expose any configuration option accepted by go/packages in their command-line interfaces, but those two functions are not intended for use other than as the entirety of the main function.

There is clearly a need for a set of finer grained components for constructing go/packages-based analysis drivers, something that cannot be done today without forking or otherwise using internal/checker. We should expose the Load, Validate, Analyze, and Print operations in some form, though care may be required to help clients maintain the necessary invariants between Load and Analyze. I plan to put together a sketch this week.

@Antonboom
Copy link
Author

I plan to put together a sketch this week.

Thank u!

Important details still need to be worked out in the proposal. Some level of glue code seems necessary to register and forward these.

I suggest waiting for @adonovan.

@gopherbot
Copy link

Change https://go.dev/cl/411907 mentions this issue: go/analysis/checker: a go/packages-based driver library

@Antonboom
Copy link
Author

@timothy-king, can u remove "waiting for info" badge please?
Or you still have any questions?

@hyangah hyangah removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jul 14, 2022
SilverRainZ pushed a commit to SilverRainZ/tools that referenced this issue Aug 24, 2022
The {single,multi}checker packages provide the main function
for a complete application, as a black box. Many users want
the ability to customize the analyzer behavior with additional
logic, as described in the attached issues.

This change creates a new package, go/analysis/checker, that
exposes an Analyze pure function---one that avoids global flags,
os.Exit, logging, profiling, and other side effects---that
runs a set of analyzers on a set of packages loaded (by the
client) using go/packages, and presents the graph of results
in a form that allows postprocessing.

This is just a sketch. API feedback welcome.

DO NOT SUBMIT

Updates golang/go#30231
Updates golang/go#30219
Updates golang/go#31007
Updates golang/go#31897
Updates golang/go#50265
Updates golang/go#53215
Updates golang/go#53336

Change-Id: I745d319a587dca506564a4624b52a7f1eb5f4751
@Antonboom
Copy link
Author

Hello, guys :)

Any updates after 10 months?

@adonovan
Copy link
Member

Any updates after 10 months?

Not really, sorry. I prepared a CL https://go.dev/cl/411907, but it served to highlight that there are quite a number of different ways we could design this (as there are quite a number of tasks involved in an analysis driver) and I didn't and still don't have have a good sense of exactly how much control and flexibility it was necessary to expose in a package of reusable components for writers of analysis drivers.

I'd like to look into this and a number of related analysis issues later this year once our gopls work becomes less intense.

SilverRainZ pushed a commit to SilverRainZ/tools that referenced this issue Apr 10, 2023
The {single,multi}checker packages provide the main function
for a complete application, as a black box. Many users want
the ability to customize the analyzer behavior with additional
logic, as described in the attached issues.

This change creates a new package, go/analysis/checker, that
exposes an Analyze pure function---one that avoids global flags,
os.Exit, logging, profiling, and other side effects---that
runs a set of analyzers on a set of packages loaded (by the
client) using go/packages, and presents the graph of results
in a form that allows postprocessing.

This is just a sketch. API feedback welcome.

DO NOT SUBMIT

Updates golang/go#30231
Updates golang/go#30219
Updates golang/go#31007
Updates golang/go#31897
Updates golang/go#50265
Updates golang/go#53215
Updates golang/go#53336

Change-Id: I745d319a587dca506564a4624b52a7f1eb5f4751
SilverRainZ pushed a commit to SilverRainZ/tools that referenced this issue Apr 11, 2023
The {single,multi}checker packages provide the main function
for a complete application, as a black box. Many users want
the ability to customize the analyzer behavior with additional
logic, as described in the attached issues.

This change creates a new package, go/analysis/checker, that
exposes an Analyze pure function---one that avoids global flags,
os.Exit, logging, profiling, and other side effects---that
runs a set of analyzers on a set of packages loaded (by the
client) using go/packages, and presents the graph of results
in a form that allows postprocessing.

This is just a sketch. API feedback welcome.

DO NOT SUBMIT

Updates golang/go#30231
Updates golang/go#30219
Updates golang/go#31007
Updates golang/go#31897
Updates golang/go#50265
Updates golang/go#53215
Updates golang/go#53336

Change-Id: I745d319a587dca506564a4624b52a7f1eb5f4751
@adonovan
Copy link
Member

I recently brought https://go.dev/cl/411907 up to date and filled in the various gaps. Please take a look at the CL, and if you have any feedback or suggestions about the new API, please comment on the CL or proposal (#61324).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Incoming
Development

No branches or pull requests

6 participants