Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/vuln/cmd/govulncheck: add quiet output or a cli flag #62315

Open
lmas opened this issue Aug 28, 2023 · 14 comments
Open

x/vuln/cmd/govulncheck: add quiet output or a cli flag #62315

lmas opened this issue Aug 28, 2023 · 14 comments
Assignees
Labels
NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. vulncheck or vulndb Issues for the x/vuln or x/vulndb repo

Comments

@lmas
Copy link

lmas commented Aug 28, 2023

What version of Go are you using (go version)?

$ go version
go version go1.20.7 freebsd/amd64

Does this issue reproduce at the latest version of golang.org/x/vuln?

Yes (govulncheck@v1.0.1 as of this time).

What did you do?

$ govulncheck ./..
Scanning your code and 175 packages across 11 dependent modules for known vulnerabilities...

No vulnerabilities found.

Share feedback at https://go.dev/s/govulncheck-feedback.

What did you expect to see?

No output upon "success", like the other go commands.

Proposal

I would like to have silent output by default (kind of like the other go commands) or at least a CLI flag to silence this verbose "success" message.

@lmas lmas added the vulncheck or vulndb Issues for the x/vuln or x/vulndb repo label Aug 28, 2023
@gopherbot gopherbot modified the milestones: Unreleased, vuln/unplanned Aug 28, 2023
@adonovan
Copy link
Member

There is a -json flag, which causes the tool to emit all its results and errors in a structured form that is easy to parse. But I agree that there ought to be an easier way (e.g. -q) to disable progress messages.

@lmas
Copy link
Author

lmas commented Aug 28, 2023

I would like to put emphasis on no output at all, just like how these tools behaves:

$ go mod tidy

$ go vet ./internal

$ gosec -quiet ./internal

So -json is of no interest in this situation. I was going to mention go get -u too but I noticed just now it shows a oneliner when it upgrades a pkg :)

@joedian joedian added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Aug 28, 2023
@zpavlinovic zpavlinovic self-assigned this Sep 20, 2023
@zpavlinovic
Copy link
Contributor

Out of curiosity, what problems is this causing?

@lmas
Copy link
Author

lmas commented Sep 20, 2023

For me personally it's causing unnecessary output when it's not needed/wanted, for example by spamming logs for automatic runners (wasting space and time needed to shift through the logs or applying work arounds, cutting away the all clear message).

That feedback link at the end? I saw it the first time and would now like to stop seeing it for ever (would be better to place that link in the docs instead).

Sorry for the negative tone but it's kinda annoying having a tool nagging at you every time.

@timothy-king
Copy link
Contributor

timothy-king commented Sep 21, 2023

In fairness to govulncheck, it is not a go command yet nor is it not as mature as the tools you listed. But I think you have a solid point about the output being a bit chatty now that it has matured a bit more.

My disorganized thoughts:

  • It is valuable to let users know that something was scanned. Users give a sequence of package patterns and it may help to let them know that their patterns matched packages. (go vet obviously takes a different apporach.) I'm not really sure about what the right level of detail here is.
  • So I think there is often value in the line: "Scanning your code and 175 packages across 11 dependent modules for known vulnerabilities...". At CLI, this is a decent warning if the tool may take a while. Once this is printed, "No vulnerabilities found" also seems okay to print. It is possible to make printing this conditional, or only if things are taking a long time?
  • Adding a -q to suppress also seems okay, but not being chatty by default also seems reasonable. Maybe chatty dialog should just be hidden behind a --verbose or a -v flag?
  • Another comparison we can make is to go test which can take a long time and prints progress by package.

@adonovan
Copy link
Member

So I think there is often value in the line: "Scanning your code and 175 packages across 11 dependent modules for known vulnerabilities...".

The go build system doesn't tell you how many packages it looked at to do its job, and it can sometimes sit there for tens of seconds for a large build from cold. Why is vulncheck different? Is it much slower? I would still prefer that it remain silent by default, and that progress messages be an option.

Most of our tools follow the UNIX philosophy and stay silent until there is either a problem or a result. We want our users to routinely assume that Go tools do what they're told, so that if you ask it to scan a given set of packages, it will either do it, or tell you why it was impossible (and exit nonzero). Optional progress messages can be enabled by a -v flag. If it's not too late, I suggest we take that approach.

@lmas
Copy link
Author

lmas commented Sep 22, 2023

I appreciate you guys for taking the time to discuss this, thanks!

@timothy-king I think you have some good points, first that it's not a mature tool yet (I keep forgetting it's in /x/), that's fair.
Secondly, I hadn't thought about it taking a long time to run. Now I'm wondering how other tools handle that, besides just printing progress.

About go test, I think it's part of the more active dev tooling where you want it to be chatty? Of course this is highly opinionated, but my reasoning is that it feels like govulncheck belongs to the linter gang instead, where you want it to be silent until an issue was found (most other linters seems to behave this way at least).

@timothy-king
Copy link
Contributor

Now that I have had a bit more time to organize my thoughts, how about the following:

  • Switch govulncheck to being quiet by default.
  • Print a warning [and stop?] if a pattern in the list matches nothing. IMO an empty pattern means a user is expecting something to be scanned and govulncheck cannot scan it. I am not sure how feasible this is for go/packages without a minor performance penalty. (But if so, that seems fixable.) govulncheck is more about no false negatives than vet or linters.
  • Add a -v flag. This can print chatty messages as it goes. It can probably be moderately chattier than today.

Package docs https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck already has a (slightly different) feedback url. I think it would be reasonable to include "https://go.dev/s/govulncheck-feedback" in the usage string for now. So the feedback url can be discovered from the package docs. But I think the tool is still not that mature, and I think it is okay to lean towards making feedback easy.

@zpavlinovic
Copy link
Contributor

As mentioned, govulncheck is still in x and for larger programs it can take quite some time to finish so some progress is likely useful. We'll consider adding the -q flag in the future (we are currently trying to keep the flag set as small as possible).

@lmas
Copy link
Author

lmas commented Dec 13, 2023

Thank you for your time.

@lmas lmas closed this as not planned Won't fix, can't repro, duplicate, stale Dec 13, 2023
@adonovan
Copy link
Member

Reopening; I think Tim's plan three notes above is a good one.

@adonovan adonovan reopened this Dec 13, 2023
@AGWA
Copy link

AGWA commented Mar 20, 2024

A concrete problem caused by govulncheck's chatty output is that I can't easily run govulncheck from cron because it will email me even if it doesn't find any vulnerabilities. I will have to hack around it with some script that buffers the output and discards it if govulncheck's exit code is 0.

I assume running govulncheck from cron will be a common use case so govulncheck should work well with cron by default.

@zpavlinovic
Copy link
Contributor

I don't think that cron or any other integration should rely on the textual output. This is why -format json is here.

We've discussed the quiet option once again and we stay with our decision to not go forward with it at this time for the reasons outlined earlier. We have also recently revamped the textual output to be less chatty. The changes are available in v1.0.4.

@AGWA
Copy link

AGWA commented Mar 20, 2024

v1.0.4 still outputs text when no vulnerabilities are found:

$ govulncheck -mode=binary test
Scanning your binary for known vulnerabilities...

No vulnerabilities found.
$ govulncheck -version                
Go: go1.22.1
Scanner: govulncheck@v1.0.4
DB: https://vuln.go.dev
DB updated: 2024-03-18 17:35:36 +0000 UTC

No vulnerabilities found.

Integrations with govulncheck need not be complex. I would like to simply get an email once a day if any of my Go modules have vulnerabilities. If I were to use -json, I would have to write a bunch of code to format it in a way similar to what govulncheck already does. If govulncheck were silent when successful (in line with other Go commands and the UNIX philosophy), then accomplishing my goal would be as simple as adding a one line cron job that runs govulncheck.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. vulncheck or vulndb Issues for the x/vuln or x/vulndb repo
Projects
None yet
Development

No branches or pull requests

7 participants