Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: factor version reading into debug/buildinfo #39301

Closed
skelterjohn opened this issue May 28, 2020 · 10 comments
Closed

cmd/go: factor version reading into debug/buildinfo #39301

skelterjohn opened this issue May 28, 2020 · 10 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@skelterjohn
Copy link
Contributor

With go version 1.14 and go modules, we get a nice report from go version -m <binary>. The code backing this command is not available to use outside the binary, as it uses internal packages.

Supply chain analysis tools, which I work on, would benefit greatly from being able to reliably get this information using a package in the standard library. Instead, I'm copying and lightly modifying the code used to run the go version -m command since shelling out to a binary is not always an option.

@seankhliao
Copy link
Member

To clarify, you're asking for a package to read the BuildInfo from a compiled executable?

@andybons andybons changed the title package to get the runtime/debug.BuildInfo data proposal: package to get the runtime/debug.BuildInfo data May 28, 2020
@gopherbot gopherbot added this to the Proposal milestone May 28, 2020
@skelterjohn
Copy link
Contributor Author

From the bytes of a compiled executable. Something like func ReadBuildInfo(io.Reader) (debug.BuildInfo, error) would be ideal.

@ianlancetaylor ianlancetaylor added this to Incoming in Proposals (old) May 28, 2020
@mvdan
Copy link
Member

mvdan commented May 28, 2020

shelling out to a binary is not always an option

Can you clarify why? If you need to inspect Go binaries, having Go installed seems pretty reasonable.

@dmitshur
Copy link
Contributor

Have you considered the rsc.io/goversion/version package?

@skelterjohn
Copy link
Contributor Author

For why I prefer this to shelling out:

The product I work with is GCP's container image vulnerability scanner. It scans XX thousand container images per minute. Each of those container images has hundreds, thousands of files. I want to find Go binaries and report on how they were made so we can do vulnerability analysis, contributor history, etc. Shelling out to go version -m for each one is impractical for a number of reasons. First, in the execution environment I'm in (google's borg), starting a subprocess is an expensive operation that involves lots of extra logging. Second, the overhead to writing a file and running the go tool is far greater than inspecting the bytes as they fly by in our streaming read. Even if I limit it to executable files, there are still probably hundreds to inspect per image. Every cycle counts at this scale.

For consideration of rsc.io/goversion/version:

I did see that, and it does to approximately what I want. However, it does not have the interface I'd like (give it a file path rather than bytes, which increases the overhead per invocation significantly), and it's also not covered by any compatibility guarantees. For instance, the code it uses to load information from the executable and extract the mod string is different than that used by go version -m.

What I have done is copied, with light modifications to make it fit, the code that is used by go version -m to extract the mod string and the code used by runtime/debug.ReadBuildInfo() to parse it. This will work fine for now, but it implies an ongoing maintenance burden to ensure that it keeps up-to-date with any changes in the executable or mod string format. Since go version -m's code and runtime/debug.ReadBuildInfo()'s code imply that burden is being shouldered by the Go distribution as well, I would like to benefit :)

@jayconrod
Copy link
Contributor

Thanks @skelterjohn for posting this. Vulnerability scanning is a use case we had in mind when embedding module version information in binaries. This would help complete the picture.

I think it would make sense to do this somewhere in x/mod, probably in a new package, since this doesn't fit in any existing package there.

cc @bcmills @FiloSottile @katiehockman @matloob @rsc

@rsc
Copy link
Contributor

rsc commented Jun 3, 2020

We discussed this some on #35667, where we decided it was OK to add go version -json to get a JSON-formatted output, and also OK to move the cmd/go version reader somewhere into x/mod. (Please use the one in cmd/go and not the rsc.io version, which is older.)

So this probably doesn't need to be a proposal, since we've already decided to do it.

@rsc rsc changed the title proposal: package to get the runtime/debug.BuildInfo data cmd/go: factor version reading into x/mod Jun 3, 2020
@rsc rsc removed this from Incoming in Proposals (old) Jun 3, 2020
@rsc rsc added NeedsFix The path to resolution is known, but the work has not been done. and removed Proposal labels Jun 3, 2020
@rsc rsc modified the milestones: Proposal, Backlog Jun 3, 2020
@jayconrod jayconrod modified the milestones: Backlog, Go1.18 Aug 16, 2021
@jayconrod jayconrod self-assigned this Aug 16, 2021
@gopherbot
Copy link

Change https://golang.org/cl/348016 mentions this issue: buildinfo: add package for reading build metadata from Go executables

@gopherbot
Copy link

Change https://golang.org/cl/353887 mentions this issue: runtime/debug: add ReadBuildInfoFrom and ReadBuildInfoFromFile

@bcmills bcmills changed the title cmd/go: factor version reading into x/mod cmd/go: factor version reading into debug/buildinfo Oct 14, 2021
@gopherbot
Copy link

Change https://golang.org/cl/356013 mentions this issue: doc/go1.18: add release notes for build and VCS info

gopherbot pushed a commit that referenced this issue Oct 15, 2021
Fixes #37475
For #39301

Change-Id: I765bc667004931ff8973b947f2e95624a4a0375d
Reviewed-on: https://go-review.googlesource.com/c/go/+/356013
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Trust: Jay Conrod <jayconrod@google.com>
@golang golang locked and limited conversation to collaborators Jun 23, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

7 participants