Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: Make it easier to embed compiled git SHA into binary #22147

Closed
kevinburke opened this issue Oct 5, 2017 · 10 comments
Closed

cmd/go: Make it easier to embed compiled git SHA into binary #22147

kevinburke opened this issue Oct 5, 2017 · 10 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@kevinburke
Copy link
Contributor

kevinburke commented Oct 5, 2017

If I compile Go from source and run go version, I get output like this:

go version devel +475d92ba4d Thu Oct 5 10:50:18 2017 +0000 darwin/amd64

I would love to get the same output in my programs. Specifically, I'm trying to debug a reported error in a program I wrote and it would be a lot easier if I knew which SHA they were running.

It's easy in the Go build tool because it can specify what code runs during the build process. From findgoversion():

	// Otherwise, use Git.
	// What is the current branch?
	branch := chomp(run(goroot, CheckExit, "git", "rev-parse", "--abbrev-ref", "HEAD"))

	// What are the tags along the current branch?
	tag := "devel"
	precise := false

	// If we're on a release branch, use the closest matching tag
	// that is on the release branch (and not on the master branch).
	if strings.HasPrefix(branch, "release-branch.") {
		tag, precise = branchtag(branch)
	}

	if !precise {
		// Tag does not point at HEAD; add hash and date to version.
		tag += chomp(run(goroot, CheckExit, "git", "log", "-n", "1", "--format=format: +%h %cd", "HEAD"))
	}

I have options for replicating this behavior, none very good:

  • Use linker flags: This blog post describes compiling Go with -ldflags -X in such a way that you can embed the SHA at compile time. https://medium.com/@joshroppo/setting-go-1-5-variables-at-compile-time-for-versioning-5b30a965d33e

    I can do this of course, but I'm not sure it's feasible or possible to ask all of my users to remember to do this when they compile my program. I can add a Make target or something to automate the instructions but that's something they have to remember to do too.

    I could release compiled libraries but that's infeasible in a company environment where tip is changing rapidly and users are used to compiling from source.

  • Tag a new release every time I add a commit to the program. I wrote this tool to do that: github.com/Shyp/bump_version. Generally though it's not practical to tag every single commit, the same reason there aren't tags for every commit between Go 1.8.3 and Go 1.9.0.

It would be nice if there was some API for detecting or automatically embedding the git SHA in the binary so I can access it at runtime, and print it out when users of my program run <mybinary> version or <mybinary> env, the same way the Go team found this information useful and found a way to embed it while the program was being compiled.

@mvdan
Copy link
Member

mvdan commented Oct 5, 2017

I have thought the same in the past, and I agree that the two current workarounds you listed are suboptimal. Let me play the devil's advocate, however:

  • What happens when there is no source control, or when there is but it's not Git?
  • Should this affect all packages, or just main ones?
  • Would this "version hash" be taken into account to know if a package needs rebuilding?
  • Could renaming a tag or removing .git break a reproducible build?

@kevinburke
Copy link
Contributor Author

Here are places where this field would come in handy. I will update this list as I come across more examples.

@ianlancetaylor
Copy link
Contributor

@kevinburke As you say above, you can of course use -ldflags=-X today. But it seems that you aren't happy with that, because you want it to be automatic. But I really don't think we want any such thing to be automatic; as @mvdan says, what if the build doesn't use git? So it has to be under the control of some option--in which case why not just use -X?

Do you have any suggestions for how this could work in practice?

Also, one of your reasons is so that you can know which git revision a user is running. But their git SHA doesn't necessarily mean anything to you, depending on how the sources have moved around. So what you want to know is actually the git SHA that they started from. You could do that by storing the git SHA in your own repo, somehow, though I agree that that too has problems.

Does the new go.mod file used by vgo help in any way here?

@ianlancetaylor ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 29, 2018
@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Mar 29, 2018
@fd0
Copy link

fd0 commented Mar 30, 2018

A small note explaining what we do in practice right now for restic: we're using a small program called build.go. It allows compiling restic from a checkout of the repo and from the extracted release tar.gz without having to setup a GOPATH and only depending on the Go compiler, so it's very end-user compatible. It requires vendoring all dependencies though.

In addition (that's why I'm mentioning it here), it embeds the Git version if available and the contents of the VERSION file in the source into the binary via ldflags, so restic version can print some nice output:

restic 0.8.3 (v0.8.3-109-gd49502a5) compiled with go1.10 on linux/amd64

The first string 0.8.3 is the content of the file VERSION, followed by the output of git describe.

It turned out to be very helpful when users report issues and bugs, we request this information in our issue template.

To everybody having a similar requirements, I encourage you to give build.go a try. Just copy it to your project, configure the section at the top and it should mostly work. To all others: please ignore this comment ;)

@bcmills
Copy link
Contributor

bcmills commented Jan 18, 2019

As of Go 1.12, you can get the versions of all modules compiled into the binary (in module mode) using runtime/debug.BuildInfo.

If the version in use is a pseudo-version, it also includes the first dozen or so characters of the VCS commit ID.

@bcmills bcmills closed this as completed Jan 18, 2019
@fd0
Copy link

fd0 commented Jan 19, 2019

@bcmills how does this work? I've built a small test program, running it with Go 1.12beta2 shows the versions for all dependencies, but not for the main module (when compiled from a checkout of the repo):

$ go version
go version go1.12beta2 linux/amd64

$ unset GOPATH
$ export GO111MODULE=on

$ go build
[...]

$ ./godebugbuildinfo
test program for debug.BuildInfo
path: github.com/fd0/godebugbuildinfo
  main: debug.Module{Path:"github.com/fd0/godebugbuildinfo", Version:"(devel)", Sum:"", Replace:(*debug.Module)(nil)}
  deps:
    &debug.Module{Path:"github.com/fatih/color", Version:"v1.7.0", Sum:"h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys=", Replace:(*debug.Module)(nil)}
    &debug.Module{Path:"github.com/mattn/go-colorable", Version:"v0.0.9", Sum:"h1:UVL0vNpWh04HeJXV0KLcaT7r06gOH2l4OW6ddYRUIY4=", Replace:(*debug.Module)(nil)}
    &debug.Module{Path:"github.com/mattn/go-isatty", Version:"v0.0.4", Sum:"h1:bnP0vzxcAdeI1zdubAl5PjU6zsERjGZb7raWodagDYs=", Replace:(*debug.Module)(nil)}

I can only see (devel) as the version for the main module, even when I'm adding a version tag in the repository (like v1.0.0).

Is there anything I'm missing?

@dominikh
Copy link
Member

I think this is affected by #29228. I'll reopen this issue until that's resolved.

@dominikh dominikh reopened this Jan 19, 2019
@fd0
Copy link

fd0 commented Jan 19, 2019

Thanks!

@mvdan
Copy link
Member

mvdan commented Jan 19, 2019

I'm pretty sure this is now being tracked under #29814, which brings up all the implementation details. I believe @bcmills was trying to consolidate all the older related issues into a single one with a proposed solution.

@dominikh
Copy link
Member

Sounds good enough to me, now that these two issues reference each other.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

7 participants