Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: should Go version be part of the go.mod specification? #23969

Closed
robpike opened this issue Feb 21, 2018 · 28 comments
Closed

cmd/go: should Go version be part of the go.mod specification? #23969

robpike opened this issue Feb 21, 2018 · 28 comments

Comments

@robpike
Copy link
Contributor

robpike commented Feb 21, 2018

If not, why not?

@gopherbot gopherbot added this to the Unreleased milestone Feb 21, 2018
@bradfitz bradfitz modified the milestones: Unreleased, vgo Feb 21, 2018
@neild
Copy link
Contributor

neild commented Feb 21, 2018

A module may contain many packages, each of which may contain many files. Specifying the version in go.mod forces all packages and files within a module to use a consistent version.

I'm not sure if that's an argument for or against.

@dsnet
Copy link
Member

dsnet commented Feb 21, 2018

Suppose the go.mod file specifies "go1.10", does that only mean the language as specified in Go1.10 or does that include other stuff?

A tangential question is whether the standard library will still be versioned along with the language specification? Clearly some packages like reflect must be versioned along with the language spec, but it's not clear to me that compress/flate must be versioned along with the language spec and compiler. Nor is this always desirable.

@spenczar
Copy link
Contributor

I don't understand what benefit this would add over build tags. What would this be used for?

@ericlagergren
Copy link
Contributor

Like @dsnet said, it depends what you mean.

From bug reports, I've noticed plenty of folks run older Go versions (> 1 version behind the current version) which causes issues when the package uses newer Go features. Build tags work, but things get hairy and you end up with little wrapper functions for the newer features—it all feels very un-Go-like. Personally, I've resorted to only supporting the most current Go version. But that isn't super user-friendly because sometimes they have legitimate reasons for staying on previous versions. It'd be nice to force things to be pinned to some minimum Go version, even if it's less than the current version.

@holygeek
Copy link

It should, but make it optional so that people are not forced to make a choice when they don't know better. Having Go version be part of go.mod would help give better error message and more importantly, save time for people who run "go get example.com/foo/bar" while running an older version of vgo-capable Go - "go get" can fail earlier rather than later.

@andrewchambers
Copy link

As a point of reference clojure leiningen specifies the language/stdlib version in the build tool.

@drewblas
Copy link

The go.mod has already implemented the concept of "This applies to my project, but dependencies can't introduce a restriction on my project" with the exclude command. The same could be applied here: A go-version command only applies to how my team builds our project. The go-version specified in any dependencies would be ignored.

@andrewchambers
Copy link

andrewchambers commented Feb 21, 2018

@drewblas there are packages that cannot be built with early go stdlibs, the only distinction between the stdlib/compiler and other libs is one is harder to fetch dynamically, conceptually they seem the same and the stdlib/compiler is just another minimal version dependency.

@jimmyfrasche
Copy link
Member

Ignoring Go2 for a moment, there are three cases that I can think of for specifying a minimum version of Go

  1. Relies on a language change introduced in that version
  2. Relies on a package or func/type/etc introduced in that version
  3. Relies on a behavior introduced in that version (For Go 1.10 this could be the funcs in bytes setting the cap or the change in runtime's os thread locking, for some examples)

(Aside: 1 and 2 could largely be determined mechanically, though the go version build tags would complicate that somewhat)

I don't see any point in distinguishing between these cases because they all come bundled together regardless.

In all of these cases, because of the compatibility guarantees of Go, it's not important which version of Go is used, as long as it's greater than or equal to the one specified. Unlike other requirements, if it's unspecified, it's implicitly Go 1.0.0.

If vgo sees that module X requires Go 1.15 but it's being run on Go 1.14 it would just have to stop and say "requires Go 1.15".

If you're trying to use a module, the build could just short circuit, let you know you need at least such and such version of Go, and save you some time.

If you're developing a module and making sure it works with newer versions of dependencies it could skip any that require newer versions of Go, let you know you need at least such and such version of Go to continue testing, and save you some time.

Having this could help with the transition to Go 2 because modules could explicitly declare that they're 2.0.0+ only.

I think it should be included but it would mean something different than other requires (perhaps it should have its own syntax to reflect this).

@spenczar
Copy link
Contributor

spenczar commented Feb 22, 2018

I am still unsure what this would solve that isn't solved with build tags.

@jimmyfrasche, you listed three cases: language changes, new objects, and behavior changes.

In the first two cases, the code will not compile. vgo doesn't need to warn you that the code won't compile. That information would be redundant.

In the third case, build tags would allow a package to require a particular version. If lower versions need to be forbidden, that can be accomplished with tagged file that deliberately fails to compile:

//+build !go1.10

package foo

// If you are seeing a compiler error here, please upgrade to Go 1.10 before
// using foo. It relies on the changes to Barbaz introduced in that version.
This package requires go1.10.

To my eyes, the only benefit is that it would preempt a compiler error with a more descriptive error message if an imported package relied upon newer Go features. Is that all?

@jimmyfrasche
Copy link
Member

@spenczar

To my eyes, the only benefit is that it would preempt a compiler error with a more descriptive error message if an imported package relied upon newer Go features. Is that all?

Yes, but that's not nothing. The benefit of having it in one well defined place is good for humans and computers.

If you're looking a module you can see at a glance what the requirements are (and vgo could compute the transitive requirements so that even if all your code only requires Go 1.5+ but something you import requires Go 1.15+ it would list Go 1.15).

Also, imagine it's 10 years from now and for whatever reason you're stuck with Go 2.1.3 at work. You need a package for Oauth4 so you search for it on godoc.org. There are many results but you're going to have to download them one by one just to see if they compile. If the minimum transitive Go version is part of the machine-readable module specification, godoc.org could let you include that constraint in the search and filter out the results that definitely won't work.

@jimmyfrasche
Copy link
Member

Also writing "require go 1.11" or what have you is just a lot easier than making a file like that so it's more likely people will document the constraint, especially if there's a tool that automatically derives it in cases 1 and 2 so they'd only ever need to think about it in case 3.

@spenczar
Copy link
Contributor

@jimmyfrasche Okay, I'm glad we've clarified, then, that this is about ease of use, not about missing capabilities. Of course, you are right that its not nothing.

I still lean against this proposal though, just on the generic grounds that it expands the spec for go.mod files without enough benefit. It requires special syntax and handling, and makes me wonder why the file doesn't also specify things GOOS and GOARCH constraints, like "this only works for linux" or "this only works for amd64."

@Merovius
Copy link
Contributor

@spenczar There is a thing that this would enable, that build-tags don't, which is verifiable/reproducible builds.

What I kind of dislike about this is how it interacts with MVS (which is kind of hypocritical, given that the same arguments also apply to anything else). If a module resolves its toolchain version to go1.4, but I have go1.10 installed, this would seem to imply that we should use go1.4. But I really don't want to use go1.4 to build anything anymore :) But if we drop/circumvent MVS for the Go version itself, we also loose almost any benefit of specifying it at all (except maybe better error messages).

@4ad
Copy link
Member

4ad commented Feb 23, 2018

I don't see any reason why the Go standard library (and implicitly language version) should be treated any different than any other package when it comes to versioning.

@noonien
Copy link

noonien commented Feb 23, 2018

This would indeed help with reproducible builds, akin to what Bazel does. However, when does one stop? Will we have to also declare the gcc/clang version used to compile cgo packages?

@Merovius raises a valid point. Will the user be required to have and use an older version of the stdlib/language, possibly with performance/security repercussion, due to the package management?

@rogpeppe
Copy link
Contributor

Another thought is that if you admit the standard library (not necessarily the compiler), then it becomes possible to replace standard library packages with local variants. I often wish to temporarily modify code in the standard library for debugging purposes. Editing the code directly in $GOROOT feels a little unclean when you could potentially do something like:

replace "stdlib" v1.10.0 => "../my-hackable-stdlib-copy"

The runtime implications for that might be a bit too far reaching though. Another possibility is to split the stdlib itself into multiple modules, allowing, for example, replacement of the "encoding" module without replacing runtime too.

@4ad
Copy link
Member

4ad commented Feb 25, 2018

What @rogpeppe described would also help the dynamic linking story, especially when it comes to packaging Go binaries in distributions. (Not that I want or care about dynamic linking, just making an observation.)

@pciet
Copy link
Contributor

pciet commented Feb 26, 2018

No. The Go version is a problem here:

  • an older version of Go needs to be supported by a maintained dependency.
  • an older version of Go shows a compile error with an unmaintained dependency.

For the first case build tags should be used, for the second case a fork should be made. Otherwise we don’t need to know the Go version.

@4ad
Copy link
Member

4ad commented Feb 27, 2018

It could be argued that go.1x build tags were a mistake, something that was invented because we didn't have explicit modules and module versions back then, and that versioned modules are a better solution for that problem.

Obviously we can't remove go.1x build tags in Go 1, but maybe we can remove them in Go 2, in which case it would be better to start supporting the replacement now, to ease eventual transition.

@Merovius
Copy link
Contributor

The build tags functionality is a superset of module versioning. It allows you to provide different implementations of the same API depending on what Go version you are running on. I disagree that they're a mistake.

@j7b
Copy link

j7b commented Feb 27, 2018

Go version is mostly significant to a module in terms of the public types and identifiers of the standard library for a given release, so my reasoning lead to the conclusion most of the standard library should actually be a set of modules and modules should support GOOS/GOARCH metadata, perhaps in the form of build constraint blocks.

Edit: I'm now at the conclusion modules need to support the full set of build constraints to express differences in dependencies that can be expressed currently with build constraints which I think implies go_*.mod

@sbuss
Copy link

sbuss commented May 22, 2018

The Go-on-App-Engine team would like to express support for including go version in go.mod. Here's a real-world example of how it is useful:

Currently we use app.yaml's runtime and api_version fields to specify the version of Go the user wants to deploy. We use go1 to refer to the latest version of Go available on App Engine, and go1.x to pin apps to particular versions of Go.

We prefer to use Go-idiomatic patterns where possible, and adding support for a minimum-go-version to vgo would allow us to use that.

We support the addition of a minimum version of Go in go.mod, the same way that all other version dependencies are defined. We would then use that version for all deployments until the user changes it. If a user omits the version from go.mod we would use the latest version of Go available on our platform. This would bring fully reproducible builds to App Engine.

@leighmcculloch
Copy link
Contributor

leighmcculloch commented May 22, 2018

The addition of the go version seems compelling to me because:

  1. It's declaring something that many projects are needing to declare in other places anyway:

    • In README's
    • In CI configs e.g. .travis.yml
    • In deployment configs e.g. AppEngine config
  2. It makes clear some confusing situations. For example, when trying to compile a package on Go1.9 that uses new APIs in Go1.10, Go displays errors that point to there being something wrong with the code when the action the user most likely needs to take is update their environment.


If the version is added, I think the Go tool should insert the value automatically, but it's less clear to me what value it should insert. Should it use the current version of the Go tool? Or will that cause packages to be unnecessarily unavailable to users of older versions of Go? Could the tool make an intelligent decision about what version of Go the module requires? Or would that be more effort than it's worth?

@leighmcculloch
Copy link
Contributor

What level of the version would it include? Would it include: major, minor and patch (e.g. Go1.10.2); major, minor (e.g. Go1.10); or, just major (e.g. Go1)? I could see arguments for all:

Major, minor, patch: Allows packages to declare they need a version that has a fix.
Major, minor: Allows packages that use new APIs to declare they need a version with those APIs.
Major: The Go1 compatibility promise.

@icearith
Copy link

It should, as the version of go compiler is part of the prerequisites for building a program successfully. This will somehow confuse users indeed, In case the user use the wrong version of go, vgo should raise a warning when a user use a wrong version of go.

@rsc rsc modified the milestones: vgo, Go1.11 Jul 12, 2018
@rsc rsc added the modules label Jul 12, 2018
@rsc rsc changed the title x/vgo: should Go version be part of the go.mod specification? cmd/go: should Go version be part of the go.mod specification? Jul 12, 2018
@gopherbot
Copy link

Change https://golang.org/cl/125940 mentions this issue: cmd/go: add 'go version' statement in go.mod

@gopherbot
Copy link

Change https://golang.org/cl/164878 mentions this issue: cmd/go: document GoVersion field in Module struct

gopherbot pushed a commit that referenced this issue Mar 6, 2019
The 'go version' statement was added during Go 1.11 development in
CL 125940. That CL added the GoVersion field to modinfo.ModulePublic
struct, but did not document it in cmd/go documentation. This was
consistent with the CL description, which stated "We aren't planning
to use this or advertise it much yet".

CL 147281, applied during Go 1.12 development, was a change to start
adding the 'go version' statement when initializing go.mod. The 'go
version' statement is now being used, and it has been documented in
the Go 1.12 release notes at https://golang.org/doc/go1.12#modules.
It's now due time to documement the GoVersion field in cmd/go as well.

Keep the Error field bottom-most, both because it makes sense not to
place it in the middle of other fields, and for consistency with the
field order in struct Package, where the Error information is located
at the very bottom.

Regenerate alldocs.go by running mkalldocs.sh.

Updates #28221
Updates #23969

Change-Id: Iaf43a0da4f6a2489d861092a1d4e002a532952cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/164878
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@golang golang locked and limited conversation to collaborators Mar 3, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests