Proposal

I propose that, if the main module's go.mod file specifies go 1.14 or higher:

• If a module path passed to go get does not have a major-version suffix, and its major version is incompatible with the path, and the corresponding version of the repository does not contain an explicit go.mod file, then the version should be resolved to a pseudo-version with a compatible major version.

  • I note that https://proxy.golang.org/k8s.io/client-go/@v/v2.0.0-alpha.1.info currently resolves the non-canonical version to the corresponding +incompatible version. Under this proposal, it would be updated to instead resolve to the compatible pseudo-version.

• If an existing +incompatible version is found in the main module's go.mod file, it should be similarly rewritten to a canonical pseudo-version.

  • Proxies may continue to serve +incompatible versions, and should not redirect a version requested with an explicit +incompatible suffix to the corresponding pseudo-version.

• If a +incompatible version is found in the requirements of some other module in the build list, that version constraint should be ignored entirely, as if the other module had not required it at all.

  • If the +incompatible module is needed to satisfy an import from some module during a build operation, go mod tidy, or go mod vendor, the version of that module should be re-resolved to latest and the dependency added to the main module's go.mod file.

• When resolving the latest or upgrade version of a module, the go command should first check the latest version of that module tagged with a compatible version.

  • If that version includes a go.mod file, then that version should be used and no further search performed.

  • Otherwise, the highest release at each major version should be checked for a go.mod file. The last such release that lacks a go.mod file should be resolved to a pseudo-version, and that pseudo-version should be used as the latest or upgrade version of the module.

• If a +incompatible version is found in the requirements of some other module in the build list, that version constraint should be ignored entirely, as if the other module had not required it at all.
  • If the +incompatible module is needed to satisfy an import from some module during a build operation, go mod tidy, or go mod vendor, the version of that module should be re-resolved to latest and the dependency added to the main module's go.mod file.

Is this saying that if you are building module foo, which requires module bar, and bar has a go.mod containing require baz v2.3.4+incompatible, then that would be interpreted as require baz/v2 latest?

If so, does that mean the version of baz selected for the build could change (that is, ending up with some later commit than the commit tagged v2.3.4 )?

And if so, what is the brief rationale for that?

Is this saying that if you are building module foo, which requires module bar, and bar has a go.mod containing require baz v2.3.4+incompatible, then that would be interpreted as require baz/v2 latest?

It would be treated as if module bar's go.mod file did not mention baz at all.

In that case, your module (foo) would resolve baz@latest (not baz/v2@latest), and add that dependency to your (foo's) go.mod file, all as usual for a package not found in the existing module graph.

That means that you could indeed end up with a version of baz more recent than v2.3.4, which is exactly what we want if (say) the tags occur in the sequence v1.0.0, v2.0.0 (incompatible), […], v2.3.4 (incompatible), v1.5.0 (with the go.mod file added in between v2.3.4 and v1.5.0).

The rationale is that I don't want to make the MVS step use any notion of “version precedence” that isn't ordinary semantic versioning.

(I especially don't want to have to walk back from v1.5.0 looking for the discontinuity in order to figure out the relative ordering of v1.5.0 and v2.3.4+incompatible, and I don't want to have to re-resolve the pseudo-version corresponding to v2.3.4+incompatible every time I reload the build list.)

In that case, your module (foo) would resolve baz@latest (not baz/v2@latest), and add that dependency to your (foo's) go.mod file, all as usual for a package not found in the existing module graph.

That means that you could indeed end up with a version of baz more recent than v2.3.4, [...]

But would that then mean go.mod is not providing reproducible builds if the versions selected in the build change upon upgrading to Go 1.14? If so, would that be reasonable to do given prior statements around future releases being able to properly consume go.mod files defined by older releases? There might be some shades of grey there, including for example Go 1.13 invalidated some go.mod files that built under Go 1.12, but there is at least an argument to be made that reporting a fatal error for a previously uncaught problem is different than shifting the versions used.

(Sorry for the multiple questions; mainly still trying to see if I understand the proposal).

But would that then mean go.mod is not providing reproducible builds if the versions selected in the build change upon upgrading to Go 1.14?

It would mean that the transition from Go 1.13 to Go 1.14 would not reproduce the same set of dependencies, but since the entire toolchain and standard library changes at each major release, that boundary is not exactly “reproducible” to begin with.

Once you run a go build or go mod tidy using 1.14, then each successive build using 1.14 would produce the same result, just not necessarily the same result as when using 1.13.

The more interesting problem occurs when going back from 1.14 to 1.13, since the version selected (and written to the go.mod file) by 1.14 may be “lower” (in the strict-semver sense) than the version selected by 1.13.

It would mean that the transition from Go 1.13 to Go 1.14 would not reproduce the same set of dependencies, but since the entire toolchain and standard library changes at each major release, that boundary is not exactly “reproducible” to begin with.

I haven't fully considered this yet, but it seems like a significant, breaking change in MVS. This would change builds in a more significant way than toolchain and standard library changes do (those tend to be in Hyrum's law territory).

An obvious consequence: suppose you depend on module A (which has migrated to modules), which depends on module B (which has not) at v2.0.0+incompatible. With this change, the B requirement is ignored, and we get a pseudo-version based on v3.0.0, the newest major version of B without a go.mod file. The B API has been rewritten between major versions, and the upgrade breaks the build. The author of the main module now needs to either replace B or go get B@v2.0.0, despite not being familiar with B or having any explanation of why the build broke.

(of course this can already happen if someone else requires B v3.0.0+incompatible)

@jayconrod, that's an interesting consideration (and somewhat ties in to #34165).

One alternative would be to only apply this proposal if the main module's go.mod file specifies go 1.14 or higher, so that the need to re-resolve versions would be opt-in at that point.

That has the advantage of retaining compatibility with as much of the +incompatible support as actually works today, but the disadvantage of not reducing the complexity (and bugs) induced by that support.

It also has the disadvantage of changing the meaning of a module's requirements depending on which other module is importing it, which makes fixes and patches more difficult to test — but perhaps the advantage of clearly attributing the change outweighs that disadvantage, and at any rate the main module can already change the meaning of a module's requirements using replace directives.

Thinking about it some more, gating on go 1.14 in the main module's go.mod file seems like the clear way to go. Edited the proposal accordingly.

If we're gating on go 1.14, we could treat +incompatible versions the same way we treat branch names (like master): in the main module, they're allowed but resolved to a valid version or pseudo-version; in other modules, they're rejected with an error. I think an error would be preferable to ignoring them.

One problem with this: adding go 1.14 to go.mod does not prevent old versions of the Go command from adding +incompatible versions. So a module could be broken unintentionally in a mixed environment.

@jayconrod, I think we could reasonably treat a +incompatible version in a dependency's go.mod file as an error if that dependency specifies go 1.14 or later. But if the dependency was published at or before go 1.13, that seems like too hard of a break.

At the very least, I think we would need to give the main module the opportunity to to replace those versions with compatible pseudo-versions, but at that point we're back to having the main module specify the dependency anyway — so we may as well just ignore the versions that would need to be replaced.

This seems plausible but I haven't thought through it all carefully.
If you (@bcmills), @jayconrod, @thepudds, and @myitcv
all agree that this is the right thing to do, then I'm willing to try it.

Full disclosure: I'm not currently close enough to the details here to opine, so I'll leave it to others to decide.

FYI @mvdan @rogpeppe given previous conversations

CC @Helcaraxan

@bcmills To help people confirm their understanding of the proposal, what do you think about giving a couple examples of final pseudo-versions? For example, for tags like v2.3.4 and 2.3.5-alpha.1, where neither corresponds to a commit with a go.mod, or something like that?

Is it correct that the final result would translate to a fourth form of pseudo-version being defined, rather than the current three forms:

vX.0.0-yyyymmddhhmmss-abcdefabcdef is used when there is no earlier versioned commit with an appropriate major version before the target commit. (This was originally the only form, so some older go.mod files use this form even for commits that do follow tags.)

vX.Y.Z-pre.0.yyyymmddhhmmss-abcdefabcdef is used when the most recent versioned commit before the target commit is vX.Y.Z-pre.

vX.Y.(Z+1)-0.yyyymmddhhmmss-abcdefabcdef is used when the most recent versioned commit before the target commit is vX.Y.Z.

Or if not a forth form, then maybe it is just a tweak to the current three definitions, but a VCS tag like v2.3.4 getting translated to v2.3.4-yyyymmddhhmmss-abcdefabcdef I think is does not fit into one of those current patterns? (If that is indeed the proposed form?)

Separately, as I understand it, part of the overall intent for the proposal is to help simplify the cmd/go code (cutting down on corner cases and corresponding bugs) as well as making things simpler for users. But if cmd/go will need to forever understand old go.mod files, is there a large gain in simplicity for the cmd/go code? (I can imagine there could be simplicity gains if only reading old modules needs to be supported, but at least wanted to ask the question).

One problem with this: adding go 1.14 to go.mod does not prevent old versions of the Go command from adding +incompatible versions. So a module could be broken unintentionally in a mixed environment.

Somewhat related to prior comment, perhaps this could get rolled out across two releases -- first, in say Go 1.14, just rewrite +incompatible entries for the main module's go.mod. Second, in say Go 1.15, introduce the error conditions. Maybe that doesn't make sense, but maybe that would help in a mixed environment, and give more time for more modules to transition. (I think the rules as outlined mean the common case might be smooth transition for an individual module, but there are a lot of gophers out there, and maybe being slightly more conservative on the roll out might cut down how many people complain about having hit a "rare" error).

Also, if these are new pseudo-versions, is there an opportunity to inject a suggestive word into the pseudo-version itself, so that people have a higher chances of (a) understanding when first encountering these (without ever having read the specific piece of doc that describes them), or at least (b) recognizing them when they see them again later? Or maybe injecting a comment?

Finally, there is some subtlety in the discussion (including it is slightly tricky in some of the discussion to be 100% sure which "major version" is being mentioned), so sorry if some of these questions are off base, or if I have otherwise misunderstood.

@bcmills Also, I think I did not fully understand the last portion of this comment (including whether or not it means Jay's immediately prior suggestion does not work, or if it is only tangentially related to Jay's comment):

At the very least, I think we would need to give the main module the opportunity to to replace those versions with compatible pseudo-versions, but at that point we're back to having the main module specify the dependency anyway — so we may as well just ignore the versions that would need to be replaced.

(It seems like at least part of the issue might be related to how do you specify a replace if the main module and a dependency disagree on how to refer to a version, but I'm not sure I understand the piece about "so we may as well just ignore the versions that would need to be replaced").

Is it correct that the final result would translate to a fourth form of pseudo-version being defined […]?

It would be one of the usual forms. The resolution process is basically:

1. Resolve the incompatible version to a commit hash.
2. Fetch the requested commit and verify that it lacks a go.mod file.
3. Re-resolve the commit to a pseudo-version as if the incompatible-major tags did not exist.

So, for example:

• k8s.io/client-go@v11.0.0 resolves to commit 6ee68ca5fd83.
• That commit has v2.0.0-alpha.0 as its only valid-semver ancestor. v2 is not compatible with an unversioned import path, so that tag would also be excluded as a pseudo-version base.
• Therefore, go list -m k8s.io/client-go@v11.0.0 would resolve to

  k8s.io/client-go v0.0.0-20190313235726-6ee68ca5fd83
  

Or:

• github.com/Azure/go-autorest@v13.0.1 resolves to commit 69b4126ece6b.
• That commit has numerous ancestors, but its last v1 ancestor seems to be v1.1.1.
• Therefore, go list -m github.com/Azure/go-autorest@v13.0.1 would resolve to

  github.com/Azure/go-autorest v1.1.2-0.20190906230412-69b4126ece6b
  

But if cmd/go will need to forever understand old go.mod files, is there a large gain in simplicity for the cmd/go code?

No; the benefit is mainly for users — particularly for maintainers with existing v2+ tags, since this would give them a way to opt-in to modules going forward.

In particular, it would give maintainers a way to opt in without forcing their existing consumers to update import paths, assuming that either those consumers have already adapted to any breaking changes up to the last major version, or the maintainers are willing to go back to the API as it stood at the last v1 release.

In full disclosure, though, there are two possible simplicity gains in cmd/go:

1. We could perhaps avoid fixing some existing bugs on +incompatible paths, since users before Go 1.14 would have those same bugs and users at or after 1.14 could avoid the bugs by upgrading their go directive.

2. We could at some point declare that the main module itself must declare go 1.14 or higher, at which point we would only need to understand how to ignore +incompatible versions (rather than how to interpret them).

perhaps this could get rolled out across two releases

A go 1.13 binary working in a go 1.14 module would already untidy it in many cases (replacing v0 or v1 pseudo-versions with +incompatible versions from transitive dependencies), and at that point there doesn't seem to be a significant advantage to delaying the error-check for go 1.14 modules.

if these are new pseudo-versions, is there an opportunity to inject a suggestive word into the pseudo-version itself

Metadata in the pseudo-version itself would not be backward-compatible to Go 1.13 and earlier, and so would prevent Go 1.13 users from consuming a 1.14 module. That's not out of the question, but seems a high cost in exchange for the marginal value of the metadata.

We could probably inject a comment, at least.

I think I did not fully understand the last portion of this comment […]

If the main module specifies go 1.14 and we retain but disallow +incompatible versions found in transitive go 1.13-or-earlier dependencies, then the main module will need to replace those versions with compatible ones. But if the main module must replace those modules anyway, then we may as well filter them out in the first place — that way, the main module only needs to require the appropriate versions (not replace them), and those version constraints will propagate appropriately as dependencies themselves upgrade to go 1.14.

Some more insights from a discussion with @jayconrod this morning.

Ideally, we would like to do our best to respect +incompatible versions by converting them to compatible pseudo-versions. Provided that the +incompatible versions all have some common v0 or v1 predecessor tag and the semantic tags are mostly applied to chronologically-ordered commits, that tends to produce the same result as running MVS on the original +incompatible versions.

The major problem with that approach is that it the version-to-pseudo-version mapping may be expensive to compute, so we don't want to recompute it every time we resolve dependencies.

However, we could do the version-to-pseudo-version resolution once and record the result in the go.mod file, and then only ignore a +incompatible version found in a transitive dependency if the main module's go.mod file specifies a version for the same module path.

That would at least preserve the selected version at the 1.13-to-1.14 transition point, but wouldn't pick up +incompatible requirements from newly-added dependencies. (But maybe we shouldn't be picking those up anyway, because they're incompatible?)

One other alternative might be to cache the +incompatible-to-pseudo-version mapping in the main module's go.mod file in some other way. For example, we could inject some directive that maps each of the resolved versions to the corresponding compatible version.

One option I'm considering for #26904 is to introduce a form of the replace directive that omits the version, meaning “alias the module path to the resolved version instead of replacing the source code”.

We could similarly overload the replace directive to omit the path, meaning “treat this version as actually referring to this other version, including during version selection”:

replace github.com/Azure/go-autorest v13.0.1 => v1.1.2-0.20190906230412-69b4126ece6b

On the other hand, I don't see much point in overloading the replace keyword at that point — we may as well just introduce a new one. I personally like remap:

remap github.com/Azure/go-autorest v13.0.1 => v1.1.2-0.20190906230412-69b4126ece6b

A remap directive (using either option for syntax) would have the advantage of also allowing module authors to more easily correct other kinds of invalid versions or semantic tagging errors.

It seems to me that from a practical perspective the need to respect the +incompatible versions of transitive dependencies is paramount. Not doing so would be a potential cause for many unexpected breakages. Both for new module adopters or for projects already using modules that add a new dependency with previously unseen +incompatible transitive dependencies.

In practice such breakages might not be too numerous, nor touch the majority of users, but it's yet another place where modules could potentially interfere with the "it just works" principle. The errors could also be very hard to decipher as they would arise in transitive and not direct dependencies of someone's project. Debugging this would then require a developer to learn in one swoop quite a bit about modules, dependency management subtleties, the actual implementation details of their dependencies and even the dependencies of their dependencies. This is not something that people would usually want to do. Hence if we can prevent it we should do so.

On the more practical side I do like the idea of the remap for both proposed syntaxes: version-less and path-less. It could indeed be used to correct a lot of breakages that have been introduced by early adopters without the necessary expertise or appropriate tooling (like described in #31543).

The downside though of remap is that, outside of the use-case discussed here, the burden of correction will fall on downstream users instead of on the offending projects themselves. This means that although it's a very useful tool, it's also a "patch" to fix things that have already gone wrong instead of a tool that adds new possibilities. I'd rather fix the root cause of those errors... but that ship has probably already sailed, at least for early adopters.

A potential middle ground could be to further make cmd/go capable of understanding the most common errors, referring to some of the proposals in #31543 and even extending them. It could pro-actively add more appropriate remap lines to a project's go.mod. One example would be to automatically add a "path" remap for a dependency at a vN version with a go.mod that misses the /vN suffix in the module's path.

All this said... I do very much like the idea of removing the +incompatible as it will help give a more "uniform" appearance, for lack of a better term, to (pseudo-)versions. I remember myself being relatively confused the first few times I encountered them and it took me a few weeks to wrap my head around the subject. In hindsight I would not have needed to worry as much as I had done. Another benefit would thus be to make things less confusing for developers with little dependency management experience who should not need to know about the subtleties of pre-module-version compatibility when using modules themselves.

I thought about this a bit more, and I have some questions and concerns listed below. In general, I'm worried this is a significant change with subtle implications. I don't expect we're going to be able to foresee all potential issues, and I'd like to see if we can solve the problems listed above by doing something narrower and possibly also #24031.

• We should ensure that, given a complete go.mod file in the main module, all versions of the go command should pick the same build list or fail due to new, unrecognized syntax (remap).
  • Non-main go.mod files with newer go statements should be understandable by older versions of the go command. Only module, go, and require statements are read, and we should not change their interpretation.
  • Any differences in chosen versions may break the build. Replacing +incompatible versions with equivalent pseudo-versions is probably okay, but it will be visible in debug.BuildInfo.
• What changes are needed by proxies? I think some buy-in may be needed by proxy maintainers.
  • /@v/list - should +incompatible versions still be advertised? Newer versions of the go command may be able to prune them out.
  • /@v/<version>.info - should a +incompatible version be resolved to a pseudo-version here? Will that break anything in 1.13 or earlier?
  • /@v/<version>.mod, .zip - I think we would still need to serve +incompatible versions to keep older versions of the go command working. Currently, 404 or 410 should be served for non-canonical versions.
  • /@latest - should probably be updated to match the new definition of @latest, but this shouldn't be strictly necessary unless /@v/list is empty.
• When resolving versions in direct mode, do we need to fetch more than we did before?
  • In normal operation, I think we fetch commits for every tagged version, but not the full history. Replacing +incompatible with a pseudo-version would force unshallowing, but that's probably okay.
• Are there situations where different versions of the go command will undo each other's changes? I don't think a pseudo-version would be replaced by the equivalent +incompatible version in 1.13, but please confirm.
• Should the go command replace +incompatible versions without a go 1.14 statement in the main module?
• I'm assuming remap would only apply in the main module, not downstream, but please confirm.
• Will remap require go 1.14?
• Would remap ever be added automatically? This would result in different build lists, but if the module has go 1.14, older versions won't work, so there's no inconsistency. Maybe that's okay.
• Do you anticipate any other ways remap can be abused? What are its limitations with respect to semantic import versioning? Could it be used by authors to bypass semantic import versioning altogether?

I'd like to reiterate the previous commenters' support for keeping resolved versions the same.
For large projects, maintaining the exact versions of dependencies can be tricky. If dependencies change when moving to a new version of Go, that version of Go will be "tainted" by the associated breakage.

For any plan moving forward I think that, as a minimum requirement, modules should continue to build with exactly the same resolved dependencies on at least the last two versions of Go. People need to be able try out the latest version while still having the option to drop back to the previous version.

So anything landing in 1.14 should change the go.mod file in such a way that 1.13 arrives at the same resolved dependencies. I suspect that a phased plan, with the final phase landing in 1.15, might be the best way forward here.

Thanks @jayconrod for bring up the issues related to the module proxy.

What changes are needed by proxies? I think some buy-in may be needed by proxy maintainers.
/@v/list - should +incompatible versions still be advertised? Newer versions of the go command may be able to prune them out.
/@v/.info - should a +incompatible version be resolved to a pseudo-version here? Will that break anything in 1.13 or earlier?
/@v/.mod, .zip - I think we would still need to serve +incompatible versions to keep older versions of the go command working. Currently, 404 or 410 should be served for non-canonical versions.
/@latest - should probably be updated to match the new definition of @latest, but this shouldn't be strictly necessary unless /@v/list is empty.

Proxies that rely on the go command internally (e.g. Go Module Mirror) need to upgrade the newer version that behaves differently with the prior go versions following this proposal. Maybe some users of the proxies may upgraded their go before the proxies, or after the proxies. Then, I fear this behavioral change from the proxy side can add more confusion and complexities.

I feel adding this in 1.14 is too rushed to evaluate the impact and added complexity to the whole go ecosystem.

@jayconrod @hyangah

should +incompatible versions still be advertised?

A proxy may continue to advertise them, but may choose to omit them. (In general the contents of a proxy's @v/list endpoint are at the discretion of its maintainers.)

I am not sure which direction we should recommend. As you note, newer versions of the go command should know what to do about them anyway, so I'm leaning toward keeping them, but still need to think through the implications of that.

/@v/<version>.info - should a +incompatible version be resolved to a pseudo-version here?

No. However, the equivalent non-+incompatible version should be resolved to a pseudo-version (instead of the +incompatible version to which it resolves today).

Will that break anything in 1.13 or earlier?

It may cause users of earlier go versions running a latest query to resolve a lower +incompatible version than what they did before. In many cases (such as #34189 and #34165), that will result in a better overall user experience, but in some cases it may cause MVS to bury the correct version (v1.0.0-something) in favor of a +incompatible dependency with a higher major version.

/@v/<version>.mod, .zip - I think we would still need to serve +incompatible versions to keep older versions of the go command working. Currently, 404 or 410 should be served for non-canonical versions.

Agreed. And go mod download would presumably need to continue to support explicitly-requested +incompatible versions.

/@latest - should probably be updated to match the new definition of @latest, but this shouldn't be strictly necessary unless /@v/list is empty.

/@latest could matter if /@v/list contains only +incompatible versions. In that case, only new versions of the go command will care, so /@latest should presumably give the compatible pseudo-version rather than one derived from a +incompatible tag.

@bcmills

github.com/Azure/go-autorest@v13.0.1 resolves to commit 69b4126ece6b.
That commit has numerous ancestors, but its last v1 ancestor seems to be v1.1.1.
Therefore, go list -m github.com/Azure/go-autorest@v13.0.1 would resolve to
github.com/Azure/go-autorest v1.1.2-0.20190906230412-69b4126ece6b

I'll use that example as a starting point for a couple of questions, but let's also make up a v12.0.1 release that also does not have a go.mod. Let's pretend it was released a while ago (say, in 2018).

That would give us:

• v12.0.1 (no go.mod) resolves as v1.1.2-0.20180101000000-aaaaaaaaaaaa (this is made up)
• v13.0.1 (no go.mod) resolves as v1.1.2-0.20190906230412-69b4126ece6b (from your example)

Those sort properly as pseudo-versions.

However, let's suppose tomorrow there is a new v12.0.2 release (given part of the point of having major versions is to allow patch releases for prior major versions).

Let's say that gives us:

• v12.0.2 (no go.mod) resolves as v1.1.2-0.20190922000000-bbbbbbbbbbbb (also made up)

At that point, that no longer sorts properly compared to the pseudo-version generated for v13.0.1. In other words, a lower major version release would be selected over a higher major version release (e.g., if there are multiple require statements in a larger build).

Let's take a second scenario. Let's pretend v12 is an import major release that they want to support for a long time, so they decide to adopt modules within the v12 train.

Let's say that then gives us:

• v12.1.0 (with a go.mod) resolves as v12.1.0

That also no longer sorts properly compared to the pseudo-version generated for v13.0.1.

Is that the way this would work? Again, sorry if any of this is off base.

Sorry, my second scenario is not as relevant, given at that point it would be go-autorest/v12 once modules were adopted within the v12 major release.

Correct me if I'm wrong but:

One nice thing about this proposal is that converting +incompatible to <psuedo-semver> will work exactly like converting a master to <pseudo-semver> right?

Which means even if the user is not using Go 1.14, but the proxy he/she is using is internally using Go 1.14, then their proxy will properly tell them to update their go.mod file to the correct version.

On the other hand, this can mean that if a proxy storage has one thousand +incompatible modules, then it will eventually have its own 1000 duplicates of the pseudo semver module.

If we can run some off-go command to know how eaxctly a +incompatible translates to the proposed pseudo semver, then go proxy maintainers can write a script to clean up the incompatible modules once they have all the corresponding duplicates

For the Go proxy we wrote at work the +incompatible situation has been a constant source of problems.
I understand that this +incompatible tag was introduced for easy backwards compatibility with libraries that had not yet upgraded to modules, but in hindsight, I feel it has more problems than benefits.

Now that almost everyone is upgrading to go modules, I think we should simply require a go.mod file for all module imports, an drop all support for the +incompatible tag. This might create some transition problems, which can be dealt with a transition over the next 2 go releases, and using go1.14 or (go1.15) in the go.mod file, but it will make go modules and go module proxies a lot easier to implement.

@beoran could you elaborate on

I think we should simply require a go.mod file for all module imports

Do you mean that we should no longer support non-module imports?

Yes. In hindsight, I think non module imports are more troublesome than they are useful. The convenience does not weigh up to the complexity. If we didn't have to support non module imports, or proxy would likely require 25% less code.

That would, in my opinion, be a much too aggressive move that risks causing an insurmountable amount of pain on the entire Go ecosystem. Especially given that it is still to be decided whether modules are going to be the default flow in 1.14, as this might be pushed again to 1.15. It would also be kind-of a breaking change (although not formally as it's not part of the Go language spec).

What such a change would mean is that projects will not be able to update their go directive in their go.mod to >=1.14 (and thus not use any new features of 1.14+) unless all their direct dependencies have adopted modules.

Given that many frequently-used projects have their oldest dependencies somewhere in the range of 2 to 7 years old that is not something that is a reasonable burden to put on these projects. Bear in mind that these dependencies are so old because they are no longer being developed and might simply have reached a stable and mature state 4+ years ago. This would mean that all such projects will need to be hard/soft forked (because the maintainer has likely moved on) or redeveloped from scratch.

Also please note: this proposal is not going to make it into 1.14 and will likely be reworked in the form of a new proposal by @bcmills, based on the golang-tools call earlier this week.

Yeah, I'm going to go ahead and withdraw this proposal — the consensus seems to be that it's too invasive. I'll file another one for something less invasive.