cmd/go: go mod assumes a lexicographically linear ordering of versions #48915
Comments
ALTree
added
NeedsInvestigation
|
In your fifth paragraph, are you mixing up versions 3.x.y with 1.x.y? For example, it says I appreciate the experience report, by the way - I've run into similar versions of this subtle problem. I think my only bit of input is that, if C's v3.1.0 is so terribly broken that v3.1.1 should always be preferred, v3.1.1 could retract v3.1.0, signaling to MVS that it should prefer v3.1.1. But I get that retraction should only be used for terribly broken versions, such as those containing security bugs, and often the bugs are not that severe - though still important. |
|
The Go module system uses the precedence rules defined by semantic versioning. At any rate, I think the heart of the issue is this:
I don't think it actually is all that weird to need to upgrade an indirect dependency to pull in a bug-fix — and note that at |
It does not merely order by commit date. The pseudo-versions that the The pseudo-version derivation does require a bit of careful tagging when using release branches (to ensure that commits on the mainline are ordered correctly relative to commits on release branches), but I don't see how we can substantially improve that logic given how little version information is encoded in the git branch structure. |
Yep, that was a typo, I have updated my original post.
That assumes that there are tags, which is not universally true. For example, the x repos are not tagged (except for x/text).
That isn't really feasible when there are a lot of indirect dependencies to account for. It would be preferable for go mod to fail (or at least warn) and force me to resolve the issue explicitly (by asking for 3.1.1 in this case) in my go.mod file. |
|
In terms of warnings to watch out for, how about:
This does not imply that v3.1.1 should always be preferred over v3.1.0, but it's certainly a strong signal in many practical scenarios. |
|
I don't think a warning is appropriate. We have no particular reason to believe that v3.0.1 fixes a bug that was present in v3.1.1, rather than (say) merely providing a backport of a fix that landed in v3.1.0, We already provide the |
I agree that as the module ecosystem currently stands, making this determination is not generally possible (or at the very least, fairly non-trivial). But, for example, if the go.mod file had a way to indicate that 3.0.1 actually came after 3.1.0, then it would be trivial.
This should work well enough for versioned libraries, although I still think it would be preferable for go mod to complain about incompatible versions rather than assuming you are actively running such a command. In any case, my understanding is that for unversioned modules, the rough equivalent would be to update to "latest"? To detect the mismatch there would require the use of git commands (to ask if the chosen commit is a descendant of all desired commits), which probably wouldn't work as it requires a full (i.e., not shallow) clone. |
According to the semver spec, v3.0.1 is not “incompatible” with v3.1.0. It is certainly possible that v3.1.0 contains a bug that is not present in v3.0.1, but that isn't fundamentally different from the situation in which v3.1.0 contains a bug and only v3.1.1 includes the fix. The
Yes. In some sense, if a module is unversioned, then the notions of “major release”, ”minor release”, and “patch release” all collapse to “later commit”. We intentionally do not attempt to track version control branches beyond identifying the appropriate base for a pseudo-version, but compare #26964 (which is still open). |
|
I don't think there is anything we can feasibly do here, but if you have a specific proposal please do feel free to file it. |
This is basically an experience report, based on some issues we have run into with go mod.
Go mod fundamentally assumes that there is a simple ordering between all versions of a module. In essence, if version X is lexicographically less than version Y, then Y is a superset of X. However, this is not always true, and can lead to some unexpected implications when dealing with transitive dependencies.
For example, suppose my main module has a direct dependency on version 1.0.0 of A, and version 2.0.0 of B. Also suppose that A has a dependency on version 3.0.0 of C, and B has a dependency on version 3.1.0 of C. As per minimum version selection, my main module will select version 3.1.0 of C to satisfy the indirect dependency.
Now suppose some bug is discovered in module C. They release a new version 3.1.1, and also backport the fix to version 3.0.1. Then module A releases a new version 1.0.1 that depends on version 3.0.1 of C. But module B has not released a new version yet.
In my main module, I update my dependency on A to the new version (1.0.1), under the pretense of pulling in the bugfix. However, due to minimal version selection, I will still be using version 3.1.0 of C. The root cause of the issue is that it incorrectly assumes that just because 3.1.0 > 3.0.1, that means that 3.1.0 contains all of 3.0.1. The only way for this to work correctly is for me to manually specify version 3.1.1 of C in my go.mod file, which is especially weird because it is not a direct dependency.
A similar issue unfolds with unversioned libraries. There go mod just orders by commit date. But the commits in question need not be from the same branch, so again there may not be a convenient linear history between them. This is especially problematic with cherry picks, where if one module is tracking a release branch, then a cherry pick would cause its HEAD to be considered newer than an older commit from the main branch.
The text was updated successfully, but these errors were encountered: