Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: foo@v2.0.0+incompatible currently allows Semantic Import Versioning to be optional #32695

Closed
thepudds opened this issue Jun 19, 2019 · 8 comments
Closed

cmd/go: foo@v2.0.0+incompatible currently allows Semantic Import Versioning to be optional #32695

thepudds opened this issue Jun 19, 2019 · 8 comments
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Milestone
Go1.13

Comments

@thepudds
Copy link
Contributor

thepudds commented Jun 19, 2019 •
edited

This is a spinout from "Factor 2" from #31543 ("cmd/go: creating v2+ modules has lower success rate than it could").

Compared to Go 1.11 and 1.12, current tip / 1.13 might have increased the modes of interaction that work with v2+ modules that have not adopted Semantic Import Versioning.

What version of Go are you using (go version)?

go 1.12.6

Does this issue reproduce with the latest release?

Yes, including tip as of yesterday (devel +bc27b64d Tue Jun 18 03:41:59 2019 +0000)

What did you do?

Observe people accidentally creating and using modules that have v2+ semver tags but that have not adopted Semantic Import Versioning.

For example, if there is a module example.com/foo that:

  • has a go.mod file (that is, it has adopted modules)
  • has a v2.0.0 semver tag as its latest tag
  • and its module line reads module example.com/foo (without the in theory required /v2)

then this still works for a module-based consumer, even though the module is in a "bad" state:

go get example.com.com/foo@v2.0.0+incompatible

This means people can and do create usable v2+ modules that did not adopt Semantic Import Versioning, and consumers can and do consume those "bad" modules.

However, other things such as upgrades or go get example.com.com/foo@latest do not work as expected, which leads to confusion.

What did you expect to see?

Some type of warning when consuming a v2+ module that has not adopted Semantic Import Versioning (or perhaps an error in the future after the ecosystem has more time to adapt and more tools exist to help prevent or catch mistakes).

What did you see instead?

Here is a concrete example. This uses github.com/pierrec/lz4:

  • v2.2.3 has an incorrect go.mod file missing /v2 in the module statement.
  • v2.0.6 seems to be when the "bad" go.mod was first introduced.
  • v2.0.5 is the tag immediately before the "bad" go.mod was introduced.

First, here is Go 1.12 successfully building a consumer importing the "bad" v2.0.6, as well as this shows a subsequent upgrade not working in 1.12.

cd $(mktemp -d)
export GOPATH=$(mktemp -d)
go mod init tempmod
echo 'require github.com/pierrec/lz4 v2.0.6+incompatible' >> go.mod
cat <<EOF > consumer.go
package consumer
import _ "github.com/pierrec/lz4"
EOF

# build works
go build
go list -m github.com/pierrec/lz4
# outputs:
# github.com/pierrec/lz4 v2.0.6+incompatible

# attempting to upgrade via 'go get -u' does not upgrade lz4
go get -u
go list -m github.com/pierrec/lz4
# outputs:
# github.com/pierrec/lz4 v2.0.6+incompatible

# attempting to upgrade via '@latest' downgrades
go get github.com/pierrec/lz4@latest
go list -m github.com/pierrec/lz4
# outputs:
# github.com/pierrec/lz4 v2.0.5+incompatible

Second, tip is also able to build using the "bad" v2.0.6, but now tip allows upgrading.

cd $(mktemp -d)
export GOPATH=$(mktemp -d)
gotip mod init tempmod
echo 'require github.com/pierrec/lz4 v2.0.6+incompatible' >> go.mod
cat <<EOF > consumer.go
package consumer
import _ "github.com/pierrec/lz4"
EOF

# build works
gotip build
gotip list -m github.com/pierrec/lz4
# outputs:
# github.com/pierrec/lz4 v2.0.6+incompatible

# upgrade now actually does upgrade to a newer "bad" version
gotip get github.com/pierrec/lz4@latest
gotip list -m github.com/pierrec/lz4
# outputs:
# github.com/pierrec/lz4 v2.2.3+incompatible

head -1 $(gotip list -f '{{.Dir}}' github.com/pierrec/lz4)/go.mod
# outputs:
# module github.com/pierrec/lz4
@thepudds
Copy link
Contributor Author

https://golang.org/cl/181881 might help with this.

@jayconrod
Copy link
Contributor

Hmm, this is unfortunate. In direct mode, it seems we correctly don't list these versions:

$ export GOPROXY=direct
$ go list -m -versions github.com/pierrec/lz4
github.com/pierrec/lz4 v1.0.1 v2.0.1+incompatible v2.0.2+incompatible v2.0.3+incompatible v2.0.4+incompatible v2.0.5+incompatible

However, the proxy knows about them.

$ export GOPROXY=https://proxy.golang.org,direct
$ go list -m -versions github.com/pierrec/lz4
github.com/pierrec/lz4 v1.0.1 v2.0.1+incompatible v2.0.2+incompatible v2.0.3+incompatible v2.0.4+incompatible v2.0.5+incompatible v2.0.6+incompatible v2.0.7+incompatible v2.1.1+incompatible v2.1.2+incompatible v2.2.3+incompatible

And in your example, the Go command validation doesn't catch the problem with these versions, even with GOPROXY=direct. So I think what happened is someone manually wrote in these versions to a go.mod. The proxy fetched them, they passed validation, and the proxy serves them as normal versions thereafter.

Patching in https://golang.org/cl/181881, it looks like a go.mod with a version like this will break the build in direct mode, since it's now validated.

We should figure out how many modules have invalid requirements like this.

@jayconrod
Copy link
Contributor

cc @bcmills

@jayconrod jayconrod added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 19, 2019
@jayconrod jayconrod added this to the Go1.13 milestone Jun 19, 2019
@bcmills
Copy link
Contributor

bcmills commented Jun 19, 2019

Ugh. Agreed, we need to fix this and we need to figure out how big the crater will be.

@zikaeroh
Copy link
Contributor

zikaeroh commented Jun 20, 2019 •
edited

I think another example of this is github.com/go-redis/redis, where its v6.15.3+incompatible is the first with a go.mod. I actually used tip just the other day to do an upgrade from 6.15.2 that 1.12 was unable to perform, thinking it was a bug that had been fixed in #30634 or similar. I guess I'm guilty of doing something I shouldn't have, but I didn't perform it by hand either.

EDIT: On a second look, setting GOPROXY=https://proxy.golang.org and doing an update the usual way on 1.12 also upgrades me to 6.15.3+incompatible, while direct does not.

@bcmills bcmills self-assigned this Jun 20, 2019
@bcmills bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Jun 20, 2019
@bcmills
Copy link
Contributor

bcmills commented Jun 20, 2019

On go1.12.6 I get the following results:

example.com$ go1.12.6 mod download -json github.com/go-redis/redis@v6.15.3
go: finding github.com/go-redis/redis v6.15.3
{
        "Path": "github.com/go-redis/redis",
        "Version": "v0.0.0-20190503082931-75795aa4236d",
        "Info": "/tmp/tmp.dmSFtGAmfo/_gopath/pkg/mod/cache/download/github.com/go-redis/redis/@v/v0.0.0-20190503082931-75795aa4236d.info",
        "GoMod": "/tmp/tmp.dmSFtGAmfo/_gopath/pkg/mod/cache/download/github.com/go-redis/redis/@v/v0.0.0-20190503082931-75795aa4236d.mod",
        "Zip": "/tmp/tmp.dmSFtGAmfo/_gopath/pkg/mod/cache/download/github.com/go-redis/redis/@v/v0.0.0-20190503082931-75795aa4236d.zip",
        "Dir": "/tmp/tmp.dmSFtGAmfo/_gopath/pkg/mod/github.com/go-redis/redis@v0.0.0-20190503082931-75795aa4236d",
        "Sum": "h1:x7dvutgTPxsHoeUboXlOp9fEUMdIRO4OJV5ly1LYE40=",
        "GoModSum": "h1:W2YCLaZryXHirdd9QqwkiVUxCQsrx8SbLq9Uqk7JS7A="
}

example.com$ go clean -modcache
example.com$ go1.12.6 mod download -json github.com/go-redis/redis@v6.15.3+incompatible
go: finding github.com/go-redis/redis v6.15.3+incompatible
{
        "Path": "github.com/go-redis/redis",
        "Version": "v6.15.3+incompatible",
        "Info": "/tmp/tmp.dmSFtGAmfo/_gopath/pkg/mod/cache/download/github.com/go-redis/redis/@v/v6.15.3+incompatible.info",
        "GoMod": "/tmp/tmp.dmSFtGAmfo/_gopath/pkg/mod/cache/download/github.com/go-redis/redis/@v/v6.15.3+incompatible.mod",
        "Zip": "/tmp/tmp.dmSFtGAmfo/_gopath/pkg/mod/cache/download/github.com/go-redis/redis/@v/v6.15.3+incompatible.zip",
        "Dir": "/tmp/tmp.dmSFtGAmfo/_gopath/pkg/mod/github.com/go-redis/redis@v6.15.3+incompatible",
        "Sum": "h1:NZ0O90AhLSvSrvLZ/S9h7D4kl1mW2PrKyxL7MyBKO2g=",
        "GoModSum": "h1:W2YCLaZryXHirdd9QqwkiVUxCQsrx8SbLq9Uqk7JS7A="
}

That first result is the intended behavior; the second is this bug.

@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 20, 2019
@gopherbot
Copy link

Change https://golang.org/cl/181881 mentions this issue: cmd/go: validate pseudo-versions against module paths and revision metadata

@dmitshur dmitshur mentioned this issue Jun 20, 2019
Closed
@bcmills bcmills mentioned this issue Jun 21, 2019
Closed
@bcmills bcmills mentioned this issue Jul 15, 2019
Closed
@bcmills
Copy link
Contributor

bcmills commented Jul 15, 2019

Two more examples of modules that now fail validation:

@bcmills bcmills mentioned this issue Jul 22, 2019
Closed
@bcmills bcmills mentioned this issue Aug 9, 2019
Closed
@bcmills bcmills mentioned this issue Sep 10, 2019
Closed
@bcmills bcmills mentioned this issue Jan 7, 2020
Open
@golang golang locked and limited conversation to collaborators Jul 14, 2020
@rsc rsc unassigned bcmills Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Projects
None yet
Milestone
Go1.13
Development

No branches or pull requests
5 participants
@jayconrod @bcmills @gopherbot @thepudds @zikaeroh