cmd/go: go get tries to download non-existing commit #33076

ayzu · 2019-07-12T13:28:23Z

Please answer these questions before submitting your issue. Thanks!

What did you do?

mkdir project
cd project
export GO111MODULE=on
go mod init project
go get github.com/micro/go-plugins@v1.0.0

What did you expect to see?

successful download of dependency

What did you see instead?

go: finding github.com/nats-io/nats.go v1.8.2-0.20190607221125-9f4d16fe7c2d
go: github.com/nats-io/nats.go@v1.8.2-0.20190607221125-9f4d16fe7c2d: unknown revision 9f4d16fe7c2d
go: error loading module requirements

Does this issue reproduce with the latest release (go1.12.7)?

yes

System details

go version go1.12.7 darwin/amd64

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/a8327/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/a8327/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12.6/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12.6/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/09/xght07y913s83cqxlwc_x3z017ny38/T/go-build408435640=/tmp/go-build -gno-record-gcc-switches -fno-common"

Note:
The case is that github.com/micro/go-plugins@v1.0.0 has a mod file https://github.com/micro/go-plugins/blob/v1.0.0/go.mod which requires github.com/nats-io/go-nats v1.7.0

Repository github.com/nats-io/go-nats is archived and probably redirects to https://github.com/nats-io/nats.go.

However, github.com/nats-io/nats.go does not have commit 9f4d16fe7c2d at all. Probably, it was here at some point but then was removed. So go get tries download non-existing commit from the master branch instead of latest tag 1.8.1 in github.com/nats-io/nats.go.

The same error will pop up, if we specify the dependency in go.mod and run go build.

The text was updated successfully, but these errors were encountered:

bcmills · 2019-07-12T16:47:35Z

This issue does not reproduce for me with go1.13beta1 and the provided steps.

project$ go1.13beta1 mod init project
go: creating new go.mod: module project

project$ go1.13beta1 get github.com/micro/go-plugins@v1.0.0
go: finding github.com/micro/go-plugins v1.0.0
go: downloading github.com/micro/go-plugins v1.0.0
go: extracting github.com/micro/go-plugins v1.0.0

project$

Nor does it reproduce if the @v1.0.0 suffix is dropped, presumably because proxy.golang.org still has the missing commit in its cache:

project$ go1.13beta1 get github.com/micro/go-plugins
go: finding github.com/micro/go-plugins v1.1.1
[…]
go: finding github.com/micro/micro v1.7.1-0.20190627121529-410a2eba67f1

project$

The failure mode with go1.13beta1 and GOPROXY=direct is different, apparently due to a tag deleted from k8s.io/client-go. (The error message below is accurate: I checked https://github.com/kubernetes/client-go, and there really is no such tag at this point.)

project$ go1.13beta1 mod init project
go: creating new go.mod: module project

project$ export GOPROXY=direct

project$ go1.13beta1 get github.com/micro/go-plugins@v1.0.0
go: finding github.com/micro/go-plugins v1.0.0
go get: github.com/micro/go-plugins@v1.0.0 requires
        github.com/micro/go-config@v0.14.0 requires
        k8s.io/client-go@v2.0.0-alpha.0.0.20190126161006-6134db91200e+incompatible: invalid pseudo-version: preceding tag (v2.0.0-alpha.0) not found

project$

I can reproduce the failure mode you observed with go1.13beta1, GOPROXY=direct, and no @v1.0.0 suffix:

project$ go1.13beta1 get github.com/micro/go-plugins
go: finding github.com/micro/go-plugins v1.1.1
go get: github.com/micro/go-plugins@v1.1.1 requires
        github.com/nats-io/nats.go@v1.8.2-0.20190607221125-9f4d16fe7c2d: unknown revision 9f4d16fe7c2d

project$

That comes from an explicit requirement here, and that error message is also accurate: the repository at https://github.com/nats-io/nats.go contains no such commit.

Sorry, but this is not a bug in the go command. The proxy.golang.org mirror is continuing to cache this content for the moment (as designed), and also as designed, GOPROXY=direct is reporting the accurate state of the underlying repositories. With the caching proxy bypassed, there isn't much we can do if someone deletes an existing tag or force-pushes their repo to remove a commit or version that is already in use.

bcmills · 2019-07-12T16:48:18Z

CC @derekcollison

derekcollison · 2019-07-12T16:53:50Z

There is no 1.8.2 release, so this is from a commit on master most likely that may no longer exist.

bcmills · 2019-07-12T16:55:06Z

Yes: judging by the pseudo-version, it is specifically a commit on master at some point after the v1.8.1 tag.

ayzu · 2019-07-12T16:56:02Z

Yes, I guess that's exactly the case. However, my question is why it tries to find this non-existing commit, instead of using existing latest tag 1.8.1? Wouldn't be that expected behaviour?

bcmills · 2019-07-12T16:58:00Z

@ayzu, builds using Go modules are expected to be 100% reproducible. If the configuration required for a reproducible build cannot be obtained, that results in an error rather than a silent change in behavior.

Note that you can explicitly change the behavior by adding a replace directive in your main module, such as

replace github.com/nats-io/nats.go v1.8.2-0.20190607221125-9f4d16fe7c2d => github.com/nats-io/nats.go v1.8.1

bcmills · 2019-07-12T17:00:05Z

(And note that, at least for now, you can explicitly set GOPROXY=https://proxy.golang.org and to be able to build both versions of github.com/micro/go-plugins in their original configuration.)

ayzu · 2019-07-12T17:19:45Z

@derekcollison @bcmills

There is no 1.8.2 release, so this is from a commit on master most likely that may no longer exist.

Yes, this is the exact issue. It tries to find non-existing commit and fails. Is it expected behaviour?

The dependency in go.mod file is github.com/nats-io/go-nats v1.7.0 This repository redirects to github.com/nats-io/nats.go

From wiki:

In this section, "latest" is the latest version with a semver tag, or the latest known commit if there are no semver tags.

In my understanding go get was supposed to download the latest tag (1.8.1 > 1.7.0) or at least the exact 1.7.0 version. Instead, it tries to download non-existing commit (not tag).

Moreover, this commit v1.8.2-0.20190607221125-9f4d16fe7c2d is not even the latest one. It was removed at some part in the past, and there were some more commits after. According to https://github.com/nats-io/nats.go/commits/master, the latest commit on master was yesterday, 11 July.

Could you please elaborate why go get has chosen it?

derekcollison · 2019-07-12T17:35:45Z

Is it listed in the go.sum?

ayzu · 2019-07-12T17:41:34Z

Is it listed in the go.sum?

You mean in go-plugins repository? https://github.com/micro/go-plugins/blob/6eef94c098315c5cfc03334fbcb5f8d43b1ca04a/go.sum#L523

Yes, it is listed:

github.com/nats-io/go-nats v1.7.0 h1:oQOfHcLr8hb43QG8yeVyY2jtarIaTjOv41CGdF3tTvQ=
github.com/nats-io/go-nats v1.7.0/go.mod h1:+t7RHT5ApZebkrQdnn6AhQJmhJJiKAvJUio1PiiCtj0=

ayzu · 2019-07-12T17:56:30Z

@bcmills @derekcollison

There is something more interesting. If I have a direct dependency github.com/micro/go-plugins@v1.0.0 which has go.mod which requries github.com/nats-io/go-nats v1.7.0, go build will fail with error github.com/nats-io/nats.go@v1.8.2-0.20190607221125-9f4d16fe7c2d: unknown revision 9f4d16fe7c2d. So instead of downloadinggithub.com/nats-io/go-nats v1.7.0 it will redirect to different repository github.com/nats-io/nats.go and will try to download non-existing (and not-latest commit). Note, that version of go-plugins is v1.0.0 and two different repos go-nats and nats.go.

But if try to download the exact dependency github.com/nats-io/go-nats v1.7.0 directly: go get github.com/nats-io/go-nats@v1.7.0. It will successfully download the required version without redirection to the other repository.

module local/dep

go 1.12

require (
	github.com/nats-io/go-nats v1.7.0 // indirect
	github.com/nats-io/nats.go v1.7.0 // indirect
	github.com/nats-io/nkeys v0.1.0 // indirect
	github.com/nats-io/nuid v1.0.1 // indirect
)

So, it looks like behaviour differs for those use cases.

bcmills · 2019-07-12T17:58:11Z

@ayzu, it seems pretty clear that the version in question is coming from the go.mod file in github.com/micro/go-plugins v1.1.1. If that's not the version of go-plugins you expected, it may be due to #29121, which is fixed in go1.13beta1.

ayzu · 2019-07-12T18:04:44Z

@ayzu, it seems pretty clear that the version in question is coming from the go.mod file in github.com/micro/go-plugins v1.1.1. If that's not the version of go-plugins you expected, it may be due to #29121, which is fixed in go1.13beta1.

Yes, it comes from v.1.1.1 But I specifically identify the version in the command: go get github.com/micro/go-plugins@v1.0.0. And this version (v1.0.0) has a go.mod file which requires github.com/nats-io/go-nats v1.7.0 and not pseudo-tag v1.8.2.

So why v1.0.0 version uses go.mod file from other verion (v1.1.1)?

bcmills · 2019-07-12T18:05:30Z

See #29121.

ayzu · 2019-07-12T18:31:02Z

@bcmills

Could you please specify how the mentioned #29121 relates to my question?

As far as I understand, the mentioned problem was that go mod assigned incorrect pseudo-tag (starting with v2.0.0) to a commit.

The issue was solved, and the commit was assigned a correct pseudo-version k8s.io/client-go v0.0.0-20181126152608-d082d5923d3c (because the module identifies itself as module k8s.io/client-go, so for go.mod it is not version 2 regardless of any tag).

In this case, we have a pseudo-tag 1.8.2. This pseudo-tag is correct because the dependency identifies itself as github.com/nats-io/go-nats and the latest tag is 1.8.1.

However, my question was - why do go get uses this commit in the first place? (instead of a tag 1.7.0 which was required in a go.mod file, or latest tag 1.8.1 or even latest commit in the master branch)

bcmills · 2019-07-12T18:42:10Z

@ayzu, #29121 causes go get github.com/micro/go-plugins@v1.0.0 to instead fetch github.com/micro/go-plugins@latest. The configuration of github.com/micro/go-plugins@latest explicitly refers to v1.8.2-0.20190607221125-9f4d16fe7c2d, for which the underlying commit was deleted (presumably by a force-push).

ayzu · 2019-07-12T20:28:12Z

@bcmills

Ok, thank you. I was under assumption that the mentioned issue has happened because kubernetes didn't followed semver.

So there is a bug in 1.12: one specifies a version "go get some/package@v1.0.8", but 'go get' downloads the latest version instead. As far understand, this bug occur only in some cases?

bcmills · 2019-07-12T20:58:00Z

Yes. Specifically, it occurs when the argument to go get is a module that contains no .go source files at its root and the -m flag is not set.

(The -m flag is removed in go1.13beta1, but the bug is fixed in it too.)

ayzu · 2019-07-12T21:19:26Z

@bcmills

Ok, thank for your time and effort. Nice to hear that this bag was already fixed.

bcmills · 2019-07-12T21:26:04Z

for which the underlying commit was deleted (presumably by a force-push).

It turns out that that commit never should have been resolvable in the first place (see #31191): github.com/nats-io/nats.go@9f4d16fe7c2d was a PR, and was merged as commit 3d6c21bef4f7 instead.

That at least gives you a more precise target for a replace directive, though. 🙂

ayzu · 2019-07-13T12:34:26Z

Ok, now I have got the title of the issue: 'go get @' should not fall back to 'latest' if does not provide

I.e., tag version does not have go files in the root directory, and therefore does not provide a package.

bcmills closed this as completed Jul 12, 2019

bcmills changed the title ~~go get tries to download non-existing commit~~ cmd/go: go get tries to download non-existing commit Jul 12, 2019

bcmills added the modules label Jul 12, 2019

wallyqs mentioned this issue Jul 12, 2019

Go Mod Revision Error (unknown revision 9f4d16f) nats-io/nats.go#493

Closed

sebbalex added a commit to italia/publiccode-crawler that referenced this issue Oct 11, 2019

Bumped golang to 1.13 due to golang/go#33076

87c7812

golang locked and limited conversation to collaborators Jul 12, 2020

gopherbot added the FrozenDueToAge label Jul 12, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: go get tries to download non-existing commit #33076

cmd/go: go get tries to download non-existing commit #33076

ayzu commented Jul 12, 2019

bcmills commented Jul 12, 2019 •

edited

Loading

bcmills commented Jul 12, 2019

derekcollison commented Jul 12, 2019

bcmills commented Jul 12, 2019

ayzu commented Jul 12, 2019

bcmills commented Jul 12, 2019

bcmills commented Jul 12, 2019

ayzu commented Jul 12, 2019 •

edited

Loading

derekcollison commented Jul 12, 2019

ayzu commented Jul 12, 2019 •

edited

Loading

ayzu commented Jul 12, 2019

bcmills commented Jul 12, 2019 •

edited

Loading

ayzu commented Jul 12, 2019 •

edited

Loading

bcmills commented Jul 12, 2019

ayzu commented Jul 12, 2019 •

edited

Loading

bcmills commented Jul 12, 2019

ayzu commented Jul 12, 2019

bcmills commented Jul 12, 2019

ayzu commented Jul 12, 2019

bcmills commented Jul 12, 2019

ayzu commented Jul 13, 2019

cmd/go: go get tries to download non-existing commit #33076

cmd/go: go get tries to download non-existing commit #33076

Comments

ayzu commented Jul 12, 2019

What did you do?

What did you expect to see?

What did you see instead?

Does this issue reproduce with the latest release (go1.12.7)?

System details

bcmills commented Jul 12, 2019 • edited Loading

bcmills commented Jul 12, 2019

derekcollison commented Jul 12, 2019

bcmills commented Jul 12, 2019

ayzu commented Jul 12, 2019

bcmills commented Jul 12, 2019

bcmills commented Jul 12, 2019

ayzu commented Jul 12, 2019 • edited Loading

derekcollison commented Jul 12, 2019

ayzu commented Jul 12, 2019 • edited Loading

ayzu commented Jul 12, 2019

bcmills commented Jul 12, 2019 • edited Loading

ayzu commented Jul 12, 2019 • edited Loading

bcmills commented Jul 12, 2019

ayzu commented Jul 12, 2019 • edited Loading

bcmills commented Jul 12, 2019

ayzu commented Jul 12, 2019

bcmills commented Jul 12, 2019

ayzu commented Jul 12, 2019

bcmills commented Jul 12, 2019

ayzu commented Jul 13, 2019

bcmills commented Jul 12, 2019 •

edited

Loading

ayzu commented Jul 12, 2019 •

edited

Loading

ayzu commented Jul 12, 2019 •

edited

Loading

bcmills commented Jul 12, 2019 •

edited

Loading

ayzu commented Jul 12, 2019 •

edited

Loading

ayzu commented Jul 12, 2019 •

edited

Loading