Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: "found, but does not contain package" error refers to replaced version instead of its replacement #34085

Closed
rajathagasthya opened this issue Sep 4, 2019 · 10 comments
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@rajathagasthya
Copy link
Contributor

What version of Go are you using (go version)?

$ go version
go version go1.13 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/rajagast/Library/Caches/go-build"
GOENV="/Users/rajagast/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY="none"
GONOSUMDB="git.company.com"
GOOS="darwin"
GOPATH="/Users/rajagast/go"
GOPRIVATE="git.company.com"
GOPROXY="https://athens.company.com"
GOROOT="/usr/local/Cellar/go/1.13/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/rajagast/go/src/git.company.com/CPSG/ccp-operator/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/d0/18x8fk8571v3t2p12n8r340h0000gn/T/go-build223406614=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I have k8s.io/code-generator as a tools dependency i.e. I have a top-level tools.go file with these contents:

// +build tools

package tools

import (
	_ "k8s.io/code-generator/cmd/client-gen"
	_ "k8s.io/code-generator/cmd/conversion-gen"
	_ "k8s.io/code-generator/cmd/deepcopy-gen"
	_ "k8s.io/code-generator/cmd/defaulter-gen"
	_ "k8s.io/code-generator/cmd/informer-gen"
	_ "k8s.io/code-generator/cmd/lister-gen"
	_ "k8s.io/code-generator/cmd/openapi-gen"
)

I also have one private dependency, so according to go1.13 I'm setting appropriate env variables.

$ go env -w GOPRIVATE=git.company.com GOPROXY=https://athens.company.com GONOPROXY=none

When I do go mod tidy, I get the following failure which is very vague:

go: finding k8s.io/code-generator latest
git.company.com/CPSG/ccp-operator imports
	k8s.io/code-generator/cmd/openapi-gen: module k8s.io/code-generator/cmd/openapi-gen: reading https://athens.company.com/k8s.io/code-generator/cmd/openapi-gen/@v/list: 500 Internal Server Error
git.company.com/CPSG/ccp-operator/pkg/controller/cluster/worker imports
	github.com/openshift/kubernetes-drain tested by
	github.com/openshift/kubernetes-drain.test imports
	github.com/go-log/log/capture: module github.com/go-log/log/capture: reading https://athens.company.com/github.com/go-log/log/capture/@v/list: 500 Internal Server Error

The interesting thing is when I turn off GOPROXY with go env -w "GOPROXY=", it shows a different error:

go: finding k8s.io/code-generator latest
git.company.com/CPSG/ccp-operator imports
	k8s.io/code-generator/cmd/openapi-gen: module k8s.io/code-generator@latest (v0.0.0-20190831074504-732c9ca86353) found, but does not contain package k8s.io/code-generator/cmd/openapi-gen
git.company.com/CPSG/ccp-operator/pkg/controller/cluster/worker imports
	github.com/openshift/kubernetes-drain tested by
	github.com/openshift/kubernetes-drain.test imports
	github.com/go-log/log/capture: module github.com/go-log/log@latest (v0.1.0) found, but does not contain package github.com/go-log/log/capture

The error k8s.io/code-generator/cmd/openapi-gen: module k8s.io/code-generator@latest (v0.0.0-20190831074504-732c9ca86353) found, but does not contain package k8s.io/code-generator/cmd/openapi-gen is strange because latest code in that repo does contain that package.

Fwiw, this used to work perfectly fine with go1.12.9 yesterday.

This is my go.mod:

module git.company.com/CPSG/ccp-operator

go 1.12

require (
	github.com/docker/distribution v2.7.1+incompatible // indirect
	github.com/docker/spdystream v0.0.0-20181023171402-6480d4af844c // indirect
	github.com/elazarl/goproxy v0.0.0-20181111060418-2ce16c963a8a // indirect
	github.com/evanphx/json-patch v4.2.0+incompatible // indirect
	github.com/ghodss/yaml v1.0.0 // indirect
	github.com/go-log/log v0.1.0 // indirect
	github.com/go-logr/logr v0.1.0
	github.com/go-logr/zapr v0.1.0
	github.com/gogo/protobuf v1.2.2-0.20190723190241-65acae22fc9d // indirect
	github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
	github.com/golang/groupcache v0.0.0-20180924190550-6f2cf27854a4 // indirect
	github.com/google/btree v1.0.0 // indirect
	github.com/google/uuid v1.1.1
	github.com/googleapis/gnostic v0.2.0 // indirect
	github.com/gorilla/mux v1.7.1
	github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7 // indirect
	github.com/hashicorp/golang-lru v0.5.1 // indirect
	github.com/imdario/mergo v0.3.6 // indirect
	github.com/json-iterator/go v1.1.7 // indirect
	github.com/mattbaird/jsonpatch v0.0.0-20171005235357-81af80346b1a // indirect
	github.com/modern-go/reflect2 v1.0.1 // indirect
	github.com/onsi/ginkgo v1.8.0
	github.com/onsi/gomega v1.5.0
	github.com/opencontainers/go-digest v1.0.0-rc1 // indirect
	github.com/openshift/cluster-network-operator v0.0.0-20190613202004-28fc1e342cde // indirect
	github.com/openshift/kubernetes-drain v0.0.0-20180831174519-c2e51be1758e
	github.com/operator-framework/operator-sdk v0.4.1
	github.com/pborman/uuid v1.2.0
	github.com/peterbourgon/diskv v2.0.1+incompatible // indirect
	github.com/pkg/errors v0.8.1
	github.com/prometheus/client_golang v0.9.2 // indirect
	github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90 // indirect
	github.com/prometheus/common v0.2.0 // indirect
	github.com/prometheus/procfs v0.0.0-20190403104016-ea9eea638872 // indirect
	github.com/renstrom/dedent v0.0.0-00010101000000-000000000000 // indirect
	github.com/rogpeppe/go-internal v1.3.0 // indirect
	github.com/spf13/pflag v1.0.3 // indirect
	github.com/vmware/govmomi v0.19.0
	go.uber.org/atomic v1.3.2 // indirect
	go.uber.org/multierr v1.1.0 // indirect
	go.uber.org/zap v1.9.1
	golang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8 // indirect
	golang.org/x/net v0.0.0-20190812203447-cdfb69ac37fc
	golang.org/x/oauth2 v0.0.0-20190402181905-9f3314589c9a // indirect
	golang.org/x/sys v0.0.0-20190616124812-15dcb6c0061f // indirect
	golang.org/x/text v0.3.2 // indirect
	golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2 // indirect
	golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac // indirect
	google.golang.org/appengine v1.5.0 // indirect
	gopkg.in/inf.v0 v0.9.1 // indirect
	gopkg.in/yaml.v2 v2.2.2 // indirect
	k8s.io/api v0.0.0-20190704094930-781da4e7b28a
	k8s.io/apiextensions-apiserver v0.0.0-20190704104408-4c62af35d786
	k8s.io/apimachinery v0.0.0-20190704094625-facf06a8f4b8
	k8s.io/apiserver v0.0.0-20190704101201-5341ddeccec6 // indirect
	k8s.io/client-go v10.0.0+incompatible
	k8s.io/cluster-bootstrap v0.0.0-20190704110328-86aca54c9e62
	k8s.io/code-generator v0.0.0-20190704094322-1ed9df051d9b
	k8s.io/gengo v0.0.0-20190826232639-a874a240740c // indirect
	k8s.io/klog v0.4.0 // indirect
	k8s.io/kube-openapi v0.0.0-20190816220812-743ec37842bf // indirect
	k8s.io/kube-proxy v0.0.0-20190704105642-5b7f4f839816 // indirect
	k8s.io/kubelet v0.0.0-20190704105832-d6717358a5ca // indirect
	k8s.io/kubernetes v1.13.5
	k8s.io/utils v0.0.0-20190607212802-c55fbcfc754a // indirect
	sigs.k8s.io/controller-runtime v0.1.9
	sigs.k8s.io/testing_frameworks v0.1.1 // indirect
	git.company.com/CPSG/ccp-net-tinker v0.0.2-0.20190614085909-34571ec5f0c5
)

replace (
	github.com/renstrom/dedent => github.com/lithammer/dedent v1.1.0
	k8s.io/code-generator => k8s.io/code-generator v0.0.0-20190704094322-1ed9df051d9b
)

What did you expect to see?

go mod tidy resolve dependencies correctly without error, as it did with go1.12.9.

What did you see instead?

Two issues:

  1. go mod tidy says packages do not exist when they are indeed present.
  2. When GOPROXY is set, it errors out without any meaningful message.
@heschi
Copy link
Contributor

heschi commented Sep 4, 2019

github.com/go-log/log/capture: module github.com/go-log/log/capture: reading https://athens.company.com/github.com/go-log/log/capture/@v/list: 500 Internal Server Error

For this you'll have to check out your Athens instance, no idea what the problem is there.

For the second part, I can reproduce with

git clone https://github.com/openshift/kubernetes-drain.git
cd kubernetes-drain
go mod init github.com/openshift/kubernetes-drain
go test -c

Let me see what I can figure out...

@heschi
Copy link
Contributor

heschi commented Sep 4, 2019

It's clearly true that kubernetes-drain depends on a package in log that doesn't exist at the latest tagged version, v0.1.0. So it seems reasonable that 1.13 is unhappy; perhaps it was a bug in 1.12 that it didn't notice. Generally speaking, if go mod tidy notices that there is a package missing, it will try to upgrade to the latest version of the module to see if it exists, but in this case there is no tagged version that contains the capture package. I'd suggest forcing an upgrade by replacing v0.1.0 with master in your go.mod to tell the Go command to use the latest commit rather than the latest tagged version.

@bcmills to confirm that this is all WAI.

@heschi heschi added modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Sep 4, 2019
@bcmills
Copy link
Contributor

bcmills commented Sep 4, 2019

@rajathagasthya, starting without your go.mod file I'm seeing a successful go mod tidy using proxy.golang.org.

example.com$ go1.13 mod init example.com
go: creating new go.mod: module example.com

example.com$ cat > tools.go
// +build tools

package tools

import (
        _ "k8s.io/code-generator/cmd/client-gen"
        _ "k8s.io/code-generator/cmd/conversion-gen"
        _ "k8s.io/code-generator/cmd/deepcopy-gen"
        _ "k8s.io/code-generator/cmd/defaulter-gen"
        _ "k8s.io/code-generator/cmd/informer-gen"
        _ "k8s.io/code-generator/cmd/lister-gen"
        _ "k8s.io/code-generator/cmd/openapi-gen"
)

example.com$ go1.13 mod tidy
go: finding k8s.io/code-generator latest
go: downloading k8s.io/code-generator v0.0.0-20190831074504-732c9ca86353
go: extracting k8s.io/code-generator v0.0.0-20190831074504-732c9ca86353
go: downloading k8s.io/gengo v0.0.0-20190822140433-26a664648505
go: downloading github.com/spf13/pflag v1.0.3
go: downloading k8s.io/kube-openapi v0.0.0-20190816220812-743ec37842bf
go: downloading k8s.io/klog v0.4.0
go: extracting github.com/spf13/pflag v1.0.3
go: extracting k8s.io/gengo v0.0.0-20190822140433-26a664648505
go: extracting k8s.io/klog v0.4.0
go: downloading golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac
go: extracting k8s.io/kube-openapi v0.0.0-20190816220812-743ec37842bf
go: downloading github.com/stretchr/testify v1.3.0
go: downloading github.com/emicklei/go-restful v2.9.5+incompatible
go: downloading github.com/go-openapi/spec v0.19.2
go: extracting github.com/stretchr/testify v1.3.0
go: downloading github.com/pmezard/go-difflib v1.0.0
go: downloading github.com/davecgh/go-spew v1.1.1
go: extracting github.com/emicklei/go-restful v2.9.5+incompatible
go: downloading github.com/json-iterator/go v1.1.7
go: extracting github.com/pmezard/go-difflib v1.0.0
go: extracting github.com/go-openapi/spec v0.19.2
go: extracting github.com/davecgh/go-spew v1.1.1
go: downloading github.com/go-openapi/jsonreference v0.19.2
go: downloading gopkg.in/yaml.v2 v2.2.2
go: downloading github.com/go-openapi/swag v0.19.2
go: downloading github.com/go-openapi/jsonpointer v0.19.2
go: extracting github.com/json-iterator/go v1.1.7
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading github.com/modern-go/reflect2 v1.0.1
go: extracting golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac
go: extracting github.com/go-openapi/jsonreference v0.19.2
go: downloading github.com/PuerkitoBio/purell v1.1.1
go: extracting github.com/go-openapi/jsonpointer v0.19.2
go: extracting gopkg.in/yaml.v2 v2.2.2
go: downloading gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127
go: extracting github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: extracting github.com/go-openapi/swag v0.19.2
go: downloading github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63
go: extracting github.com/modern-go/reflect2 v1.0.1
go: extracting github.com/PuerkitoBio/purell v1.1.1
go: downloading golang.org/x/net v0.0.0-20190812203447-cdfb69ac37fc
go: downloading golang.org/x/text v0.3.2
go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578
go: extracting gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127
go: downloading github.com/kr/pretty v0.1.0
go: extracting github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578
go: extracting github.com/kr/pretty v0.1.0
go: downloading github.com/kr/text v0.1.0
go: extracting github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63
go: extracting github.com/kr/text v0.1.0
go: extracting golang.org/x/net v0.0.0-20190812203447-cdfb69ac37fc
go: extracting golang.org/x/text v0.3.2

example.com$

When I add in the replace directive, I get the failure message that you report:

example.com$ go mod edit -replace k8s.io/code-generator=k8s.io/code-generator@v0.0.0-20190704094322-1
ed9df051d9b

example.com$ go1.13 mod tidy
go: downloading k8s.io/code-generator v0.0.0-20190704094322-1ed9df051d9b
go: extracting k8s.io/code-generator v0.0.0-20190704094322-1ed9df051d9b
go: finding k8s.io/gengo latest
go: finding k8s.io/klog v0.4.0
go: finding github.com/spf13/pflag v1.0.3
go: finding k8s.io/code-generator latest
go: downloading k8s.io/gengo v0.0.0-20190826232639-a874a240740c
go: extracting k8s.io/gengo v0.0.0-20190826232639-a874a240740c
go: finding golang.org/x/tools latest
go: downloading golang.org/x/tools v0.0.0-20190903163617-be0da057c5e3
go: extracting golang.org/x/tools v0.0.0-20190903163617-be0da057c5e3
example.com imports
        k8s.io/code-generator/cmd/openapi-gen: module k8s.io/code-generator@latest (v0.0.0-20190831074504-732c9ca86353) found, but does not contain package k8s.io/code-generator/cmd/openapi-gen

example.com$

As far as I can tell this is all working as intended, except that the failure message refers to the replaced version instead of its replacement. That is a bug.

@bcmills bcmills changed the title cmd/go: 'go mod tidy' complains about packages not existing when it's present cmd/go: "found, but does not contain package" error refers to replaced version instead of its replacement Sep 4, 2019
@bcmills bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Sep 4, 2019
@bcmills bcmills added this to the Go1.14 milestone Sep 4, 2019
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 4, 2019
@rajathagasthya
Copy link
Contributor Author

@heschik Thanks for checking it out. It does look like 1.12 was ignoring this because go-log/log was always v0.1.0. Same case with k8s.io/code-generator.

@bcmills Thanks for confirming the bug. If the fix is simple, I'm happy to help.

@gopherbot
Copy link

Change https://golang.org/cl/193519 mentions this issue: cmd/go/internal/modload: fix wrong module info in error message

@gopherbot
Copy link

Change https://golang.org/cl/193617 mentions this issue: cmd/go/internal/modload: annotate replacements in PackageNotInModuleError

@gopherbot
Copy link

Change https://golang.org/cl/193618 mentions this issue: cmd/go/internal/module: document Version usage for filesystem replacements

@bcmills
Copy link
Contributor

bcmills commented Sep 5, 2019

@gopherbot, please backport to 1.13: this produces very confusing error messages, and the fix is small.

@gopherbot
Copy link

Backport issue(s) opened: #34118 (for 1.13).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

gopherbot pushed a commit that referenced this issue Sep 10, 2019
…ments

Updates #34085

Change-Id: I57250d0c51a27f0bd3e223588dde9d6d955e0fcf
Reviewed-on: https://go-review.googlesource.com/c/go/+/193618
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
@bcmills bcmills self-assigned this Sep 17, 2019
@gopherbot
Copy link

Change https://golang.org/cl/197317 mentions this issue: [release-branch.go1.13] cmd/go/internal/modload: annotate replacements in PackageNotInModuleError

gopherbot pushed a commit that referenced this issue Sep 26, 2019
…s in PackageNotInModuleError

Updates #34085
Fixes #34118

Change-Id: I3111f5997466ad33f51e80c71f5fb2ccebdcc6e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/193617
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
(cherry picked from commit 8189a06)
Reviewed-on: https://go-review.googlesource.com/c/go/+/197317
@golang golang locked and limited conversation to collaborators Sep 24, 2020
@rsc rsc unassigned bcmills Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

4 participants