cmd/go: GOFLAGS=-mod=vendor affects result of `go list -m all` #35589

pohly · 2019-11-14T16:18:17Z

What version of Go are you using (`go version`)?

$ go version
go version go1.13.4 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/pohly/.cache/go-build"
GOENV="/home/pohly/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/nvme/gopath"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/nvme/gopath/go1.13.4"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/nvme/gopath/go1.13.4/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/nvme/gopath/src/github.com/intel/pmem-csi/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build014465635=/tmp/go-build -gno-record-gcc-switches"

What did you do?

check out https://github.com/pohly/pmem-csi/tree/go-vendor-test-2019-11-14 (tag: go-vendor-test-2019-11-14)
run diff <(GOFLAGS=-mod=vendor go list -f {{.Path}} -m all) <(go list -f {{.Path}} -m all)

What did you expect to see?

There should be no difference or, if that for some reason isn't feasible, at least all modules that are in the vendor directory should be listed.

What did you see instead?

The command without GOFLAGS=-mod=vendor reports a lot more modules. For example:

$ diff <(GOFLAGS=-mod=vendor go list -f {{.Path}} -m all) <(go list -f {{.Path}} -m all)
1a2,18
> bitbucket.org/bertimus9/systemstat
> cloud.google.com/go
> github.com/Azure/azure-sdk-for-go
> github.com/Azure/go-ansiterm
> github.com/Azure/go-autorest/autorest
> github.com/Azure/go-autorest/autorest/adal
> github.com/Azure/go-autorest/autorest/date
> github.com/Azure/go-autorest/autorest/mocks
> github.com/Azure/go-autorest/autorest/to
> github.com/Azure/go-autorest/autorest/validation
> github.com/Azure/go-autorest/logger
> github.com/Azure/go-autorest/tracing
> github.com/BurntSushi/toml
> github.com/BurntSushi/xgb
> github.com/GoogleCloudPlatform/k8s-cloud-provider
> github.com/JeffAshton/win_pdh
> github.com/MakeNowJust/heredoc
...
> k8s.io/kubectl
...

Many of these extra modules are not actually used and thus not present in vendor (for example, bitbucket.org/bertimus9/systemstat).

But some that aren't reported by the first command are used (for example, k8s.io/kubectl). That's more problematic because additional unused modules can be ignored, but if the module isn't even reported, it will be missed when analyzing dependencies.

The text was updated successfully, but these errors were encountered:

andybons · 2019-11-15T18:57:36Z

@bcmills @jayconrod

bcmills · 2019-11-15T19:04:51Z

There should be no difference or, if that for some reason isn't feasible, at least all modules that are in the vendor directory should be listed.

It is, unfortunately, not possible for go list to compute a correct, complete version of all using the information in the vendor directory. In Go 1.14, go list -m will explicitly reject wildcard patterns when -mod=vendor is in use. (That is already implemented.)

go list -m will continue to be able to list any module that provides an actual package in the vendor directory.

bcmills · 2019-11-15T19:05:51Z

Addressed by CL 199821 and CL 203138. (See also #33848.)

jayconrod · 2019-11-15T19:06:18Z

In general, go list -m all cannot give complete results with -mod=vendor. Listing all modules requires loading the module graph, and we don't save enough information in go.mod and vendor/modules.txt to be able to do that.

In Go 1.14, the following message is printed:

go list -m: can't compute 'all' using the vendor directory
	(Use -mod=mod or -mod=readonly to bypass.)

@bcmills Maybe this is a documentation issue? go help list should probably explain the interaction when describing -m.

bcmills · 2019-11-15T19:07:38Z

@jayconrod, it may be that we should update the documentation, although I doubt that anyone will read it ahead of time and I think the error message mostly speaks for itself.

workaround for golang/go#35589

* test/edge: Delete dead code Signed-off-by: Xiang Dai <long0dai@foxmail.com> * return details info to edge node when registration failed Signed-off-by: Kevin Zhao <kevin.zhao@linaro.org> * print reason from cloud core * fix_connection * Document volume support Signed-off-by: Xiang Dai <long0dai@foxmail.com> * workaround fix on cloudcore crash of device without nodeselector * Fix issue 1905. Changes: 1. Refine some function names. Those function names use past tense which are unreasonable. Refine them to present tense. 2. Refine some variable names. Device and device model are two different definitions. In this case, it's device. * Added JYLink to adopter Added Jingying Shuzhi Technology Co., Ltd to adopter * Fix issue 1905 Changes: Refine some function names. Those function names use past tense which are unreasonable. Refine them to present tense. Refine some variable names. Device and device model are two different definitions. In this case, it's device. * fix use certificate from local directory problem * Fix Modbus Mapper Metric Param Error. * fix downstream * upate go version * updating vendor * updated sigs.k8s.io/kind * updated sigs.k8s.io/kind * updating go modules * exited programm if command failed * remove warning incosistent vendoring * update vendor * cleaning up vendor * fix go list * Refine a function name. * travis workaround go1.14 issue workaround for golang/go#35589 * add cert rotation * Bump lodash from 4.17.14 to 4.17.19 in /mappers/modbus_mapper/src Bumps [lodash](https://github.com/lodash/lodash) from 4.17.14 to 4.17.19. - [Release notes](https://github.com/lodash/lodash/releases) - [Commits](lodash/lodash@4.17.14...4.17.19) Signed-off-by: dependabot[bot] <support@github.com> * add reportCycle and collectCycle in property vistor * add customized protocol support * add data section * re-org device model and device instance * tweak log info Signed-off-by: Kevin Zhao <kevin.zhao@linaro.org> * EdgeMesh: remove useless scripts Signed-off-by: Xiang Dai <long0dai@foxmail.com> * Lint: check broken links Signed-off-by: Xiang Dai <long0dai@foxmail.com> * Fix issue 1824 The action in one schedule can be more than one. "actionExists" will always be true once it enters line 85, which is not expected. Morever, if one action cannot be found, other actions should be run. * upgrade device api from v1alpha1 to v1alpha2 * move data section from top level into Spec section * enhance on device protocol config * Fix an incorrect link to keadm doc * Update eventbus.md * edged implement GetExec funtion Signed-off-by: kadisi <iamkadisi@163.com> * Docs: fix all dead links Signed-off-by: Xiang Dai <long0dai@foxmail.com> * Fix device configMap can not be deleted Signed-off-by: Xiang Dai <long0dai@foxmail.com> * The same info is used by several files. Refine them all. Signed-off-by: sailorvii <challengingway@hotmail.com> * add testcase Signed-off-by: Xiang Dai <long0dai@foxmail.com> * update edge authentication proposal about cert rotation * Add proposal for device management enhancement * fix typos in docs * move property vistor codes to device_instance_types * Remove redundant code * Fix: print token with newline * Use Fprintln to print token * get new device crd and fix keadm ci issue * Fix const int to string issue Signed-off-by: Xiang Dai <long0dai@foxmail.com> * Docs: upgrade device crds version Signed-off-by: Xiang Dai <long0dai@foxmail.com> * add v1alpha1 device crd back * Fix wrong ContainerRuntimeType in display * Bump up csidriver example image version Signed-off-by: Xiang Dai <long0dai@foxmail.com> * Keadm: add test when use `keadm gettoken` in cmdline Signed-off-by: Xiang Dai <long0dai@foxmail.com> * Lint: add more help info Signed-off-by: Xiang Dai <long0dai@foxmail.com> * clean up unused constants * update node limit default value * fix typo * Add kill signal support Signed-off-by: Xiang Dai <long0dai@foxmail.com> * make lint work in Mac and fix lint issues * lint: address review comment * Move the setting out of the rotation. * Give the capacity of slice and map * fixed goroutine leak, when ws connection reconnect, but fifo's channel not close * extend property types * Refine the error messages to errors.New. * Update upstream dependency to K8s 1.18.5 Co-authored-by: Yin Ding <dingyin@gmail.com> Co-authored-by: Rio Zhu <32083634+zzxgzgz@users.noreply.github.com> * Updated k8s to 1.18.5 and modified code around API changes. * Updated kube-scheduler dependency. * added filed ExperimentalTopologyManagerPolicy to matched the upgraded k8s * Changed some dependencies' versions to x.18.0 * Added function to start cpu manager in clcm. * update vendor to k8s 1.18 * Lint: fix formats etc. * add license file * skip structured-merge-diff license check this package doesn't exist, but has v3 subdirectory as a different package so it can't be filtered by the previous rule, temporarily treat this way until find out a better rule. * update generated code * update to k8s 1.18, fix more context issues * Lint: fix formats * R1186 (#8) * Upgraded k8s to x.18.6 * Updated vendor folder * added files in vendor * lint: clean up go module * update vendor * fix an error from merged pr * update client-go storage lister fix mount host path error * lint: empty lines * StartCPUManager when edged start * Refine the edgecontroller message group name. * tunnel cert optimization Co-authored-by: Xiang Dai <long0dai@foxmail.com> Co-authored-by: Kevin Zhao <kevin.zhao@linaro.org> Co-authored-by: Poor12 <1185060656@qq.com> Co-authored-by: KubeEdge Bot <48982446+kubeedge-bot@users.noreply.github.com> Co-authored-by: luogangyi <luogangyi@cmsr.chinamobile.com> Co-authored-by: sailorvii <challengingway@hotmail.com> Co-authored-by: coversb <chenhao_haha@163.com> Co-authored-by: threestoneliu <liuzhilei93@outlook.com> Co-authored-by: zongke <zongk@shangyuantech.com> Co-authored-by: guojianyu <276381225@qq.com> Co-authored-by: Jan Unterbrink <jan.unterbrink@subpath.de> Co-authored-by: Xujiejie <406347136@qq.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: wzharies <861613130@qq.com> Co-authored-by: sailorvii <35483373+sailorvii@users.noreply.github.com> Co-authored-by: kadisi <iamkadisi@163.com> Co-authored-by: ZYZ <yaozhongzyz@126.com> Co-authored-by: dwq <41563853+dddddai@users.noreply.github.com> Co-authored-by: ESWZY <1006460505@qq.com> Co-authored-by: xufei 00416946 <xufei40@huawei.com> Co-authored-by: Qiang Zheng <zhengqianglc@inspur.com> Co-authored-by: Yin Ding <dingyin@gmail.com> Co-authored-by: dddddai <853238012@qq.com> Co-authored-by: loongify <273368612@qq.com> Co-authored-by: Rio Zhu <32083634+zzxgzgz@users.noreply.github.com> Co-authored-by: Liangshuang <liangshuang889@163.com>

workaround for golang/go#35589

pohly mentioned this issue Nov 14, 2019

Go 1.13 + Moving to go modules intel/pmem-csi#461

Merged

andybons added GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Nov 15, 2019

andybons added this to the Unplanned milestone Nov 15, 2019

bcmills modified the milestones: Unplanned, Go1.14 Nov 15, 2019

bcmills self-assigned this Nov 15, 2019

bcmills closed this as completed Nov 15, 2019

sgnn7 mentioned this issue Jan 6, 2020

Non-vendored Go modules are not properly detected pivotal/LicenseFinder#654

Open

subpathdev pushed a commit to subpathdev/kubeedge that referenced this issue Jul 14, 2020

travis workaround go1.14 issue

9a5738a

workaround for golang/go#35589

subpathdev pushed a commit to subpathdev/kubeedge that referenced this issue Jul 14, 2020

travis workaround go1.14 issue

5c795fd

workaround for golang/go#35589

Congrool pushed a commit to Congrool/kubeedge that referenced this issue Aug 4, 2020

travis workaround go1.14 issue

760791b

workaround for golang/go#35589

Congrool pushed a commit to Congrool/kubeedge that referenced this issue Aug 4, 2020

travis workaround go1.14 issue

4c58f44

workaround for golang/go#35589

sailorvii pushed a commit to sailorvii/kubeedge that referenced this issue Oct 27, 2020

travis workaround go1.14 issue

37e68ec

workaround for golang/go#35589

golang locked and limited conversation to collaborators Nov 14, 2020

gopherbot added the FrozenDueToAge label Nov 14, 2020

rsc unassigned bcmills Jun 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: GOFLAGS=-mod=vendor affects result of `go list -m all` #35589

cmd/go: GOFLAGS=-mod=vendor affects result of `go list -m all` #35589

pohly commented Nov 14, 2019 •

edited

andybons commented Nov 15, 2019

bcmills commented Nov 15, 2019

bcmills commented Nov 15, 2019 •

edited

jayconrod commented Nov 15, 2019

bcmills commented Nov 15, 2019

cmd/go: GOFLAGS=-mod=vendor affects result of go list -m all #35589

cmd/go: GOFLAGS=-mod=vendor affects result of go list -m all #35589

Comments

pohly commented Nov 14, 2019 • edited

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

andybons commented Nov 15, 2019

bcmills commented Nov 15, 2019

bcmills commented Nov 15, 2019 • edited

jayconrod commented Nov 15, 2019

bcmills commented Nov 15, 2019

cmd/go: GOFLAGS=-mod=vendor affects result of `go list -m all` #35589

cmd/go: GOFLAGS=-mod=vendor affects result of `go list -m all` #35589

pohly commented Nov 14, 2019 •

edited

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

bcmills commented Nov 15, 2019 •

edited