Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: 'go list -m all' doesn't diagnose conflicts in non-imported packages #41652

Closed
yihuang opened this issue Sep 27, 2020 · 5 comments
Closed

cmd/go: 'go list -m all' doesn't diagnose conflicts in non-imported packages #41652

yihuang opened this issue Sep 27, 2020 · 5 comments
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone
Unplanned

Comments

@yihuang
Copy link

yihuang commented Sep 27, 2020 •
edited

Currently vgo2nix is using go list -mod mod -json -m all to find out all the dependencies of current project, but we find this command will output conflicts modules.

A simple example:

cat > go.mod << EOF
module testmod1

go 1.15

require (
	github.com/ugorji/go v1.1.4 // indirect
	github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8 // indirect
)
EOF
cat > main.go << EOF
package main

import "fmt"

func main() {
	fmt.Println("hello world")
}
EOF

These too dependencies are in conflict, but not imported by code, so go build and go mod vendor runs just fine.
But go list -mod mod -m all will list them both.

Is it considered an issue of go list, or is there a command to serve our purpose?
@yihuang yihuang mentioned this issue Sep 27, 2020
Merged
@andybons
Copy link
Member

@jayconrod @bcmills @matloob

@andybons andybons added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 29, 2020
@andybons andybons added this to the Unplanned milestone Sep 29, 2020
@bcmills
Copy link
Contributor

bcmills commented Sep 29, 2020

go mod vendor copies in non-test dependencies matching the all package pattern. go list -m all reports the all module pattern.

The package namespace and module namespace are related, but not quite the same. The all module pattern includes (as you have seen) all of the dependencies imposed by the go.mod file, but the all package pattern (the one used by go build all and go mod vendor!) examines only those packages transitively imported by the main module. So there is no conflict in the dependencies of this module, because the only package that conflicts is not in the all package pattern.

The general solution is to give the go command the source code that you intend to build, so that go mod vendor can trace the import statements accurately. If that general solution does not suit your needs, we'll need to know more about the concrete problem you're trying to solve.

@bcmills bcmills added modules WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Sep 29, 2020
@bcmills bcmills changed the title cmd/go: go list -mod mod -m all don't return the same module list actual used when building or vendering cmd/go: 'go list -m all' doesn't diagnose conflicts in non-imported packages Sep 29, 2020
@yihuang
Copy link
Author

yihuang commented Sep 29, 2020 •
edited

The general solution is to give the go command the source code that you intend to build, so that go mod vendor can trace the import statements accurately. If that general solution does not suit your needs, we'll need to know more about the concrete problem you're trying to solve.

what vgo2nix do is try to get the locked dependency version informations from current source directory, and convert that information to nix expression and build in GOPATH mode, to get a more re-producible build (from nix perspective).
so i think maybe one possibility is to work on the all package list, and try to figure out the module list and versions?
another approach is just rely on go mod vendor, and treat the whole vendor directory as dependency, which is taken by buildGoModule, I wonder if the vgo2nix approach gonna work out?

@bcmills
Copy link
Contributor

bcmills commented Sep 29, 2020

The all package list seems like a good starting point. Once we have lazy loading (#36460), you may be able to do something simpler using the original go.mod file and replace directives.

@seankhliao seankhliao added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Dec 26, 2022
@gopherbot
Copy link

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)

@golang golang locked and limited conversation to collaborators Jan 26, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
Unplanned
Development

No branches or pull requests
5 participants
@yihuang @andybons @bcmills @gopherbot @seankhliao