Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: 'go mod tidy' does not add indirect private dependencies #28397

Closed
thanasik opened this issue Oct 25, 2018 · 7 comments
Closed

cmd/go: 'go mod tidy' does not add indirect private dependencies #28397

thanasik opened this issue Oct 25, 2018 · 7 comments
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@thanasik
Copy link

What version of Go are you using (go version)?

go version go1.11.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/karysto/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/karysto/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build201986556=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I have a private repo I'm working on called parent_repo, it imports another private repo I wrote called child_repo. child_repo imports another private repo I wrote called grandchild_repo. All of these repos have go.mod files. In this case, grandchild_repo is an indirect dependency of parent_repo.

When I run go mod tidy on parent_repo, child_repo appears, but grandchild_repo does not. After adding grandchild_repo manually to parent_repo's mod file (with an associated replace directive to find it locally), go mod vendor and go build -mod=vendor works perfectly, the build succeeds. The problem is that if I do go mod tidy on parent_repo, my manually added grandchild_repo entry is removed, and my builds fail.

My use case is wanting to automate gathering dependencies to vendor before running a CI pipeline, but I can't right now because of this issue.

What did you expect to see?

I expected to see go mod tidy add my indirect dependency to go.mod (so that I didn't have to add it manually) and the build to succeed as a result.

What did you see instead?

I see go.mod tidy remove the grandchild dependency and the build failing

@andybons
Copy link
Member

@bcmills

@gopherbot needsinvestigation modules

@andybons andybons added this to the Unplanned milestone Oct 26, 2018
@andybons andybons added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. modules labels Oct 26, 2018
@myitcv
Copy link
Member

myitcv commented Oct 26, 2018

@karysto taking your example; given child_repo is a module, then its dependencies (including grandchild_repo) will not appear in parent_repo. The dependency relationship still exists: go list -m all will show you all your dependencies in case you need to see that list.

This is working as intended.

@myitcv myitcv closed this as completed Oct 26, 2018
@thanasik
Copy link
Author

thanasik commented Oct 26, 2018

@myitcv thanks for taking a look. If the required dependencies (in this case grandchild_repo) aren't put into in go.mod by go mod tidy, then go build fails. Is there a workflow or tool I'm missing that automates adding these sub-dependencies to go.mod to allow the build to succeed?

@thepudds
Copy link
Contributor

@karysto sorry for just a quick comment, but does it seem like #26241 might be related?

Also, would you be able to put together 3 or so short but representative go.mod files that capture the essence of what you are seeing?

(A full blown repository on github that shows what you are seeing would be even better, but maybe to start some sample go.mod files could help make the nature of the problem more concrete).

@myitcv
Copy link
Member

myitcv commented Oct 28, 2018

@karysto If you're seeing a build failure then there is something else going on, because grandchild should not be required in the parent's go.mod:

$ cd $(mktemp -d)
$ mkdir parent child grandchild
$ pushd grandchild
/tmp/tmp.VJ58bS5pTj/grandchild /tmp/tmp.VJ58bS5pTj
$ go mod init example.com/grandchild
go: creating new go.mod: module example.com/grandchild
$ cat <<EOD >main.go
package grandchild

func Count() int {
        return 5
}
EOD
$ go mod tidy
$ popd
/tmp/tmp.VJ58bS5pTj
$ pushd child
/tmp/tmp.VJ58bS5pTj/child /tmp/tmp.VJ58bS5pTj
$ go mod init example.com/child
go: creating new go.mod: module example.com/child
$ cat <<EOD >main.go
package child

import "example.com/grandchild"

func Children() int {
        return grandchild.Count()
}
EOD
$ go mod edit -require=example.com/grandchild@v0.0.0 -replace=example.com/grandchild=$PWD/../grandchild
$ go mod tidy
$ cat go.mod
module example.com/child

require example.com/grandchild v0.0.0

replace example.com/grandchild => /tmp/tmp.VJ58bS5pTj/child/../grandchild
$ popd
/tmp/tmp.VJ58bS5pTj
$ pushd parent
/tmp/tmp.VJ58bS5pTj/parent /tmp/tmp.VJ58bS5pTj
$ go mod init example.com/parent
go: creating new go.mod: module example.com/parent
$ cat <<EOD >main.go
package main

import (
        "example.com/child"
        "fmt"
)

func main() {
        fmt.Printf("My child has %[1]v children; hence I have %[1]v grand children\n", child.Children())
}
EOD
$ go mod edit -require=example.com/child@v0.0.0 -replace=example.com/grandchild=$PWD/../grandchild -replace=example.com/child=$PWD/../child
$ go mod tidy
$ cat go.mod
module example.com/parent

require example.com/child v0.0.0

replace example.com/grandchild => /tmp/tmp.VJ58bS5pTj/parent/../grandchild

replace example.com/child => /tmp/tmp.VJ58bS5pTj/parent/../child
$ go run .
My child has 5 children; hence I have 5 grand children

Notice I'm using replace directives here (per @thepudds' comment above) because I haven't published the dependencies. Hence I do need a replace for grandchild until I do publish them, but not a require.

@thanasik
Copy link
Author

@thepudds thanks for the response, I'll setup a repo demonstrating the issue so that it can be reproduced.

@myitcv I agree, grandchild_repo should not be needed, but for some reason the build fails if it's not included in the parent_repo go.mod. I have the set of replace directives for the local dependencies in the go.mod as well, the issue persists despite them. Do you have an idea of what other possible issue this could be related to that I might be missing?

@myitcv
Copy link
Member

myitcv commented Oct 31, 2018

@karysto

Do you have an idea of what other possible issue this could be related to that I might be missing?

Not without a repro, no.

@golang golang locked and limited conversation to collaborators Oct 31, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

5 participants