Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: broken go.mod resolution across modules #31345

Closed
y3llowcake opened this issue Apr 8, 2019 · 4 comments
Closed

cmd/go: broken go.mod resolution across modules #31345

y3llowcake opened this issue Apr 8, 2019 · 4 comments
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone

Comments

@y3llowcake
Copy link

y3llowcake commented Apr 8, 2019

What version of Go are you using (go version)?

% go version
go version go1.12.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
% go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/cy/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/cy/go"
GOPROXY=""
GORACE=""
GOROOT="/home/cy/go/go1.12.2"
GOTMPDIR=""
GOTOOLDIR="/home/cy/go/go1.12.2/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/cy/tmp/relmods/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build084314890=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I am experimenting with a multi-module project setup where one module is located in a sub-directory of another module. When executing a build/run/test command for the child directory from the parent directory, it updates the go.mod in the parent. It's not totally clear to me if this is expected behavior. At the very least it is unintuitive and makes managing a multi module repository a bit tricky.

% ls -R
.:
go.mod  submod

./submod:
go.mod  go.sum  main.go

% cat go.mod 
module relmods

go 1.12

% cat submod/main.go 
package main

import log "github.com/sirupsen/logrus"

func main() {
        log.Info("hello")
}

)% cat submod/go.mod 
module submod

go 1.12

require github.com/sirupsen/logrus v1.4.1 // indirect

% cd submod && go run main.go && cd ../
INFO[0000] hello                                        

% cat go.mod 
module relmods

go 1.12

% go run submod/main.go 
INFO[0000] hello                                        

% cat go.mod 
module relmods

go 1.12

require github.com/sirupsen/logrus v1.4.1 // indirect

What did you expect to see?

The go.mod of the parent is untouched, and the go.mod of the child is used to calculate dependencies.

What did you see instead?

The go.mod of the parent is modified.

@ALTree ALTree added this to the Go1.13 milestone Apr 9, 2019
@ALTree ALTree added modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Apr 9, 2019
@ALTree ALTree changed the title broken go.mod resolution across modules cmd/go: broken go.mod resolution across modules Apr 9, 2019
@rsc
Copy link
Contributor

rsc commented Apr 12, 2019

If you put multiple modules in a repository they are considered completely unrelated as far as requirements. The repo ends up just being a container for two independent things. In particular, when the "parent" imports something from the "child", it imports a fresh copy downloaded as any other module; it does not care that there is a working copy in a subdirectory.

Similarly, when you run go run anything.go from a particular module (in your example, the parent), what matters is the working directory of the go run command, not where the source file was found. The fact that it is in a directory with a different go.mod is not relevant.

I am skeptical that you really want to be using multiple modules in the repo. You probably don't.

@agnivade agnivade added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Apr 15, 2019
@WhiteHexagon
Copy link

Switching to vscode helped me work around this issue, since it takes directory into account when running tools.

I also use Repos as containers for multiple modules. This allows me to break down a large module/package into a few smaller modules/packages, each with it's own set of dependencies clearly documented in each go.mod.

A parent module is then just referencing sub modules for it's own dependencies. (using the go.mod 'replace' directive during development makes this a lot easier to work on the submodules at development time, but it would be nice if this work-around wasn't required).

But maybe the tools could be a bit more verbose about what they are assuming and doing. eg I had another situation where the tools were trying to download a dependency that was local only (using the 'replace' directive), and strange error messages after a module rename, when the tools were still searching for the old name. Both with multi-module projects.

Maybe I have just spent too many years working with dependency tools like Maven where parent modules is a very common pattern.

@y3llowcake
Copy link
Author

For context I am experimenting with golang 'monorepo' repository structures.

Multi-module seemed to allow owners of a service/project/component to have more control over their dependency versioning, but seems confusing and unadvised as evidenced by this bug.

A single module repo is workable for us. I think we can even achieve allowing different components to run different versions of third party dependencies by using 'replace' directives. The only annoyance is that we must manage all of this from a single file at the root of a repo, there does not seem to be a clean way to create a hierarchy that works well with an OWNERs file system.

@bcmills
Copy link
Contributor

bcmills commented May 7, 2019

@y3llowcake, this seems to be working as designed.

The main module is always the module containing the working directory of the go command invocation. When you pass a list of .go files explicitly on the command line, those files should be compiled as if they are a distinct package within the main module, and the main module's dependencies are updated such that the command you just invoked will be reproducible.

In contrast, you should find that go run ./submod produces an error due to crossing a module boundary.

@bcmills bcmills closed this as completed May 7, 2019
@golang golang locked and limited conversation to collaborators May 6, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests

7 participants