Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: cannot find module for path #32886

Closed
triztian opened this issue Jul 2, 2019 · 18 comments
Closed

cmd/go: cannot find module for path #32886

triztian opened this issue Jul 2, 2019 · 18 comments
Labels
Documentation FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.

Comments

@triztian
Copy link

triztian commented Jul 2, 2019

What version of Go are you using (go version)?

go version go1.12.6 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN="/Users/tristian/go/bin"
GOCACHE="/Users/tristian/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/tristian/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12.6/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12.6/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/tristian/Repositories/github.com/triztian/my.binary.project/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/g1/v3hz1sm92nv7wbzpg3dx_h_h0000gp/T/go-build549198414=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Tried to build the executable after running rm -rf $GOPATH/pkg, I then ran go mod download and tried to do a build:

λ ~/Repositories/code.blockwave.com/blockwave/core.bin: go build -x -a -v
WORK=/var/folders/g1/v3hz1sm92nv7wbzpg3dx_h_h0000gp/T/go-build723678509
build code.blockwave.com/blockwave/core.bin: cannot find module for path code.blockwave.com/blockwave/core.components.engine/wave/plugin-master

After getting that error message, I ensured that the module is fetched and downloaded:

 λ ~/Repositories/code.blockwave.com/blockwave/core.bin: go get code.blockwave.com/blockwave/core.components.engine/wave/plugin-master
go: finding code.blockwave.com/blockwave/core.components.engine/wave/plugin-master latest
go: finding code.blockwave.com/blockwave/core.components.engine/wave latest
go: finding code.blockwave.com/blockwave/core.components.engine latest

Then I retried the build:

 λ ~/Repositories/code.blockwave.com/blockwave/core.bin: go build -x -a -v
WORK=/var/folders/g1/v3hz1sm92nv7wbzpg3dx_h_h0000gp/T/go-build629362765
build code.blockwave.com/blockwave/core.bin: cannot find module for path code.blockwave.com/blockwave/core.components.engine/wave/plugin-master

Same result, so I now tried cleaning the cache:

 λ ~/Repositories/code.blockwave.com/blockwave/core.bin: go clean -modcache
build code.blockwave.com/blockwave/core.bin: cannot find module for path code.blockwave.com/blockwave/core.components.engine/wave/plugin-master

I've also ran go mod verify, and it reports no issues:

λ ~/Repositories/code.blockwave.com/blockwave/core.bin (issue/blockwave.trunk.11): go mod verify
all modules verified

What did you expect to see?

I expected to see a core.bin executable in my current directory.

What did you see instead?

build code.blockwave.com/blockwave/core.bin: cannot find module for path code.blockwave.com/blockwave/core.components.engine/wave/plugin-master

I tried to reproduce the issue and to setup a project that mimics the structure of the one that I'm experiencing issues with but it seems to be specific to my setup:

However with such setup I'm able to produce a valid executable:

 λ ~/Repositories/github.com/triztian/my.binary.project (master): go build -v
 λ ~/Repositories/github.com/triztian/my.binary.project (master): ls
build_commands.txt	go.mod			main.go
cmd			go.sum			my.binary.project

And it runs correctly:

./my.binary.project
42

Is there a way for me to get more information about the specific package that is failing?

@triztian
Copy link
Author

triztian commented Jul 2, 2019

I've updated my go.mod file's module line to not have a ".":

module code.blockwave.com/blockwave/blockwave

and no success, same error message:

build code.blockwave.com/blockwave/blockwave: cannot find module for path code.blockwave.com/blockwave/core.components.engine/wave/plugin-master

@triztian
Copy link
Author

triztian commented Jul 2, 2019

I've cloned the go repo and set the printStackInDie variable to true and basically the call to loaded.pkgCache.Get returns false:

cmd/go/internal/modload/build.go
(Line ~246 ish)

// findModule returns the module containing the package at path,
// needed to build the package at target.
func findModule(target, path string) module.Version {
	pkg, ok := loaded.pkgCache.Get(path).(*loadPkg)
	if ok {
		if pkg.err != nil {
			base.Fatalf("build %v: cannot load %v: %v", target, path, pkg.err)
		}
		return pkg.mod
	}

	if path == "command-line-arguments" {
		return Target
	}

	if printStackInDie {
		debug.PrintStack()
	}
	base.Fatalf("build %v: cannot find module for path %v", target, path)
	panic("unreachable")
}

Trying to trace the moment when my code.blockwave.com/blockwave/core.components.engine/wave/plugin-master package gets added to that cache and see how it gets added and why it's not found again ... maybe it's not added at all or it's added under a different key?

@jayconrod
Copy link
Contributor

The cannot find module for path error message is shown when a package doesn't exist or can't be found.

You mentioned a few commands that were failing. go get code.blockwave.com/blockwave/core.components.engine/wave/plugin-master has the full package path, so let's focus on that. I'm assuming this is part of your main module, and not another module dependency.

  • What directory is the go.mod file in? Did you run this command from that directory, a subdirectory, or somewhere outside that tree?
  • You mentioned you changed the module path to code.blockwave.com/blockwave/blockwave. This is not a prefix of code.blockwave.com/blockwave/core.components.engine/wave/plugin-master though. Should it be code.blockwave.com/blockwave instead?
  • Assuming the module path is code.blockwave.com/blockwave, is there a subdirectory core.components.engine/wave/plugin-master within the directory containing the go.mod file? For example, if your go.mod file is ~/Repositories/code.blockwave.com/blockwave, there should be a directory ~/Repositories/code.blockwave.com/blockwave/core.components.engine/wave/plugin-master.

@jayconrod jayconrod added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jul 2, 2019
@triztian
Copy link
Author

triztian commented Jul 2, 2019

Morning @jayconrod thank you for the reply. Yes code.blockwave.com/blockwave/core.components.engine/wave/plugin-master is a sub-package of the code.blockwave.com/blockwave/core.components.engine module, it declares the module as such:

 λ ~/Repositories/code.blockwave.com/blockwave/core.components.engine (master): head -n3 go.mod
module code.blockwave.com/blockwave/core.components.engine

go 1.12

1. What directory is the go.mod file in?

In the case of code.blockwave.com/blockwave/blockwave (the project that produces the executable) the go.mod file is under $HOME/Repositories/code.blockwave.com/blockwave/blockwave/, in the case of the dependency that contains the troublesome module the go.mod file is under $HOME/Repositories/code.blockwave.com/blockwave/core.components.engine/

2. You mentioned you changed the module path to code.blockwave.com/blockwave/blockwave. This is not a prefix of code.blockwave.com/blockwave/core.components.engine/wave/plugin-master though. Should it be code.blockwave.com/blockwave instead?

Not for this situation, in our repo server, code.blockwave.com is the root host name and blockwave is a group; based on that theblockwave/blockwave means that the blockwave project lives under the blockwave group, for the dependency it means that the core.components.engine lives under the blockwave group, basically on disk I mimic the org on the repo server and it looks like this:

.
├── blockwave
│   ├── blockwave
│   ├── core.components.engine

Both directories each contain their own go.mod files and are accessible using go get from different projects.

3. Assuming the module path is code.blockwave.com/blockwave, is there a subdirectory core.components.engine/wave/plugin-master within the directory containing the go.mod file? For example, if your go.mod file is ~/Repositories/code.blockwave.com/blockwave, there should be a directory ~/Repositories/code.blockwave.com/blockwave/core.components.engine/wave/plugin-master.

Yes a directory that contains the plugin-master package exists:

 λ ~/Repositories/code.blockwave.com: ls -la blockwave/core.components.engine/wave/plugin-master/
total 40
drwxr-xr-x   8 tristian  staff   256 Jun 14 18:31 .
drwxr-xr-x  10 tristian  staff   320 Jun 14 18:31 ..
-rw-r--r--   1 tristian  staff  8196 Jun 14 18:31 .DS_Store
-rw-r--r--   1 tristian  staff  1389 Jun 14 18:31 loader.go

I can cd into it and build it:

 λ ~/Repositories/code.blockwave.com/blockwave/core.components.engine/wave/plugin-master (master): go build -v -x .
WORK=/var/folders/g1/v3hz1sm92nv7wbzpg3dx_h_h0000gp/T/go-build001579710

But if I try to build it from the root module core.components.engine it fails:

λ ~/Repositories/code.blockwave.com/blockwave/core.components.engine (master): go build -x -a ./blockwave/core.components.engine/wave/plugin-master
WORK=/var/folders/g1/v3hz1sm92nv7wbzpg3dx_h_h0000gp/T/go-build061459711
can't load package: package code.blockwave.com/blockwave/core.components.engine/blockwave/core.components.engine/wave/plugin-master: unknown import path "code.blockwave.com/blockwave/core.components.engine/blockwave/core.components.engine/wave/plugin-master": cannot find package

@jayconrod
Copy link
Contributor

Ah, ok, so code.blockwave.com/blockwave/core.components.engine is a separate module, and the executable builds correctly within that module but not in code.blockwave.com/blockwave/blockwave.

Do you have any requirements or replacements in the go.mod for code.blockwave.com/blockwave/blockwave related to code.blockwave.com/blockwave/core.components.engine?

Assuming the module is not already required and is not replaced, the Go command will clone the git repository for this module, then will look at the latest semantic version tag ("latest" as defined in go help module-get). This happens within the module cache in $GOPATH/pkg/mod; the other local repository is not consulted.

You can see which version is selected with go list -m -json code.blockwave.com/blockwave/core.components.engine. Do you know if there is a wave/plugin-master subdirectory in that repository at that version?

@triztian
Copy link
Author

triztian commented Jul 2, 2019

@jayconrod Thank you for the help!.

so code.blockwave.com/blockwave/core.components.engine is a separate module, and the executable builds correctly within that module but not in code.blockwave.com/blockwave/blockwave.

I'd say it's more like code.blockwave.com/blockwave/core.components.engine is a library module (no executable produced) and I'm able to successfully build it if I cd into the plugin-master (the "missing" package) directory and do the build from there:

 λ ~/Repositories/code.blockwave.com/blockwave/core.components.engine/wave/plugin-master (master): go build -v && echo $?
0

It also succeeds if I build it from the root of the module (where go.mod lives):

 λ ~/Repositories/code.blockwave.com/blockwave/core.components.engine (master): go build -v ./wave/plugin-master/ && echo $?
0

But if fails to build or be found if I import it in to another module (like blockwave/blockwave)

Do you have any requirements or replacements in the go.mod for code.blockwave.com/blockwave/blockwave related to code.blockwave.com/blockwave/core.components.engine?

Nope, no replacement directives; I have one but it's for a local dependency that is not related to core.components.engine if I remove it I get the same error:

build code.blockwave.com/blockwave/blockwave: cannot find module for path code.blockwave.com/blockwave/core.components.engine/wave/plugin-master

You can see which version is selected with go list -m -json code.blockwave.com/blockwave/core.components.engine. Do you know if there is a wave/plugin-master subdirectory in that repository at that version?

Yes, here is the output:

 λ ~/Repositories/code.blockwave.com/blockwave/blockwave (issue/blockwave.trunk.11): go list -m -json code.blockwave.com/blockwave/core.components.engine
{
	"Path": "code.blockwave.com/blockwave/core.components.engine",
	"Version": "v0.0.0-20190627051003-0b1a2b723317",
	"Time": "2019-06-27T05:10:03Z",
	"Dir": "/Users/tristian/go/pkg/mod/code.blockwave.com/blockwave/core.components.engine@v0.0.0-20190627051003-0b1a2b723317",
	"GoMod": "/Users/tristian/go/pkg/mod/cache/download/code.blockwave.com/blockwave/core.components.engine/@v/v0.0.0-20190627051003-0b1a2b723317.mod",
	"GoVersion": "1.12"
}

@jayconrod
Copy link
Contributor

Does the directory wave/plugin-master exist at commit 0b1a2b723317? The Go command will look for it in /Users/tristian/go/pkg/mod/code.blockwave.com/blockwave/core.components.engine@v0.0.0-20190627051003-0b1a2b723317/wave/plugin-master.

@triztian
Copy link
Author

triztian commented Jul 2, 2019

Yes, and it' is pushed and accessible from the remote repo:

 λ ~/Repositories/code.blockwave.com/blockwave/core.components.engine (master): git show --summary --oneline
0b1a2b7 (HEAD -> master, origin/master, origin/HEAD) Merge branch 'master' of code.blockwave.com:blockwave/core.components.engine

And it does exist under $GOPATH/pkg:

 λ ~/go/pkg/mod/code.blockwave.com/blockwave/core.components.engine@v0.0.0-20190627051003-0b1a2b723317/wave/plugin-master: ls -la
total 40
dr-xr-xr-x   8 tristian  staff   256 Jul  1 18:56 .
dr-xr-xr-x  10 tristian  staff   320 Jul  1 18:56 ..
-r--r--r--   1 tristian  staff  8196 Jul  1 18:56 .DS_Store
-r--r--r--   1 tristian  staff  1390 Jul  1 18:56 loader.go

@jayconrod
Copy link
Contributor

Hmm, strange. Anything usual about that file? For example, any build tags that would exclude it from being built on the target platform?

If you run go list code.blockwave.com/blockwave/core.components.engine/... from the other module (assuming code.blockwave.com/blockwave/core.components.engine is required in go.mod at version v0.0.0-20190627051003-0b1a2b723317), are there other packages listed there, but just not that one?

@triztian
Copy link
Author

triztian commented Jul 2, 2019

If run go list code.blockwave.com/blockwave/core.components.engine/... from the module that produces the executable (blockwave/blockwave) I get the following list:

 λ ~/Repositories/code.blockwave.com/blockwave/blockwave (issue/blockwave.trunk.11): go list code.blockwave.com/blockwave/core.components.engine/... | sort -r
code.blockwave.com/blockwave/core.components.engine/wave/plugin-master/virtualfactory
code.blockwave.com/blockwave/core.components.engine/wave/plugin-master/virtualblock
code.blockwave.com/blockwave/core.components.engine/wave/plugin-master
# ... filtered output ...

And code.blockwave.com/blockwave/core.components.engine is a dep in go.mod:

module code.blockwave.com/blockwave/blockwave

go 1.12

exclude github.com/go-yaml/yaml v0.0.0-20181115110504-51d6538a90f8

require (
    // ...
    code.blockwave.com/blockwave/core.components.engine v0.0.0-20190627051003-0b1a2b723317
    // ...
)

The code.blockwave.com/blockwave/core.components.engine/wave/plugin-master package is a really simple package its a single loader.go file and it just exports a function and imports the following and has no build tags:

package pluginmaster

import (
    "path/filepath"

    "github.com/pkg/errors"
)

triztian pushed a commit to triztian/go that referenced this issue Jul 2, 2019
@triztian
Copy link
Author

triztian commented Jul 2, 2019

Not sure if it provides additional insights but I've patched/butchered a bit the Go source as depicted here:

And the missing package code.blockwave.com/blockwave/core.components.engine/wave/plugin-master is not added to the file that logs the cache calls, i.e., it never makes it here:

You can change the location of the file by changing this line, so whatever calls pkg is skipping the plugin-master dependency (ah yes private repo sorry 😄 )

@jayconrod
Copy link
Contributor

It's curious that go list finds that package but go get and go build do not. At the point where this error is emitted, they run the same code.

What happens with go build code.blockwave.com/blockwave/core.components.engine/...?

Are you able to reduce this to a reproducible example? code.blockwave.com/blockwave/core.components.engine is not public, so I can't debug this directly.

Does this reproduce with go1.13beta1?

If you're interested in debugging this further yourself, ImportPaths in cmd/go/internal/modload is the entry point for converting command line arguments to packages and modules that provide them. loader.doPkg in the same file processes individual packages.

@triztian
Copy link
Author

triztian commented Jul 2, 2019

What happens with go build code.blockwave.com/blockwave/core.components.engine/...?

When I run go build code.blockwave.com/blockwave/core.components.engine/... from the dependency directory it builds successfully:

λ ~/Repositories/code.blockwave.com/blockwave/core.components.engine (master): go build code.blockwave.com/blockwave/core.components.engine/...
λ ~/Repositories/code.blockwave.com/blockwave/core.components.engine (master): echo $?
0

When I run it from the project that produces the executable (I've reverted back to the original core.bin name, blockwave/blockwave == blockwave/core.bin):

λ ~/Repositories/code.blockwave.com/blockwave/core.bin (issue/blockwave.trunk.11): go build code.blockwave.com/blockwave/core.components.engine/...
λ ~/Repositories/code.blockwave.com/blockwave/core.bin (issue/blockwave.trunk.11): go build .
build code.blockwave.com/blockwave/core.bin: cannot find module for path code.blockwave.com/blockwave/core.components.engine/wave/plugin-master

and it fails.

Are you able to reduce this to a reproducible example?

I tried reproducing it here, to no avail:

Thinking that the "." that we have as part of the module names would cause an issue, but that project builds successfully ..., do you think it could be the 2 "." in code.blockwave.com that's causing it to not find it or skip it?

Does this reproduce with go1.13beta1?

Will give it a go

@triztian
Copy link
Author

triztian commented Jul 2, 2019

Just tried with go1.13beta1 and it is reproducible with it:

 λ ~/Repositories/code.blockwave.com/blockwave/core.bin (issue/blockwave.trunk.11): go1.13beta1 build -x -v .
WORK=/var/folders/g1/v3hz1sm92nv7wbzpg3dx_h_h0000gp/T/go-build223193417
build code.blockwave.com/blockwave/core.bin: cannot find module for path code.blockwave.com/blockwave/core.components.engine/wave/plugin-master

For reference I followed this instructions to install it:

@triztian
Copy link
Author

triztian commented Jul 2, 2019

After moving the fmt.Fprintf call to the doPkg function, the function never receives the plugin-master package as something to resolve:

If I search the cachecalls.txt file for such entry it does not show up:

λ ~/Workspace: grep plugin-master cachecalls.txt
λ ~/Workspace:

For reference, other packages do show up:

λ ~/Workspace: ag fmt cachecalls.txt
8:doPkg: path: fmt
22:doPkg: path: internal/fmtsort

As a side note ... is there a faster way to do Go builds from source without running all the tests?, I'm currently building after my changes using ./all.bash ...

A nvm found a way:

If you do not want to run the test suite use make.bash (or make.bat) instead.

@triztian
Copy link
Author

triztian commented Jul 2, 2019

@jayconrod I was able to reproduce it!!!, thank you so much for your time and guidance. 😃

I've pushed an update to the project:

Basically just clone it and run go build and you should see it happen:

 λ ~/Repositories/github.com/triztian/my.binary.project (master): go build
build github.com/triztian/my.binary.project: cannot find module for path github.com/triztian/my.special.library/really/deep/nested/sub-package

The fix for this is basically applying the following patch, after that it builds successfully:

diff --git a/cmd/func.go b/cmd/func.go
index 4350b06..4f756e5 100644
--- a/cmd/func.go
+++ b/cmd/func.go
@@ -1,4 +1,4 @@
-// +build !some.constraint.with-dash
+// +build !some.constraint.withdash
 
 package cmd
 
diff --git a/cmd/func_stub.go b/cmd/func_stub.go
index c8abfa9..78259fb 100644
--- a/cmd/func_stub.go
+++ b/cmd/func_stub.go
@@ -1,4 +1,4 @@
-// +build some.constraint.with-dash
+// +build some.constraint.withdash
 
 package cmd

In the case of my source I had a build constraint that looked like this: xcore.blocks.native-exec which triggered the issue.

If not allowing for build constraints to have dashes is the desired behavior I think the documentation should be updated to mention that limitation:

If build constraints with dashes are allowed then this would be the bug/issue right?, should a new issue be opened if it's the case?

The way I figured it out was because the file that had the constraint with dashes was excluded from adding it's declared imports

Basically plugin-master package would not be part of it so basically the build tags were not on the dependency but on the file that imported the dependency.

@jayconrod
Copy link
Contributor

Glad you figured it out. Indeed, dashes are not allowed in build tags. Agreed that the documentation should be updated.

@jayconrod jayconrod added Documentation NeedsFix The path to resolution is known, but the work has not been done. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Jul 3, 2019
@gopherbot
Copy link

Change https://golang.org/cl/184917 mentions this issue: go/build: document characters allowed in build tags

@golang golang locked and limited conversation to collaborators Jul 2, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Documentation FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

3 participants