Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go/build: Context.Import no longer returns a NoGoError with -mod=vendor #58903

Closed
dprotaso opened this issue Mar 6, 2023 · 8 comments
Closed
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.

Comments

@dprotaso
Copy link

dprotaso commented Mar 6, 2023

What version of Go are you using (go version)?

go version go1.20.1 darwin/amd64

Does this issue reproduce with the latest release?

Yes,

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
go version go1.20.1 darwin/amd64
dprotasowski@dprotasowsBPN7C eventing % go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/dprotasowski/Library/Caches/go-build"
GOENV="/Users/dprotasowski/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/dprotasowski/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/dprotasowski/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.20.1/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.20.1/libexec/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.20.1"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/dprotasowski/work/eventing/go.mod"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/xs/xfp1z4cn643c46w69lv8bv8w0000gp/T/go-build3514065930=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Some tooling uses go/build::Context.ImportDir to collect type/package info to use to generate docs and code. ahmetb/gen-crd-api-reference-docs#61

https://github.com/kubernetes/gengo/blob/ab3349d207d4cc9f4bc5c37ba6aa062b4b2ffacc/parser/parse.go#L157

What did you expect to see?

Tool runs

What did you see instead?

Starting with go1.20 these tools started to fail. Creating a simple repro example the error from go/build is

cannot query module due to -mod=vendor
	(Go version in go.mod is at least 1.14 and vendor directory exists.)

Repro case (go1.19):

$ git clone https://github.com/dprotaso/eventing.git
$ cd eventing
$ git checkout broken-go-build
$ go run main.go
panic: cannot query module due to -mod=vendor
	(Go version in go.mod is at least 1.14 and vendor directory exists.)

goroutine 1 [running]:
main.main()
	/Users/dprotasowski/work/eventing/main.go:13 +0xba
exit status 2

With 1.19 it panics it doesn't return any packages (which is fine because the above linked tools handle this case)

panic: no buildable Go source files in /eventing/pkg/apis

goroutine 1 [running]:
main.main()
	/eventing/main.go:13 +0xba
exit status 2
@bcmills bcmills changed the title affected/package: go/build Context.Import - cannot query module due to -mod=vendor go/build: Context.Import no longer returns a NoGoError with -mod=vendor Mar 6, 2023
@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 6, 2023
@bcmills
Copy link
Contributor

bcmills commented Mar 6, 2023

It isn't clear to me when or why this would have changed, but I think the reported Go 1.20 behavior of go/build is correct.

  • (*Context).Import requests package data for a package import path, not a directory.

  • Given the possibility of nested module paths, the lack of .go source files in a directory does not necessarily imply that the package does not exist (although that is often the case).

  • With -mod=vendor, we cannot load the full module graph to check whether it contains another module that provides the package. (For example, in this case the package could be provided by a module with the path knative.dev, or one with the path knative.dev/eventing/pkg.) For that matter, even if we could load the full module graph, we can't say anything definitive about an import path that it doesn't contain.

@dprotaso
Copy link
Author

dprotaso commented Mar 6, 2023

FWIW - I can add a doc.go file as a work around to fix this.

Given the possibility of nested module paths, the lack of .go source files in a directory does not necessarily imply that the package does not exist (although that is often the case).

Can you clarify this point? Are you saying that because knative.dev/eventing/pkg/apis does not contain go code there could be some nested module that declares that a module at a parent path. So the module name/path doesn't match the actual location in the repo?

ie. moduleA - go.mod declares it's package as knative.dev/eventing/pkg/apis

go.mod
pkg/apis/
- moduleA/go.mod

For example, in this case the package could be provided by a module with the path knative.dev, or one with the path knative.dev/eventing/pkg

Likewise - I don't really understand. My assumption is a go.mod file at knative.dev/eventing would take ownership of all directories under knative.dev/eventing/* unless another module specified something more specific ie. knative.dev/eventing/moduleB - is that a bad assumption?

@dprotaso
Copy link
Author

dprotaso commented Mar 6, 2023

Probably also worth calling out is that a lot of modules don't have *.go files at all the intermediate levels in a path. It was never the case beforehand.

ie.

moduleA/
- go.mod
- some/path/
   - some-file.go

ie. empirical examples
https://github.com/kubernetes/client-go/tree/master/util
https://github.com/knative/networking/tree/main/pkg/apis

Should they be adding these empty files now?

@bcmills
Copy link
Contributor

bcmills commented Mar 7, 2023

Are you saying that because knative.dev/eventing/pkg/apis does not contain go code there could be some nested module that declares that a module at a parent path.

Yes. You could have a module structured like:

-- go.mod --
module knative.dev
go 1.20
-- eventing/pkg/apis/apis.go --
package apis

Or like:

-- go.mod --
module knative.dev/eventing/pkg
go 1.20
-- apis/apis.go --
package apis

@dprotaso
Copy link
Author

dprotaso commented Mar 7, 2023

In that example wouldn't you assume knative.dev/eventing/pkg is part of the same module in both cases - hence NoGoError might be the right value to return?

@bcmills
Copy link
Contributor

bcmills commented Mar 7, 2023

Should they be adding these empty files now?

No. I think you are conflating modules and packages?

A module is “a collection of packages that are released, versioned, and distributed together.”
A package “is constructed from one or more source files….”

In this case, the module is knative.dev/eventing. It contains packages like knative.dev/eventing/pkg/apis/config and knative.dev/eventing/pkg/apis/eventing. It does not contain packages knative.dev/eventing/pkg or knative.dev/eventing/pkg/apis, because those directories within the module do not contain source files. They can't be imported by other Go packages or programs, can't be built by go build, and so on.

However, the fact that they do not exist in the current module does not imply that they don't exist in some other module.

(*go/build.Context).Import imports packages — not modules or directories.

In contrast, ImportDir does import directories, and I would expect it to return a NoGoError in this case (but I haven't tested, and of course bugs are possible).

@dprotaso
Copy link
Author

dprotaso commented Mar 7, 2023

OK that clears things up - looks like ImportDir returns NoGoError

@dprotaso
Copy link
Author

dprotaso commented Mar 7, 2023

Thanks for all the clarifying answers.

@dprotaso dprotaso closed this as completed Mar 7, 2023
@golang golang locked and limited conversation to collaborators Mar 6, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

3 participants