Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: suppress errors for 'go get' of package paths whose source files are all excluded by build constraints #33526

Closed
jeanbza opened this issue Aug 7, 2019 · 12 comments
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@jeanbza
Copy link
Member

jeanbza commented Aug 7, 2019

What version of Go are you using (go version)?

$ go version
go version go1.13beta1 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/deklerk/Library/Caches/go-build"
GOENV="/Users/deklerk/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/deklerk/workspace/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/tmp/foo/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/c2/cvxltzcd66v5lx14hm1j76q000h16k/T/go-build777475143=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

deklerk at deklerk-macbookpro2 in /tmp/foo
$ go get google.golang.org/genproto@v0.0.0-20190801165951-fa694d86fc64
go: downloading golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961
go: downloading golang.org/x/tools v0.0.0-20190226205152-f727befe758c
go: downloading honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099
go: extracting honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099
go: extracting golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961
go: extracting golang.org/x/tools v0.0.0-20190226205152-f727befe758c
can't load package: package google.golang.org/genproto: build constraints exclude all Go files in /Users/deklerk/workspace/go/pkg/mod/google.golang.org/genproto@v0.0.0-20190801165951-fa694d86fc64
deklerk at deklerk-macbookpro2 in /tmp/foo

What did you expect to see?

deklerk at deklerk-macbookpro2 in /tmp/foo
$ go get google.golang.org/genproto/...
go: downloading golang.org/x/sys v0.0.0-20180830151530-49385e6e1522
go: extracting golang.org/x/sys v0.0.0-20180830151530-49385e6e1522
go: finding google.golang.org/genproto latest
deklerk at deklerk-macbookpro2 in /tmp/foo

What did you see instead?

(see above)

TLDR genproto has a build constraint at the root (+tools, nothing else at root), but many packages below the root do not have this build constraint. When I use go get in module mode - esp when I provide a version - I expect to only interact with the module system. Instead, it's a weird mix of module and package, and in this case though I wanted to get a module it gave me an error about a package of the same name.

@bcmills bcmills changed the title go get in module mode does build constraint checking cmd/go: go get in module mode does build constraint checking Aug 7, 2019
@bcmills
Copy link
Contributor

bcmills commented Aug 7, 2019

go get requests to build a binary. If you know that what you want is just the module (and not the binary at the root of that module), go get -d informs the go command that it should stop without attempting to build such a binary.

It may be that we should suppress the error without -d anyway, though. (See #29268, which is very closely related.)

@bcmills bcmills added modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Aug 7, 2019
@bcmills bcmills added this to the Go1.14 milestone Aug 7, 2019
@bcmills bcmills changed the title cmd/go: go get in module mode does build constraint checking cmd/go: suppress errors for 'go get' of package paths whose source files are all excluded by build constraints Sep 17, 2019
@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@jayconrod
Copy link
Contributor

#40145 is a duplicate of this issue, but that case seems more serious. go get -d fails when given a module path where the module root directory contains ignored .go files. The intent of the command is to update the dependency, not to build anyway.

I think this is happening because go get tries to decide fairly early whether each argument refers to a package or a module. If an argument is a module path, and there are no .go files at the module root, it's assumed to be a module path, and go get will not attempt to load a package. In this case, there are .go files, even though they're ignored, so go get tries and fails to load the package.

go get should probably handle the error from modload.ImportPathsQuiet more gracefully in this case.

@jayconrod jayconrod modified the milestones: Backlog, Go1.16 Jul 10, 2020
@jayconrod jayconrod added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jul 10, 2020
@mvdan
Copy link
Member

mvdan commented Jul 10, 2020

Thanks, @jayconrod! Should I ping the protobuf maintainers about this? If this won't be fixed until 1.16, they probably want to work around the issue by moving the file until most developers are on 1.16 or later. FYI @dsnet

@jayconrod
Copy link
Contributor

I think it would be good to put in a workaround for this. The file in question, integration_test.go seems like it could be moved out of the root directory.

Unfortunately, I think we're too deep in the freeze to land a bug fix for 1.15. Hopefully though that means 1.15 will be out on time and we'll have a longer development window for 1.16!

@bcmills
Copy link
Contributor

bcmills commented Feb 26, 2021

As of Go 1.16, go get now does suppress this error if the -d flag is set (see CL 289769), but not otherwise (because go get is being asked to actually build the package).

We should probably suppress it when the -d flag is not set, too, since we're phasing out the need for -d in general (see #43684).

@gopherbot
Copy link

Change https://golang.org/cl/297009 mentions this issue: cmd/go: suppress errors for 'go get' of module paths that are also constrained-out packages

@mvdan
Copy link
Member

mvdan commented May 18, 2021

@bcmills @jayconrod I'm encountering this issue again with go get -d on Go 1.16.4, so I'm a bit confused:

$ go version
go version go1.16.4 linux/amd64
$ go get -d mvdan.cc/sh/v3@v3.2.0 # no "+build gofuzz" file
go: downloading mvdan.cc/sh/v3 v3.2.0
$ go get -d mvdan.cc/sh/v3@v3.3.0 # "+build gofuzz" file appears here
mvdan.cc/sh/v3: no Go source files

Edit: I should note that go version devel go1.17-ce92a2023c Sat May 15 02:39:08 2021 +0000 linux/amd64 works fine there.

@bcmills
Copy link
Contributor

bcmills commented May 18, 2021

@mvdan, interesting! I see the same behavior with go1.16.4 but not with gotip:

$ go version
go version devel go1.17-8b0901fd3 Tue May 18 07:50:25 2021 +0000 linux/amd64

$ go get -d mvdan.cc/sh/v3@v3.3.0
go: downloading mvdan.cc/sh/v3 v3.3.0
go get: added mvdan.cc/sh/v3 v3.3.0

$ go1.16.4 get -d mvdan.cc/sh/v3@v3.3.0
mvdan.cc/sh/v3: no Go source files

$

So we must have fixed something else during the 1.17 cycle, but I'm not sure what.

@mvdan
Copy link
Member

mvdan commented May 18, 2021

Should we reopen this temporarily while we investigate, or open a new issue about it? I'll tend to forget about these things if they're buried in a closed issue :)

@bcmills
Copy link
Contributor

bcmills commented May 18, 2021

Since this doesn't reproduce at head, I'd say no: I don't think we would backport a fix even if we found it, given that it's fairly easy to work around. (You can go get -d the specific package(s) you need instead of the module root.)

@bcmills
Copy link
Contributor

bcmills commented May 18, 2021

(If it were a regression from 1.15 to 1.16 that would be another matter, but the situation was even worse on 1.15... 😅)

@mvdan
Copy link
Member

mvdan commented May 19, 2021

Fair enough, I'll add a workaround on my end for now - since I do think it's reasonable to go get -d module-path, and I do it pretty often :)

@golang golang locked and limited conversation to collaborators May 19, 2022
@rsc rsc unassigned bcmills Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

6 participants