Skip to content

cmd/go: can't load package when GOPATH is under GOROOT #32621

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Helflym opened this issue Jun 14, 2019 · 21 comments
Closed

cmd/go: can't load package when GOPATH is under GOROOT #32621

Helflym opened this issue Jun 14, 2019 · 21 comments
Labels
help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@Helflym
Copy link
Contributor

Helflym commented Jun 14, 2019

The go command seems to have trouble parsing the package name when it's under $GOROOT/src.
The error is obvious with gccgo (so also with version 1.12). However, I'm not sure if the error from TIP is related or not.

$ go env
GOPATH="/opt/freeware/src/packages/BUILD/go-path"
GOROOT="/opt/freeware"
$ cd $GOPATH/src/golang.org/x/sys/unix/

WITH GCCGO
$ go.gcc version  
go version go1.12.2 gccgo (GCC) 9.1.0 aix/ppc64
$ go.gcc build 
can't load package: package packages/BUILD/go-path/src/golang.org/x/sys/unix: cannot find package "packages/BUILD/go-path/src/golang.org/x/sys/unix" in any of:
        /opt/freeware/src/packages/BUILD/go-path/src/golang.org/x/sys/unix (from $GOROOT)
        /opt/freeware/src/packages/BUILD/go-path/src/packages/BUILD/go-path/src/golang.org/x/sys/unix (from $GOPATH)

WITH GO 1.12 
$ go version
go version go1.12.5 aix/ppc64
$ go.golang build
can't load package: package packages/BUILD/go-path/src/golang.org/x/sys/unix: code in directory /opt/freeware/src/packages/BUILD/go-path/src/golang.org/x/sys/unix expects import "golang.org/x/sys/unix"

WITH TIP
$ go version
go version devel +d2f0628430 Thu Jun 13 09:51:55 2019 -0500 aix/ppc64
$ go build
go: inconsistent vendoring in /opt/freeware/src/packages/BUILD/go-path/src/golang.org/x/sys:
        go.mod requires golang.org/x/sys  but vendor/modules.txt does not include it.
        run 'go mod tidy; go mod vendor' to sync

This alos happen on Fedora 30 (at least with gccgo).
I do agree that having GOPATH under GOROOT isn't the best idea ever. But this is sometime the case on AIX (and maybe Fedora), because default GOROOT is "/opt/freeware" and the RPM are built under "/opt/freeware/src/packages/BUILD".

Therefore, is it a true bug in go command or a problem related to how Go RPMs are delivered ?

@FiloSottile
Copy link
Contributor

Setting GOPATH within GOROOT is definitely not a supported configuration, but maybe we should generate a better error for it.

/cc @jayconrod

@FiloSottile FiloSottile added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 14, 2019
@FiloSottile FiloSottile added this to the Unplanned milestone Jun 14, 2019
@bcmills
Copy link
Contributor

bcmills commented Jun 17, 2019

A GOROOT of /opt/freeware seems like a problem in general. GOROOT has a lot of really generic directories such as src, test, and misc that are likely to conflict with other programs.

In particular, if we were to add a packages package to the standard library, then a GOROOT of /opt/freeware would place that package's source at /opt/freeware/src/packages.

@bcmills
Copy link
Contributor

bcmills commented Jun 17, 2019

At any rate, we do have a similar existing diagnostic, if someone wants to expand it using search.InDir:

go/src/cmd/go/main.go

Lines 106 to 111 in 82cf8bc

// Diagnose common mistake: GOPATH==GOROOT.
// This setting is equivalent to not setting GOPATH at all,
// which is not what most people want when they do it.
if gopath := cfg.BuildContext.GOPATH; filepath.Clean(gopath) == filepath.Clean(runtime.GOROOT()) {
fmt.Fprintf(os.Stderr, "warning: GOPATH set to GOROOT (%s) has no effect\n", gopath)
} else {

@Helflym
Copy link
Contributor Author

Helflym commented Jun 17, 2019

GOROOT is set to /opt/freeware only with gccgo. And I think this is because we are using --prefix="/opt/freeware" when building gcc in general.
Maybe @ianlancetaylor, you're aware of such problems in a different OS ?
Edit: I do confirm with this is linked with `--prefix`` and comes from https://github.com/golang/gofrontend/blob/master/libgo/Makefile.am#L556. Maybe it can be adapted for AIX.

@ianlancetaylor
Copy link
Member

The gccgo default for GOROOT is definitely problematic. See https://gcc.gnu.org/PR89171. I've been intending to take a look at it before the next GCC release.

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/185540 mentions this issue: cmd/go: Warn when GOPATH is a subdir of GOROOT

@qbradq
Copy link
Contributor

qbradq commented Jul 10, 2019

I am unable to reproduce the issue. The only way I can convince go to not find something under GOPATH is to reference it incorrectly. Even when GOPATH is a sub-directory of GOROOT. Can you post the code or at least the import block @Helflym ?

@Helflym
Copy link
Contributor Author

Helflym commented Jul 11, 2019

You're right, my error occurs only because GOPATH is a sub-directory of $GOROOT/src (and not simply $GOROOT). It's very unlikely that it'll ever occur with Golang as GOROOT as a true meaning. However, with the current gccgo, GOROOT is more a placeholder AFAIK.

$ go env 
GOROOT="/tmp/go-root"
GOPATH="/tmp/go-root/src/go-path"
$ cd /tmp/go-root/src/go-path/src/golang.org/x/sys/unix 
$ GO111MODULE=off go build
can't load package: package go-path/src/golang.org/x/sys/unix: code in directory /tmp/go-root/src/go-path/src/golang.org/x/sys/unix expects import "golang.org/x/sys/unix"

GO111MODULE=off is just to make a more explicit error.

@qbradq
Copy link
Contributor

qbradq commented Jul 11, 2019

@Helflym I still have not be able to reproduce this. Can you at least post the import block of your main go file? I have tried all the combinations I can think of as far as file and import structure to cause this, but I get no errors.

qbradq@qbradq-AB350-Gaming:~/golang/src/go-path/src/golang.org/x/sys/unix$ set | grep GO
GOPATH=/home/qbradq/golang/src/go-path
GOROOT=/home/qbradq/golang
qbradq@qbradq-AB350-Gaming:~/golang/src/go-path/src/golang.org/x/sys/unix$ cat main.go
package main

import "golang.org/x/sys/unix/sub"

func main() {
	sub.SayHello()
}
qbradq@qbradq-AB350-Gaming:~/golang/src/go-path/src/golang.org/x/sys/unix$ cat sub/lib.go
package sub

import "fmt"

// SayHello says hello
func SayHello() {
	fmt.Println("Hello, #32621!")
}
qbradq@qbradq-AB350-Gaming:~/golang/src/go-path/src/golang.org/x/sys/unix$ GO111MODULE=off $GOROOT/bin/go build
qbradq@qbradq-AB350-Gaming:~/golang/src/go-path/src/golang.org/x/sys/unix$ ./unix
Hello, #32621!

@Helflym
Copy link
Contributor Author

Helflym commented Jul 12, 2019

I was using the whole x/sys/unix package (https://github.com/golang/sys) not a simple main.go and its sub package.
But indeed, it does work with your simple example. It could be interesting to know why it does with this simple package but it doesn't with a complex package like x/sys/unix.
Anyway, here is the imports of x/sys/unix,

$ go list -f '{{ .Imports }}'
[bytes encoding/binary net runtime sort strings sync syscall time unsafe]

@qbradq
Copy link
Contributor

qbradq commented Jul 12, 2019

@Helflym Thank you the patience :) I understand now that you are trying to build the golang.org/x/sys/unix package. I was able to reproduce the issue when building this package. I am continuing analysis now.

To reproduce:

$ mkdir /tmp/issue-32621
$ cd /tmp/issue-32621
$ export GOROOT=$(pwd)
$ export GOPATH=$GOROOT/src/gopath
$ mkdir -p $GOPATH
$ go get golang.org/x/sys/unix
package bytes: unrecognized import path "bytes" (import path does not begin with hostname)
package encoding/binary: unrecognized import path "encoding/binary" (import path does not begin with hostname)
package runtime: unrecognized import path "runtime" (import path does not begin with hostname)
package sort: unrecognized import path "sort" (import path does not begin with hostname)
package strings: unrecognized import path "strings" (import path does not begin with hostname)
package sync: unrecognized import path "sync" (import path does not begin with hostname)
package syscall: unrecognized import path "syscall" (import path does not begin with hostname)
package time: unrecognized import path "time" (import path does not begin with hostname)
package unsafe: unrecognized import path "unsafe" (import path does not begin with hostname)
$ cd src/gopath/src/golang.org/x/sys/unix/
$ go build
can't load package: package gopath/src/golang.org/x/sys/unix: code in directory /tmp/issue-32621/src/gopath/src/golang.org/x/sys/unix expects import "golang.org/x/sys/unix"

@FiloSottile Should we just add a warning about this unsupported configuration (and fix the GCC default prefix) or try to find the root cause?

@FiloSottile
Copy link
Contributor

Let's just detect it early, print a helpful error, and exit.

@qbradq
Copy link
Contributor

qbradq commented Jul 16, 2019

@FiloSottile We can't exit. The go build process and as well as some internal test cases change GOPATH to be a subdirectory of GOROOT and expects it to work. So should it just print a helpful warning?

@FiloSottile
Copy link
Contributor

Which part of the build process, and what tests? It seems weird to rely on an unsupported and apparently broken configuration.

@qbradq
Copy link
Contributor

qbradq commented Jul 16, 2019

qbradq@qbradq-AB350-Gaming:~/godev/goroot/src$ ./all.bash
Building Go cmd/dist using /usr/lib/go-1.10.
Building Go toolchain1 using /usr/lib/go-1.10.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
go: GOPATH (/home/qbradq/godev/goroot/pkg/obj/gopath) under GOROOT (/home/qbradq/godev/goroot) is not supported
go tool dist: FAILED: /home/qbradq/godev/goroot/pkg/tool/linux_amd64/go_bootstrap install -gcflags=all= -ldflags=all= -i cmd/asm cmd/cgo cmd/compile cmd/link: exit status 2

Building toolchain2 apparently uses this configuration.

For an example of a test that uses this, see

func TestShadowingLogic(t *testing.T) {

@bcmills
Copy link
Contributor

bcmills commented Aug 1, 2019

@qbradq, can you get things to work by narrowing the check to GOROOT/src rather than all of GOROOT?

@qbradq
Copy link
Contributor

qbradq commented Aug 8, 2019

This was going to be my next try :) Unfortunately I broke my hand. That slowed me down. The Go build and test cases are all going to be under $GOROOT/src though, so it doesn't help.

@Pranavvks

This comment has been minimized.

@ianlancetaylor

This comment has been minimized.

@sarinkejohn

This comment has been minimized.

@davecheney
Copy link
Contributor

davecheney commented Jul 7, 2021

Timed out. If the issue reoccurs please open a fresh report.

@golang golang locked as resolved and limited conversation to collaborators Jul 7, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

9 participants