Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: module loader fails to resolve imports within symlinked source file #28107

Closed
zx2c4 opened this issue Oct 9, 2018 · 10 comments
Closed

cmd/go: module loader fails to resolve imports within symlinked source file #28107

zx2c4 opened this issue Oct 9, 2018 · 10 comments
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Go1.12

Comments

@zx2c4
Copy link
Contributor

zx2c4 commented Oct 9, 2018 •
edited

This succeeds:

mkdir source-files && cd source-files
cp /path/to/somewhere/file1.go file1.go
cp /path/to/somewhere/file2.go file2.go
cp /path/to/somewhere/file3.go file3.go
cp /path/to/somewhere/go.mod go.mod
cp /path/to/somewhere/go.sum go.sum
go build

However, this fails:

mkdir source-files && cd source-files
ln -s /path/to/somewhere/file1.go file1.go
ln -s /path/to/somewhere/file2.go file2.go
ln -s /path/to/somewhere/file3.go file3.go
ln -s /path/to/somewhere/go.mod go.mod
ln -s /path/to/somewhere/go.sum go.sum
go build
[...FAILURES...]

It seems like go's module system might be trying to resolve things relative to the symlink target and not the actual symlink, which makes all sorts of nice overlay arrangements impossible unless you either hardlink (ew) or copy (yuck).

Here's a full sequence to reproduce

zx2c4@thinkpad ~ $ cd /tmp
zx2c4@thinkpad /tmp $ mkdir blah
zx2c4@thinkpad /tmp $ cd blah/
zx2c4@thinkpad /tmp/blah $ export GOPATH=$PWD/gopath
zx2c4@thinkpad /tmp/blah $ printf 'package main\nimport "golang.org/x/sys/unix"\nfunc main() { unix.Exit(0) }\n' > blah.go
zx2c4@thinkpad /tmp/blah $ go mod init blah
go: creating new go.mod: module blah
zx2c4@thinkpad /tmp/blah $ go build
go: finding golang.org/x/sys/unix latest
go: finding golang.org/x/sys latest
go: downloading golang.org/x/sys v0.0.0-20181005133103-4497e2df6f9e
zx2c4@thinkpad /tmp/blah $ ./blah
zx2c4@thinkpad /tmp/blah $ chmod -R +w gopath && rm -rf gopath
zx2c4@thinkpad /tmp/blah $ mkdir symlink
zx2c4@thinkpad /tmp/blah $ cd symlink
zx2c4@thinkpad /tmp/blah/symlink $ export GOPATH=$PWD/gopath
zx2c4@thinkpad /tmp/blah/symlink $ ln -s ../go.mod ../go.sum ../blah.go .
zx2c4@thinkpad /tmp/blah/symlink $ ls -l *
lrwxrwxrwx 1 zx2c4 zx2c4 10 Oct 10 06:27 blah.go -> ../blah.go
lrwxrwxrwx 1 zx2c4 zx2c4  9 Oct 10 06:27 go.mod -> ../go.mod
lrwxrwxrwx 1 zx2c4 zx2c4  9 Oct 10 06:27 go.sum -> ../go.sum
zx2c4@thinkpad /tmp/blah/symlink $ go build
go: finding golang.org/x/sys v0.0.0-20181005133103-4497e2df6f9e
blah.go:2:8: unknown import path "golang.org/x/sys/unix": cannot find package
@agnivade
Copy link
Contributor

@bcmills @myitcv

@bcmills bcmills added modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Oct 10, 2018
@bcmills
Copy link
Contributor

bcmills commented Oct 10, 2018

Thanks for the straightforward reproducer. The problem seems to be the .go source file.


~/go/src/cmd/go$ pushd $(mktemp -d)
/tmp/tmp.0t9zEAmI9v ~/go/src/cmd/go

$ export GOPATH=$PWD/gopath

$ printf 'package main\nimport "golang.org/x/sys/unix"\nfunc main() { unix.Exit(0) }\n' > blah.go

$ go mod init blah
go: creating new go.mod: module blah

$ go build
go: finding golang.org/x/sys/unix latest
go: finding golang.org/x/sys latest
go: downloading golang.org/x/sys v0.0.0-20181005133103-4497e2df6f9e

$ ./blah

$ mkdir symlink

$ cd symlink

symlink$ export GOPATH=$PWD/gopath

symlink$ ln -s ../go.mod ../go.sum ../blah.go .

symlink$ go build
go: finding golang.org/x/sys v0.0.0-20181005133103-4497e2df6f9e
blah.go:2:8: unknown import path "golang.org/x/sys/unix": cannot find package

symlink$ rm blah.go

symlink$ cp ../blah.go ./

symlink$ go build
go: downloading golang.org/x/sys v0.0.0-20181005133103-4497e2df6f9e

symlink$

@bcmills bcmills added this to the Go1.12 milestone Oct 10, 2018
@agnivade agnivade changed the title modules fail with symlinked files cmd/go: modules fail with symlinked files Oct 10, 2018
@gopherbot
Copy link

Change https://golang.org/cl/141097 mentions this issue: cmd/go/internal/imports: resolve symlinks in ScanDir

@bcmills bcmills added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Oct 10, 2018
@bcmills bcmills changed the title cmd/go: modules fail with symlinked files cmd/go: module loader fails to resolve imports within symlinked source file Oct 10, 2018
@zx2c4
Copy link
Contributor Author

zx2c4 commented Oct 10, 2018

@bcmills I'm not too certain on the cause of this, but broadly speaking, shouldn't the fix involve resolving fewer symlinks, not more?

@bcmills
Copy link
Contributor

bcmills commented Oct 11, 2018

Nope. The bug turned out to be in how we detect whether an entry within the package directory is a file or a directory: in order to do that for a symlink, we have to first resolve it.

@zx2c4
Copy link
Contributor Author

zx2c4 commented Oct 11, 2018

👍

@gopherbot
Copy link

Change https://golang.org/cl/163517 mentions this issue: cmd/go/internal/imports: use the full path to resolve symlinks

gopherbot pushed a commit that referenced this issue Feb 23, 2019
cmd/go/internal/imports: use the full path to resolve symlinks
3726d91 
info.Name returns a name relative to the directory, so we need to
prefix that directory in the Stat call.

(This was missed in CL 141097 due to the fact that the test only
happened to check symlinks in the current directory.)

This allows the misc/ tests to work in module mode on platforms that
support symlinks.

Updates #30228
Updates #28107

Change-Id: Ie31836382df0cbd7d203b7a8b637c4743d68b6f3
Reviewed-on: https://go-review.googlesource.com/c/163517
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
@katiehockman katiehockman mentioned this issue Apr 30, 2019
Closed
@katiehockman
Copy link
Contributor

@gopherbot please open a backport to 1.12. This is a low risk patch that has enough benefits to warrant a cherrypick.

@gopherbot gopherbot mentioned this issue Apr 30, 2019
Closed
@gopherbot
Copy link

Backport issue(s) opened: #31763 (for 1.12).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

@gopherbot
Copy link

Change https://golang.org/cl/175441 mentions this issue: [release-branch.go1.12] cmd/go/internal/imports: use the full path to resolve symlinks

gopherbot pushed a commit that referenced this issue May 6, 2019
[release-branch.go1.12] cmd/go/internal/imports: use the full path to…
4a1d39b 
… resolve symlinks

info.Name returns a name relative to the directory, so we need to
prefix that directory in the Stat call.

(This was missed in CL 141097 due to the fact that the test only
happened to check symlinks in the current directory.)

This allows the misc/ tests to work in module mode on platforms that
support symlinks.

Updates #30228
Updates #28107
Fixes #31763

Change-Id: Ie31836382df0cbd7d203b7a8b637c4743d68b6f3
Reviewed-on: https://go-review.googlesource.com/c/163517
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/175441
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@golang golang locked and limited conversation to collaborators May 5, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Milestone
Go1.12
Development

No branches or pull requests
5 participants
@zx2c4 @agnivade @bcmills @gopherbot @katiehockman