Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: get and build can interact badly on case-insensitive but case-preserving file systems #20264

Closed
josharian opened this issue May 6, 2017 · 3 comments
Milestone

Comments

@josharian
Copy link
Contributor

josharian commented May 6, 2017

Reproduce, using stock macOS filesystem, which is case-insensitive but case-preserving:

$ go get -d github.com/0x263b/Porygon2
$ go build -toolexec="toolstash -cmp" -a -o /dev/null github.com/0x263b/...

Result: Rare object file mismatches in which only a few bytes differ. Even rarer build failures in which we emit corrupt object files.

The reason is that go get downloads the code into $GOPATH/src/github.com/0x263b/Porygon2. go build then walks $GOPATH/src/github.com/0x263b, and adds the import paths corresponding to the directory names it sees, e.g. github.com/0x263b/Porygon2/web; note the capital P. Then it reads the import paths found in the code, which are lower case, and adds those import paths, including github.com/0x263b/porygon2/web; lower case p.

There's a check in PackagesForBuild for this exact scenario, aptly described in a comment:

	// Check for duplicate loads of the same package.
	// That should be impossible, but if it does happen then
	// we end up trying to build the same package twice,
	// usually in parallel overwriting the same files,
	// which doesn't work very well.

However, that check is case-sensitive, so this package sneaks past the check. Making the check do a strings.ToLower on the package import paths catches it:

internal error: duplicate loads of github.com/0x263b/porygon2
internal error: duplicate loads of github.com/0x263b/porygon2/web

This problem is not theoretical; I just wasted four hours tracking it down, starting from non-deterministic build failures.

This seems like a can of worms, and I don't know what the right fix is, but I think we should do something. Input requested.

cc @rsc @bradfitz @shurcooL

@josharian josharian added this to the Go1.9 milestone May 6, 2017
@dmitshur
Copy link
Contributor

dmitshur commented May 6, 2017

Worth noting, in the README, it says:

go get -u github.com/0x263b/porygon2

So the canonical import path is meant to be with lower-case p. However, there is no import path comment enforcing that, and it seems there's nothing to stop one from go geting the unintended case of import path and trying to build that. Especially since the canonical name of the repository is with an upper-case P, so the chance is even higher than usual.

One solution that I want to consider is to make it so that import paths should be case sensitive on case-preserving file systems, if that's possible. If you have a package at $GOPATH/src/foo path, doing import "FOO" should tell you package "FOO" doesn't exist, even though cd $GOPATH/src/FOO would work (due to case insensitive filesystem).

@rsc
Copy link
Contributor

rsc commented Jun 22, 2017

Will send CL. We already detect this when they're both in the same program, but we should detect it when they're just in the same build instead.

@gopherbot
Copy link

CL https://golang.org/cl/46424 mentions this issue.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants