Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: race causes go mod download to error with mkdir /go/pkg/mod..: no such file or directory #30068

Closed
dnephin opened this issue Feb 2, 2019 · 6 comments
Closed

cmd/go: race causes go mod download to error with mkdir /go/pkg/mod..: no such file or directory #30068

dnephin opened this issue Feb 2, 2019 · 6 comments
Labels
FrozenDueToAge GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone
Go1.13

Comments

@dnephin
Copy link
Contributor

dnephin commented Feb 2, 2019

What version of Go are you using (go version)?

$ go version
go version go1.11.5 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

linux/amd64

What did you do?

We run builds in a docker container as part of CI. From a container with a fresh git clone of a project we run the following:

mkdir -p /go/pkg/mod
go mod download

or

mkdir -p /go/pkg/mod
go test -v ./...

These commands will download dependencies. Frequently (roughly 1 in 3 builds) these commands will fail with an error like one of the following.

mkdir /go/pkg/mod/cache/vcs: no such file or directory
mkdir /go/pkg/mod/cache/vcs/da5145fda272732cd74527dacbe4967cc6a648dad514ccb83c288a28ea4c0671: no such file or directory
mkdir /go/pkg/mod/cache/vcs/6df073f72200185af52f4179ee546de39a8cde072c63a5c452831c700297adb1: no such file or directory

Re-running the build will eventually fix the problem. The directory is different almost every time.

What did you expect to see?

I expected to see the go: finding... output, followed by a successful download or test run.

What did you see instead?

go: github.com/mitchellh/multistep@v0.0.0-20170316185339-391576a156a5: mkdir /go/pkg/mod/cache/vcs: no such file or directory
go: github.com/anmitsu/go-shlex@v0.0.0-20161002113705-648efa622239: mkdir /go/pkg/mod/cache/vcs/6df073f72200185af52f4179ee546de39a8cde072c63a5c452831c700297adb1: no such file or directory
go: finding github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78
go: finding github.com/aws/aws-sdk-go v1.13.48
...
go: error loading module requirements
@oiooj oiooj added modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Feb 4, 2019
@jayconrod jayconrod added the GoCommand cmd/go label Feb 4, 2019
@jayconrod
Copy link
Contributor

Probably unrelated, but #29667 is another cache locking issue (but for the build cache, not the module cache).

@bcmills
Copy link
Contributor

bcmills commented Feb 4, 2019

Duplicate of #26794: the go command in Go 1.11 is not safe to invoke concurrently in module mode.

Workaround is to use go1.12beta2 instead.

@bcmills
Copy link
Contributor

bcmills commented Feb 4, 2019

@dnephin please do let us know if you can still reproduce these failures with 1.12.

@bcmills bcmills closed this as completed Feb 4, 2019
@yangchenyun
Copy link

yangchenyun commented Apr 8, 2019 •
edited

@bcmills After upgrading to 1.12.2, the same issue still comes up from time to time in production.

Our CI and night build pipeline all observes this failure multiple times in a day.

go: github.com/erikstmartin/go-testdb@v0.0.0-20160219214506-8d10e4a1bae5: mkdir /go/pkg/mod: no such file or directory

@bcmills bcmills reopened this Apr 13, 2019
@bcmills bcmills added this to the Go1.13 milestone Apr 13, 2019
@rsc
Copy link
Contributor

rsc commented Apr 24, 2019

@yangchenyun Concurrency problems here should cause problems where two processes might try to create the same directory at the same time.
But the transcript shows a directory being removed: mkdir /go/pkg/mod: no such file or directory.
That is, the mkdir failed because /go/pkg or /go does not exist.
Is something in your CI system doing rm -rf /go/pkg?
The go command itself is not.

@bcmills bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Apr 24, 2019
@gopherbot
Copy link

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)

@golang golang locked and limited conversation to collaborators May 23, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
Go1.13
Development

No branches or pull requests
7 participants
@rsc @jayconrod @dnephin @yangchenyun @oiooj @bcmills @gopherbot