Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: test, list etc. fails when run in parallel on Windows #27372

Closed
bep opened this issue Aug 30, 2018 · 7 comments
Closed

cmd/go: test, list etc. fails when run in parallel on Windows #27372

bep opened this issue Aug 30, 2018 · 7 comments
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone
Go1.12

Comments

@bep
Copy link
Contributor

bep commented Aug 30, 2018 •
edited

The workaround is straightforward (serialize build), but I think this deserves an issue as 1) this worked fine in Go 1.10 and 2) people in the same boat as me can get something to search for when scratching their head

Also note that this works fine on macOS and Linux.

Environment:

go version
go version go1.11 windows/amd64
go env
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\appveyor\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\GOPATH\
set GOPROXY=
set GORACE=
set GOROOT=C:\go
set GOTMPDIR=
set GOTOOLDIR=C:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\appveyor\AppData\Local\Temp\1\go-build440396294=/tmp/go-build -gno-record-gcc-switches
  1. The commit that fixed the build:

gohugoio/hugo@ea8ef57

  1. Extract of the error log:
# github.com/gohugoio/hugo/common/maps
bufferpool\bufpool_test.go:17:2: unknown import path "github.com/stretchr/testify/assert": target directory C:\GOPATH\pkg\mod\github.com\stretchr\testify@v1.2.2 exists and is not empty
FAIL	github.com/gohugoio/hugo/common/maps [setup failed]
# github.com/gohugoio/hugo/common/math
package github.com/gohugoio/hugo/common/math (test)
	imports github.com/alecthomas/assert
	imports github.com/alecthomas/repr: unknown import path "github.com/alecthomas/repr": target directory C:\GOPATH\pkg\mod\github.com\alecthomas\repr@v0.0.0-20180724025338-ead21651dde3 exists and is not empty
FAIL	github.com/gohugoio/hugo/common/math [setup failed]
# github.com/gohugoio/hugo/common/types
cache\partitioned_lazy_cache_test.go:21:2: unknown import path "github.com/stretchr/testify/require": target directory C:\GOPATH\pkg\mod\github.com\stretchr\testify@v1.2.2 exists and is not empty
FAIL	github.com/gohugoio/hugo/common/types [setup failed]
# github.com/gohugoio/hugo/config
cache\partitioned_lazy_cache_test.go:21:2: unknown import path "github.com/stretchr/testify/require": target directory C:\GOPATH\pkg\mod\github.com\stretchr\testify@v1.2.2 exists and is not empty
FAIL	github.com/gohugoio/hugo/config [setup failed]
# github.com/gohugoio/hugo/config/privacy
cache\partitioned_lazy_cache_test.go:21:2: unknown import path "github.com/stretchr/testify/require": target directory C:\GOPATH\pkg\mod\github.com\stretchr\testify@v1.2.2 exists and is not empty
FAIL	github.com/gohugoio/hugo/config/privacy [setup failed]
# github.com/gohugoio/hugo/config/services
cache\partitioned_lazy_cache_test.go:21:2: unknown import path "github.com/stretchr/testify/require": target directory C:\GOPATH\pkg\mod\github.com\stretchr\testify@v1.2.2 exists and is not empty
FAIL	github.com/gohugoio/hugo/config/services [setup failed]
# github.com/gohugoio/hugo/create
cache\partitioned_lazy_cache_test.go:21:2: unknown import path "github.com/stretchr/testify/require": target directory C:\GOPATH\pkg\mod\github.com\stretchr\testify@v1.2.2 exists and is not empty

/cc @natefinch
@bcmills
Copy link
Contributor

bcmills commented Aug 30, 2018

Please provide more detail. I don't see answers for the two most important questions from the issue template:

What did you do?

What did you expect to see?

@bcmills bcmills added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Aug 30, 2018
@bcmills bcmills added this to the Go1.12 milestone Aug 30, 2018
@bep
Copy link
Contributor Author

bep commented Aug 30, 2018

Please provide more detail. I don't see answers for the two most important questions from the issue template:

It's in there if you read more carefully. I have spent enough time on this issue. Use my issue report to improve your software/documentation, or don't.

@bep
Copy link
Contributor Author

bep commented Sep 6, 2018

I have another case, which is more problematic (and I think a more common problem), now in a Linux container on CircleCI:

See error log below.

da00d00bdef65874ff644eaa26: exit status 128:
	fatal: cannot copy '/usr/share/git-core/templates/hooks/update.sample' to '/go/pkg/mod/cache/vcs/cb1953cbdfd14fc2ffce4dfd06487e8d5a0c96da00d00bdef65874ff644eaa26/hooks/update.sample': File exists
go: github.com/dsnet/golib@v0.0.0-20171103203638-1ea166775780: git init --bare in /go/pkg/mod/cache/vcs/e56cb0fb3504244c08f9270d26097e67273a5d49f0367a918d5322d12df3a9b8: exit status 128:
	error: could not lock config file /go/pkg/mod/cache/vcs/e56cb0fb3504244c08f9270d26097e67273a5d49f0367a918d5322d12df3a9b8/config: No such file or directory
	fatal: could not set 'core.symlinks' to 'false'
go: golang.org/x/sync@v0.0.0-20180314180146-1d60e4601c6f: git init --bare in /go/pkg/mod/cache/vcs/55179c5d8c4db2eaed9fae4682d4c84a1fd3612df666b372bef3bbb997c9601f: exit status 128:
	error: could not lock config file /go/pkg/mod/cache/vcs/55179c5d8c4db2eaed9fae4682d4c84a1fd3612df666b372bef3bbb997c9601f/config: File exists
	fatal: could not set 'core.repositoryformatversion' to '0'
go: golang.org/x/net@v0.0.0-20180826012351-8a410e7b638d: git init --bare in /go/pkg/mod/cache/vcs/4a22365141bc4eea5d5ac4a1395e653f2669485db75ef119e7bbec8e19b12a21: exit status 128:
	error: could not lock config file /go/pkg/mod/cache/vcs/4a22365141bc4eea5d5ac4a1395e653f2669485db75ef119e7bbec8e19b12a21/config: File exists
	fatal: could not set 'core.repositoryformatversion' to '0'
go: gopkg.in/yaml.v2@v2.2.1: git init --bare in /go/pkg/mod/cache/vcs/4f939cf8921e8662ef2d8cf78b0db3bcacccda0d79c9e1b4cef06c640b4a8cef: exit status 128:
	error: could not lock config file /go/pkg/mod/cache/vcs/4f939cf8921e8662ef2d8cf78b0db3bcacccda0d79c9e1b4cef06c640b4a8cef/config: File exists
	fatal: could not set 'core.repositoryformatversion' to '0'
go: golang.org/x/text@v0.3.0: git remote add origin https://go.googlesource.com/text in /go/pkg/mod/cache/vcs/5b03666c2d7b526129bad48c5cea095aad8b83badc1daa202e7b0279e3a5d861: exit status 128:
	error: could not lock config file config: File exists
	fatal: could not set 'remote.origin.url' to 'https://go.googlesource.com/text'
go: error loading module requirements

/cc @caarlos0

@bcmills
Copy link
Contributor

bcmills commented Sep 6, 2018

Which go commands were you trying to run in parallel, and which of those failed?
Did you run go mod tidy and/or go mod download beforehand?

@bep
Copy link
Contributor Author

bep commented Sep 6, 2018

@bcmills the links above are to the actual scripts with reference to Docker image used etc. For the last one, I suspect a go mod download before starting the parallel builds would have done the trick.

@caarlos0
Copy link
Contributor

@bep yes, parallel go mod download will likely not work... this is a WIP on GoReleaser side as well: goreleaser/goreleaser#779

@bcmills
Copy link
Contributor

bcmills commented Sep 11, 2018

Closing as a duplicate of #26794.

@bcmills bcmills closed this as completed Sep 11, 2018
wfscheper added a commit to wfscheper/hugo that referenced this issue Sep 15, 2018
@wfscheper
Simplify travis-ci build
2c69e26 
This change avoids the need to copy the source out of the GOPATH by
applying GO111MODULE=on as an environment variable. Additionally, avoids
issues with parallel module downloads (see golang/go#26794 and
golang/go#27372) by running 'go mod download' during the install step.
wfscheper added a commit to wfscheper/hugo that referenced this issue Sep 15, 2018
@wfscheper
Simplify travis-ci build
53821b2 
This change avoids the need to copy the source out of the GOPATH by
applying GO111MODULE=on as an environment variable. Additionally, avoids
issues with parallel module downloads (see golang/go#26794 and
golang/go#27372) by running 'go mod download' during the install step.
wfscheper added a commit to wfscheper/hugo that referenced this issue Sep 15, 2018
@wfscheper
Simplify travis-ci build
9992078 
This change avoids the need to copy the source out of the GOPATH by
applying GO111MODULE=on as an environment variable. Additionally, avoids
issues with parallel module downloads (see golang/go#26794 and
golang/go#27372) by running 'go mod download' during the install step.
@wfscheper wfscheper mentioned this issue Sep 15, 2018
Closed
@golang golang locked and limited conversation to collaborators Sep 11, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
Go1.12
Development

No branches or pull requests
5 participants
@caarlos0 @bep @FiloSottile @bcmills @gopherbot