Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: occasional failures in TestScript/mod_convert on Windows with 'exec: "git": executable file not found' #45659

Closed
bcmills opened this issue Apr 21, 2021 · 7 comments
Closed

cmd/go: occasional failures in TestScript/mod_convert on Windows with 'exec: "git": executable file not found' #45659

bcmills opened this issue Apr 21, 2021 · 7 comments
Labels
FrozenDueToAge GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. okay-after-beta1 Used by release team to mark a release-blocker issue as okay to resolve either before or after beta1 release-blocker
Milestone
Go1.17

Comments

@bcmills
Copy link
Contributor

bcmills commented Apr 21, 2021

2021-04-21T00:54:39-760d3b2/windows-amd64-longtest
2021-04-20T16:44:18-109d758/windows-amd64-longtest
2021-04-19T23:02:15-7252e1e/windows-amd64-longtest
2021-04-19T21:27:43-e97d8eb/windows-amd64-longtest
2021-04-19T18:37:15-f889214/windows-amd64-longtest
2021-04-19T16:21:39-6b8e3e2/windows-amd64-longtest
2021-04-17T00:02:01-fd3612e/windows-amd64-longtest

--- FAIL: TestScript (0.01s)
    --- FAIL: TestScript/mod_convert (113.82s)
        script_test.go:252: 
            > [short] skip
            > env GO111MODULE=on
            > env GOPROXY=
            > env GOSUMDB=
            > go mod download github.com/docker/distribution@v0.0.0-20150410205453-85de3967aa93
            > mkdir x/Godeps
            > cp $GOPATH/pkg/mod/github.com/docker/distribution@v0.0.0-20150410205453-85de3967aa93/Godeps/Godeps.json x/Godeps
            > cd x
            $WORK\gopath\src\x
            > go mod init github.com/docker/distribution
            [stderr]
            go: creating new go.mod: module github.com/docker/distribution
            go: downloading github.com/bugsnag/osext v0.0.0-20130617224835-0dd3f918b21b
            go: downloading github.com/docker/docker v1.4.2-0.20150204013315-165ea5c158cf
            go: downloading github.com/bugsnag/bugsnag-go v1.0.3-0.20141110184014-b1d153021fcd
            go: downloading github.com/Sirupsen/logrus v0.7.3
            go: downloading github.com/gorilla/context v0.0.0-20140604161150-14f550f51af5
            go: downloading github.com/MSOpenTech/azure-sdk-for-go v0.0.0-20150323223030-d90753bcad2e
            go: downloading github.com/bugsnag/panicwrap v0.0.0-20141110184334-e5f9854865b9
            go: downloading github.com/docker/libtrust v0.0.0-20150114040149-fa567046d9b1
            go: downloading github.com/codegangsta/cli v1.4.2-0.20150131031259-6086d7927ec3
            go: downloading github.com/AdRoll/goamz v0.0.0-20150130162828-d3664b76d905
            go: downloading github.com/garyburd/redigo v0.0.0-20150301180006-535138d7bcd7
            go: downloading github.com/gorilla/handlers v0.0.0-20140825150757-0e84b7d810c1
            go: downloading github.com/gorilla/mux v0.0.0-20140926153814-e444e69cbd2e
            go: downloading github.com/jlhawn/go-crypto v0.0.0-20150401213827-cd738dde20f0
            go: downloading github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43
            go: downloading github.com/yvasiyarov/gorelic v0.0.7-0.20141212073537-a9bba5b9ab50
            go: downloading github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f
            go: downloading golang.org/x/net v0.0.0-20150202051010-1dfe7915deaf
            go: downloading gopkg.in/check.v1 v1.0.0-20141024133853-64131543e789
            go: downloading gopkg.in/yaml.v2 v2.0.0-20150116202057-bef53efd0c76
            go: converting Godeps\Godeps.json: stat code.google.com/p/go-uuid/uuid@35bc42037350f0078e3c974c6ea690f1926603ab: unrecognized import path "code.google.com/p/go-uuid/uuid": parse https://code.google.com/p/go-uuid/uuid?go-get=1: no go-import meta tags ()
            go: converting Godeps\Godeps.json: stat github.com/docker/docker/vendor/src/code.google.com/p/go/src/pkg/archive/tar@165ea5c158cff3fc40d476ffe233a5ccc03e7d61: module github.com/docker/docker@165ea5c158cff3fc40d476ffe233a5ccc03e7d61 found (v1.4.2-0.20150204013315-165ea5c158cf), but does not contain package github.com/docker/docker/vendor/src/code.google.com/p/go/src/pkg/archive/tar
            go: copying requirements from Godeps/Godeps.json
            go: to add module requirements and sums:
            	go mod tidy
            > cmpenv go.mod go.mod.want
            > go mod download github.com/fishy/gcsbucket@v0.0.0-20180217031846-618d60fe84e0
            > cp $GOPATH/pkg/mod/github.com/fishy/gcsbucket@v0.0.0-20180217031846-618d60fe84e0/Gopkg.lock ../y
            > cd ../y
            $WORK\gopath\src\y
            > go mod init github.com/fishy/gcsbucket
            [stderr]
            go: creating new go.mod: module github.com/fishy/gcsbucket
            go: downloading github.com/golang/protobuf v1.0.0
            go: downloading cloud.google.com/go v0.18.0
            go: downloading google.golang.org/grpc v1.10.0
            go: downloading google.golang.org/appengine v1.0.0
            go: downloading github.com/fishy/fsdb v0.0.0-20180217030800-5527ded01371
            go: downloading google.golang.org/genproto v0.0.0-20180206005123-2b5a72b8730b
            go: downloading golang.org/x/text v0.3.1-0.20180208041248-4e4a3210bb54
            go: downloading github.com/googleapis/gax-go v2.0.0+incompatible
            go: downloading golang.org/x/oauth2 v0.0.0-20180207181906-543e37812f10
            go: downloading golang.org/x/net v0.0.0-20180216171745-136a25c244d3
            go: converting Gopkg.lock: stat google.golang.org/api@c7a403bb5fe12fe4644bb956edd5cd677626120a: git init --bare in $WORK\gopath\pkg\mod\cache\vcs\14bb952cc8902e20fb82cb08a9d29b90b010f5d6d0d3737e9cb402b3b57a25b5: exec: "git": executable file not found in %PATH%
            go: copying requirements from Gopkg.lock
            > cmpenv go.mod go.mod.want
            [diff -go.mod +go.mod.want]
             module github.com/fishy/gcsbucket
             
             go 1.17
             
             require (
             	cloud.google.com/go v0.18.0
             	github.com/fishy/fsdb v0.0.0-20180217030800-5527ded01371
             	github.com/golang/protobuf v1.0.0
             	github.com/googleapis/gax-go v2.0.0+incompatible
             	golang.org/x/net v0.0.0-20180216171745-136a25c244d3
             	golang.org/x/oauth2 v0.0.0-20180207181906-543e37812f10
             	golang.org/x/text v0.3.1-0.20180208041248-4e4a3210bb54
            +	google.golang.org/api v0.0.0-20180217000815-c7a403bb5fe1
             	google.golang.org/appengine v1.0.0
             	google.golang.org/genproto v0.0.0-20180206005123-2b5a72b8730b
             	google.golang.org/grpc v1.10.0
             )
            
            FAIL: testdata\script\mod_convert.txt:18: go.mod and go.mod.want differ
            
FAIL
FAIL	cmd/go	199.933s

CC @matloob @jayconrod @aofei
@bcmills bcmills added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. GoCommand cmd/go release-blocker modules okay-after-beta1 Used by release team to mark a release-blocker issue as okay to resolve either before or after beta1 labels Apr 21, 2021
@bcmills bcmills added this to the Go1.17 milestone Apr 21, 2021
@bcmills
Copy link
Contributor Author

bcmills commented Apr 21, 2021

(This initially masked #45658.)

@aofei
Copy link
Contributor

aofei commented Apr 22, 2021

This is weird, https://proxy.golang.org/google.golang.org/api/@v/c7a403bb5fe12fe4644bb956edd5cd677626120a.info returns 200 OK, it shouldn't fall back to direct. I haven't reproduced this failure yet.

@aofei
Copy link
Contributor

aofei commented Apr 22, 2021 •
edited

Okay, another weird thing, https://proxy.golang.org/google.golang.org/api/@v/c7a403bb5fe12fe4644bb956edd5cd677626120a.info occasionally (maybe every 1800s) hangs and then returns not found: fetch timed out. I believe this is the reason that fetching occasionally falls back to direct. @bcmills

aofei@as-server:~$ curl -i https://proxy.golang.org/google.golang.org/api/@v/c7a403bb5fe12fe4644bb956edd5cd677626120a.info
HTTP/2 410
access-control-allow-origin: *
cache-control: public, max-age=60
content-type: text/plain; charset=UTF-8
date: Thu, 22 Apr 2021 06:54:51 GMT
expires: Thu, 22 Apr 2021 06:55:51 GMT
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-xss-protection: 0
content-length: 26

not found: fetch timed out

@gopherbot
Copy link

Change https://golang.org/cl/312629 mentions this issue: cmd/go: update TestScript/mod_convert

@aofei
Copy link
Contributor

aofei commented Apr 22, 2021

CL 312629 may be able to fix this failure.

Incidentally, I think proxy.golang.org should probably consider persisting the full commit hash (maybe short commit hash too) lookup result instead of just caching it for a short period of time (currently 1800s). Since they have never changed, it's really unnecessary to recalculate it over and over again. Those operations are time-consuming and occasionally time out.

@bcmills
Copy link
Contributor Author

bcmills commented Apr 22, 2021

Since they have never changed, it's really unnecessary to recalculate it over and over again.

Unfortunately, the “most appropriate” canonical version for a given commit hash can change over time. (For example, if a release tag is added to the commit we should prefer that release tag over the previous pseudo-version for the commit, even though the pseudo-version also remains valid.)

@bcmills
Copy link
Contributor Author

bcmills commented Apr 22, 2021

(And by the way, thanks for the great analysis and CL, @aofei!)

@golang golang locked and limited conversation to collaborators Apr 22, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge GoCommand cmd/go modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. okay-after-beta1 Used by release team to mark a release-blocker issue as okay to resolve either before or after beta1 release-blocker
Projects
None yet
Milestone
Go1.17
Development

No branches or pull requests
3 participants
@aofei @bcmills @gopherbot