Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: many tests in -short mode overwrite installed packages #22965

Closed
rsc opened this issue Dec 1, 2017 · 2 comments
Closed

cmd/go: many tests in -short mode overwrite installed packages #22965

rsc opened this issue Dec 1, 2017 · 2 comments
Labels
FrozenDueToAge
Milestone
Go1.10

Comments

@rsc
Copy link
Contributor

rsc commented Dec 1, 2017

Found while debugging #22709 (and possibly root cause of that).
I ran

GOARCH=386 ./all.bash

and when the ##### Testing packages banner was printed, I ran in another window

cd $GOROOT/pkg
chmod -R -w tool darwin_amd64 darwin_386 && sudo chown -R root tool darwin_amd64 darwin_386

and then a while later the cmd/go test failed in the original window:

--- FAIL: TestRelativeImportsGoTestDashI (0.11s)
	go_test.go:1452: running testgo [test -i ./testdata/testimport]
	go_test.go:1452: standard error:
	go_test.go:1452: go test errors: open /Users/rsc/go/pkg/darwin_386/errors.a: permission denied
		go test math/bits: open /Users/rsc/go/pkg/darwin_386/math/bits.a: permission denied
		go test internal/race: open /Users/rsc/go/pkg/darwin_386/internal/race.a: permission denied
		go test unicode: open /Users/rsc/go/pkg/darwin_386/unicode.a: permission denied
		go test unicode/utf8: open /Users/rsc/go/pkg/darwin_386/unicode/utf8.a: permission denied
		go test sync/atomic: open /Users/rsc/go/pkg/darwin_386/sync/atomic.a: permission denied
		go test runtime/internal/sys: open /Users/rsc/go/pkg/darwin_386/runtime/internal/sys.a: permission denied
		go test runtime/internal/atomic: open /Users/rsc/go/pkg/darwin_386/runtime/internal/atomic.a: permission denied
		go test math: open /Users/rsc/go/pkg/darwin_386/math.a: permission denied
		
	go_test.go:1452: go [test -i ./testdata/testimport] failed unexpectedly: exit status 1
--- FAIL: TestPackageNotStaleWithTrailingSlash (0.07s)
	go_test.go:1576: running testgo [install runtime os io]
	go_test.go:1576: standard error:
	go_test.go:1576: go install runtime: open /Users/rsc/go/pkg/darwin_386/runtime.a: permission denied
		
	go_test.go:1576: go [install runtime os io] failed unexpectedly: exit status 1
--- FAIL: TestGoVetWithExternalTests (2.24s)
	go_test.go:3070: running testgo [install cmd/vet]
	go_test.go:3070: standard error:
	go_test.go:3070: go install cmd/vet: open /Users/rsc/go/pkg/tool/darwin_386/vet: permission denied
		
	go_test.go:3070: go [install cmd/vet] failed unexpectedly: exit status 1
--- FAIL: TestGoVetWithTags (0.50s)
	go_test.go:3080: running testgo [install cmd/vet]
	go_test.go:3080: standard error:
	go_test.go:3080: go install cmd/vet: open /Users/rsc/go/pkg/tool/darwin_386/vet: permission denied
		
	go_test.go:3080: go [install cmd/vet] failed unexpectedly: exit status 1
--- FAIL: TestGoVetWithFlagsOn (0.48s)
	go_test.go:3090: running testgo [install cmd/vet]
	go_test.go:3090: standard error:
	go_test.go:3090: go install cmd/vet: open /Users/rsc/go/pkg/tool/darwin_386/vet: permission denied
		
	go_test.go:3090: go [install cmd/vet] failed unexpectedly: exit status 1
--- FAIL: TestGoVetWithFlagsOff (0.48s)
	go_test.go:3100: running testgo [install cmd/vet]
	go_test.go:3100: standard error:
	go_test.go:3100: go install cmd/vet: open /Users/rsc/go/pkg/tool/darwin_386/vet: permission denied
		
	go_test.go:3100: go [install cmd/vet] failed unexpectedly: exit status 1
--- FAIL: TestBuildTagsNoComma (0.03s)
	go_test.go:4324: running testgo [install -tags tag1 tag2 math]
	go_test.go:4324: standard error:
	go_test.go:4324: go install math: open /Users/rsc/go/pkg/darwin_386/math.a: permission denied
		
	go_test.go:4324: go [install -tags tag1 tag2 math] failed unexpectedly: exit status 1
--- FAIL: TestGcflagsPatterns (1.36s)
	go_test.go:5207: running testgo [build -v -gcflags= 	
		 -e fmt]
	go_test.go:5207: standard error:
	go_test.go:5207: errors
		internal/race
		unicode/utf8
		sync/atomic
		runtime/internal/atomic
		unicode
		runtime/internal/sys
		math
		runtime
		strconv
		sync
		io
		reflect
		syscall
		time
		internal/poll
		os
		fmt
		
	go_test.go:5209: incorrectly rebuilt reflect
	go_test.go:5209: pattern reflect found unexpectedly in standard error
--- FAIL: TestInstallDeps (0.69s)
	go_test.go:5158: running testgo [list -f={{.Target}} p1]
	go_test.go:5158: standard output:
	go_test.go:5158: /var/folders/qk/vp4mp_hj5qlb_1_w97rgqc4r000n9d/T/gotest206586869/pkg/darwin_386/p1.a
		
	go_test.go:5160: running testgo [list -f={{.Target}} p2]
	go_test.go:5160: standard output:
	go_test.go:5160: /var/folders/qk/vp4mp_hj5qlb_1_w97rgqc4r000n9d/T/gotest206586869/pkg/darwin_386/p2.a
		
	go_test.go:5162: running testgo [list -f={{.Target}} main1]
	go_test.go:5162: standard output:
	go_test.go:5162: /var/folders/qk/vp4mp_hj5qlb_1_w97rgqc4r000n9d/T/gotest206586869/bin/main1
		
	go_test.go:5165: running testgo [install main1]
	go_test.go:5171: running testgo [install p2]
	go_test.go:5175: running testgo [install -i main1]
	go_test.go:5175: standard error:
	go_test.go:5175: go install runtime/internal/sys: open /Users/rsc/go/pkg/darwin_386/runtime/internal/sys.a: permission denied
		go install runtime/internal/atomic: open /Users/rsc/go/pkg/darwin_386/runtime/internal/atomic.a: permission denied
		
	go_test.go:5175: go [install -i main1] failed unexpectedly: exit status 1
--- FAIL: TestIssue22588 (0.16s)
	go_test.go:4945: running testgo [list -f={{.Stale}} runtime]
	go_test.go:4945: standard output:
	go_test.go:4945: true
		
	go_test.go:4946: running testgo [list -toolexec=/usr/bin/time -f={{.Stale}} runtime]
	go_test.go:4946: standard output:
	go_test.go:4946: true
		
	go_test.go:4947: incorrectly reported runtime as stale
	go_test.go:4947: pattern false not found in standard output
--- FAIL: TestBuildDashIInstallsDependencies (1.13s)
	go_test.go:2892: running testgo [build -v -i x/y/bar]
	go_test.go:2892: standard error:
	go_test.go:2892: x/y/foo
		x/y/bar
		
	go_test.go:2894: running testgo [build -v -i x/y/bar]
	go_test.go:2892: running testgo [build -v -i x/y/bar]
	go_test.go:2892: standard error:
	go_test.go:2892: x/y/foo
		go build runtime/internal/atomic: open /Users/rsc/go/pkg/darwin_386/runtime/internal/atomic.a: permission denied
		go build runtime/internal/sys: open /Users/rsc/go/pkg/darwin_386/runtime/internal/sys.a: permission denied
		
	go_test.go:2892: go [build -v -i x/y/bar] failed unexpectedly: exit status 1
--- FAIL: TestIssue22531 (5.32s)
	go_test.go:4961: running testgo [install -x m]
	go_test.go:4961: standard error:
	go_test.go:4961: WORK=/var/folders/qk/vp4mp_hj5qlb_1_w97rgqc4r000n9d/T/go-build414480557
		mkdir -p $WORK/b003/
		mkdir -p $WORK/b005/
		cd /Users/rsc/go/src/runtime/internal/atomic
		/Users/rsc/go/pkg/tool/darwin_386/compile -o $WORK/b003/_pkg_.a -trimpath $WORK/b003 -p runtime/internal/atomic -std -+ -buildid STdC8uxLn3BPr64Lj_uP/STdC8uxLn3BPr64Lj_uP -D "" -pack -asmhdr $WORK/b003/go_asm.h -c=4 ./atomic_386.go ./stubs.go
		cd /Users/rsc/go/src/runtime/internal/sys
		/Users/rsc/go/pkg/tool/darwin_386/compile -o $WORK/b005/_pkg_.a -trimpath $WORK/b005 -p runtime/internal/sys -std -+ -buildid s0EW6otPs3FdIcHFqcPK/s0EW6otPs3FdIcHFqcPK -D "" -pack -asmhdr $WORK/b005/go_asm.h -c=4 ./arch.go ./arch_386.go ./intrinsics_stubs.go ./stubs.go ./sys.go ./zgoarch_386.go ./zgoos_darwin.go ./zversion.go
		/Users/rsc/go/pkg/tool/darwin_386/asm -trimpath $WORK/b005 -I $WORK/b005/ -I /Users/rsc/go/pkg/include -D GOOS_darwin -D GOARCH_386 -o $WORK/b005/intrinsics_386.o ./intrinsics_386.s
		cd /Users/rsc/go/src/runtime/internal/atomic
		/Users/rsc/go/pkg/tool/darwin_386/asm -trimpath $WORK/b003 -I $WORK/b003/ -I /Users/rsc/go/pkg/include -D GOOS_darwin -D GOARCH_386 -o $WORK/b003/asm_386.o ./asm_386.s
		cd /Users/rsc/go/src/runtime/internal/sys
		/Users/rsc/go/pkg/tool/darwin_386/pack r $WORK/b005/_pkg_.a $WORK/b005/intrinsics_386.o # internal
		/Users/rsc/go/pkg/tool/darwin_386/buildid -w $WORK/b005/_pkg_.a # internal
		cp $WORK/b005/_pkg_.a /var/folders/qk/vp4mp_hj5qlb_1_w97rgqc4r000n9d/T/gotest288394745/cache/ee/ee3099e891785d5359b886476bb062086871e10f9276fe0f6f38bef03ac5dc1d-d # internal
		cd /Users/rsc/go/src/runtime/internal/atomic
		/Users/rsc/go/pkg/tool/darwin_386/pack r $WORK/b003/_pkg_.a $WORK/b003/asm_386.o # internal
		/Users/rsc/go/pkg/tool/darwin_386/buildid -w $WORK/b003/_pkg_.a # internal
		cp $WORK/b003/_pkg_.a /var/folders/qk/vp4mp_hj5qlb_1_w97rgqc4r000n9d/T/gotest288394745/cache/6e/6e83fde00e8ff3faa9d5273860ef1cebe2c2c58e5e11895045bb1d1578c221dc-d # internal
		mkdir -p $WORK/b002/
		cat >$WORK/b002/importcfg << 'EOF' # internal
		packagefile runtime/internal/atomic=$WORK/b003/_pkg_.a
		packagefile runtime/internal/sys=$WORK/b005/_pkg_.a
		EOF
		cd /Users/rsc/go/src/runtime
		/Users/rsc/go/pkg/tool/darwin_386/compile -o $WORK/b002/_pkg_.a -trimpath $WORK/b002 -p runtime -std -+ -buildid dBVxNbFznfVta5YOiXLs/dBVxNbFznfVta5YOiXLs -D "" -importcfg $WORK/b002/importcfg -pack -asmhdr $WORK/b002/go_asm.h -c=4 ./alg.go ./atomic_pointer.go ./cgo.go ./cgocall.go ./cgocallback.go ./cgocheck.go ./chan.go ./compiler.go ./complex.go ./cpuprof.go ./cputicks.go ./debug.go ./defs_darwin_386.go ./env_posix.go ./error.go ./extern.go ./fastlog2.go ./fastlog2table.go ./float.go ./hash32.go ./hashmap.go ./hashmap_fast.go ./heapdump.go ./iface.go ./lfstack.go ./lfstack_32bit.go ./lock_sema.go ./malloc.go ./mbarrier.go ./mbitmap.go ./mcache.go ./mcentral.go ./mem_darwin.go ./mfinal.go ./mfixalloc.go ./mgc.go ./mgclarge.go ./mgcmark.go ./mgcsweep.go ./mgcsweepbuf.go ./mgcwork.go ./mheap.go ./mmap.go ./mprof.go ./msan0.go ./msize.go ./mstats.go ./mwbbuf.go ./netpoll.go ./netpoll_kqueue.go ./os_darwin.go ./panic.go ./plugin.go ./print.go ./proc.go ./profbuf.go ./proflabel.go ./race0.go ./rdebug.go ./relax_stub.go ./runtime.go ./runtime1.go ./runtime2.go ./rwmutex.go ./select.go ./sema.go ./signal_386.go ./signal_darwin.go ./signal_darwin_386.go ./signal_sighandler.go ./signal_unix.go ./sigqueue.go ./sizeclasses.go ./slice.go ./softfloat64.go ./sqrt.go ./stack.go ./string.go ./stubs.go ./stubs2.go ./stubs32.go ./stubs_asm.go ./stubs_nonlinux.go ./symtab.go ./sys_nonppc64x.go ./sys_x86.go ./time.go ./timeasm.go ./trace.go ./traceback.go ./type.go ./typekind.go ./unaligned1.go ./utf8.go ./vlrt.go ./write_err.go
		/Users/rsc/go/pkg/tool/darwin_386/asm -trimpath $WORK/b002 -I $WORK/b002/ -I /Users/rsc/go/pkg/include -D GOOS_darwin -D GOARCH_386 -o $WORK/b002/asm.o ./asm.s
		/Users/rsc/go/pkg/tool/darwin_386/asm -trimpath $WORK/b002 -I $WORK/b002/ -I /Users/rsc/go/pkg/include -D GOOS_darwin -D GOARCH_386 -o $WORK/b002/asm_386.o ./asm_386.s
		/Users/rsc/go/pkg/tool/darwin_386/asm -trimpath $WORK/b002 -I $WORK/b002/ -I /Users/rsc/go/pkg/include -D GOOS_darwin -D GOARCH_386 -o $WORK/b002/duff_386.o ./duff_386.s
		/Users/rsc/go/pkg/tool/darwin_386/asm -trimpath $WORK/b002 -I $WORK/b002/ -I /Users/rsc/go/pkg/include -D GOOS_darwin -D GOARCH_386 -o $WORK/b002/memclr_386.o ./memclr_386.s
		/Users/rsc/go/pkg/tool/darwin_386/asm -trimpath $WORK/b002 -I $WORK/b002/ -I /Users/rsc/go/pkg/include -D GOOS_darwin -D GOARCH_386 -o $WORK/b002/memmove_386.o ./memmove_386.s
		/Users/rsc/go/pkg/tool/darwin_386/asm -trimpath $WORK/b002 -I $WORK/b002/ -I /Users/rsc/go/pkg/include -D GOOS_darwin -D GOARCH_386 -o $WORK/b002/rt0_darwin_386.o ./rt0_darwin_386.s
		/Users/rsc/go/pkg/tool/darwin_386/asm -trimpath $WORK/b002 -I $WORK/b002/ -I /Users/rsc/go/pkg/include -D GOOS_darwin -D GOARCH_386 -o $WORK/b002/sys_darwin_386.o ./sys_darwin_386.s
		/Users/rsc/go/pkg/tool/darwin_386/asm -trimpath $WORK/b002 -I $WORK/b002/ -I /Users/rsc/go/pkg/include -D GOOS_darwin -D GOARCH_386 -o $WORK/b002/vlop_386.o ./vlop_386.s
		/Users/rsc/go/pkg/tool/darwin_386/pack r $WORK/b002/_pkg_.a $WORK/b002/asm.o $WORK/b002/asm_386.o $WORK/b002/duff_386.o $WORK/b002/memclr_386.o $WORK/b002/memmove_386.o $WORK/b002/rt0_darwin_386.o $WORK/b002/sys_darwin_386.o $WORK/b002/vlop_386.o # internal
		/Users/rsc/go/pkg/tool/darwin_386/buildid -w $WORK/b002/_pkg_.a # internal
		cp $WORK/b002/_pkg_.a /var/folders/qk/vp4mp_hj5qlb_1_w97rgqc4r000n9d/T/gotest288394745/cache/05/053a128a95b05122fe5bf8b82e4ce5c8882d8d48db78221ead8a7d35e77f8e35-d # internal
		mkdir -p $WORK/b001/
		cat >$WORK/b001/importcfg << 'EOF' # internal
		packagefile runtime=$WORK/b002/_pkg_.a
		EOF
		cd /var/folders/qk/vp4mp_hj5qlb_1_w97rgqc4r000n9d/T/gotest288394745/src/m
		/Users/rsc/go/pkg/tool/darwin_386/compile -o $WORK/b001/_pkg_.a -trimpath $WORK/b001 -p main -complete -buildid 5mEJQI3z8OVSdBlx6L98/5mEJQI3z8OVSdBlx6L98 -D "" -importcfg $WORK/b001/importcfg -pack -c=4 ./main.go
		/Users/rsc/go/pkg/tool/darwin_386/buildid -w $WORK/b001/_pkg_.a # internal
		cp $WORK/b001/_pkg_.a /var/folders/qk/vp4mp_hj5qlb_1_w97rgqc4r000n9d/T/gotest288394745/cache/ef/ef6298497e7f7e33d27d60927a3bcd204d4e776722e1b517ce4df976f794ca28-d # internal
		cat >$WORK/b001/importcfg.link << 'EOF' # internal
		packagefile m=$WORK/b001/_pkg_.a
		packagefile runtime=$WORK/b002/_pkg_.a
		packagefile runtime/internal/atomic=$WORK/b003/_pkg_.a
		packagefile runtime/internal/sys=$WORK/b005/_pkg_.a
		EOF
		mkdir -p $WORK/b001/exe/
		cd .
		/Users/rsc/go/pkg/tool/darwin_386/link -o $WORK/b001/exe/a.out -importcfg $WORK/b001/importcfg.link -buildmode=exe -buildid=kv31CPzzYEV8Wb4yYE84/5mEJQI3z8OVSdBlx6L98/hJR9XtxD_JMA3anjzy-Z/kv31CPzzYEV8Wb4yYE84 -extld=clang $WORK/b001/_pkg_.a
		/Users/rsc/go/pkg/tool/darwin_386/buildid -w $WORK/b001/exe/a.out # internal
		mkdir -p /var/folders/qk/vp4mp_hj5qlb_1_w97rgqc4r000n9d/T/gotest288394745/bin/
		mv $WORK/b001/exe/a.out /var/folders/qk/vp4mp_hj5qlb_1_w97rgqc4r000n9d/T/gotest288394745/bin/m
		rm -r $WORK/b001/
		
	go_test.go:4962: running testgo [list -f {{.Stale}} m]
	go_test.go:4962: standard output:
	go_test.go:4962: true
		
	go_test.go:4963: reported m as stale after install
	go_test.go:4963: pattern false not found in standard output
--- FAIL: TestGoTestDashIDashOWritesBinary (0.11s)
	go_test.go:2081: running testgo [test -v -i -o /var/folders/qk/vp4mp_hj5qlb_1_w97rgqc4r000n9d/T/gotest662367651/myerrors.test errors]
	go_test.go:2081: standard error:
	go_test.go:2081: go test errors: open /Users/rsc/go/pkg/darwin_386/errors.a: permission denied
		go test internal/race: open /Users/rsc/go/pkg/darwin_386/internal/race.a: permission denied
		go test math/bits: open /Users/rsc/go/pkg/darwin_386/math/bits.a: permission denied
		go test unicode/utf8: open /Users/rsc/go/pkg/darwin_386/unicode/utf8.a: permission denied
		go test runtime/internal/atomic: open /Users/rsc/go/pkg/darwin_386/runtime/internal/atomic.a: permission denied
		go test unicode: open /Users/rsc/go/pkg/darwin_386/unicode.a: permission denied
		go test sync/atomic: open /Users/rsc/go/pkg/darwin_386/sync/atomic.a: permission denied
		go test runtime/internal/sys: open /Users/rsc/go/pkg/darwin_386/runtime/internal/sys.a: permission denied
		go test math: open /Users/rsc/go/pkg/darwin_386/math.a: permission denied
		
	go_test.go:2081: go [test -v -i -o /var/folders/qk/vp4mp_hj5qlb_1_w97rgqc4r000n9d/T/gotest662367651/myerrors.test errors] failed unexpectedly: exit status 1
--- FAIL: TestGoInstallDetectsRemovedFilesInPackageMain (0.30s)
	go_test.go:1062: running testgo [install mycmd]
	go_test.go:1063: running testgo [list -f {{.Stale}}:{{.StaleReason}} mycmd]
	go_test.go:1063: standard output:
	go_test.go:1063: true:stale dependency: runtime/internal/atomic
		
	go_test.go:1063: ./testgo list mypkg claims mycmd is stale, incorrectly
--- FAIL: TestPackageMainTestImportsArchiveNotBinary (1.75s)
	go_test.go:1558: running testgo [test main_test]
	go_test.go:1558: standard output:
	go_test.go:1558: ok  	main_test	0.017s
		
	go_test.go:1559: running testgo [install main_test]
	go_test.go:1560: running testgo [list -f {{.Stale}}:{{.StaleReason}} main_test]
	go_test.go:1560: standard output:
	go_test.go:1560: true:stale dependency: runtime/internal/atomic
		
	go_test.go:1560: after go install, main listed as stale
FAIL
FAIL	cmd/go	62.291s
@rsc rsc added this to the Go1.10 milestone Dec 1, 2017
@rsc
Copy link
Contributor Author

rsc commented Dec 1, 2017

Most of these appear to be tests that usually don't reinstall packages but are doing so because in this cross-compiling setting the packages look stale.

This may be why some cross-compiling builders have gotten slower than expected too: things look stale and are being reinstalled when they should not be.

@gopherbot
Copy link

Change https://golang.org/cl/81516 mentions this issue: cmd/go: disable tests when GOOS/GOARCH != GOHOSTARCH/GOHOSTARCH

@golang golang locked and limited conversation to collaborators Dec 4, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
Go1.10
Development

No branches or pull requests
2 participants
@rsc @gopherbot