Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: TestBuildCache failing on Plan 9 #22572

Closed
0intro opened this issue Nov 3, 2017 · 7 comments
Closed

cmd/go: TestBuildCache failing on Plan 9 #22572

0intro opened this issue Nov 3, 2017 · 7 comments
Labels
FrozenDueToAge OS-Plan9
Milestone
Go1.10

Comments

@0intro
Copy link
Member

0intro commented Nov 3, 2017

CL 75473 added TestBuildCache. This test is failing on Plan 9.

--- FAIL: TestBuildCache (2.73s)
	go_test.go:4719: running testgo [build -x complex/w]
	go_test.go:4719: standard error:
	go_test.go:4719: WORK=/tmp/go-build120783117
		mkdir -p $WORK/b001/
		cat >$WORK/b001/importcfg << 'EOF' # internal
		packagefile runtime=/tmp/workdir/go/pkg/plan9_386/runtime.a
		EOF
		cd /tmp/workdir/go/src/cmd/go/testdata/src/complex/w
		/tmp/workdir/go/pkg/tool/plan9_386/compile -o $WORK/b001/_pkg_.a -trimpath $WORK/b001 -p complex/w -complete -buildid jrm1srpTyde9pGMsjSZX/jrm1srpTyde9pGMsjSZX -dwarf=false -D "" -importcfg $WORK/b001/importcfg -pack ./w.go
		/tmp/workdir/go/pkg/tool/plan9_386/buildid -w $WORK/b001/_pkg_.a # internal
		
	go_test.go:4722: running testgo [build -x complex/w]
	go_test.go:4722: standard error:
	go_test.go:4722: WORK=/tmp/go-build972466103
		
	go_test.go:4727: running testgo [build -o /dev/null -x complex]
	go_test.go:4727: standard error:
	go_test.go:4727: WORK=/tmp/go-build104673264
		mkdir -p $WORK/b003/
		cat >$WORK/b003/importcfg << 'EOF' # internal
		packagefile runtime=/tmp/workdir/go/pkg/plan9_386/runtime.a
		EOF
		cd /tmp/workdir/go/src/cmd/go/testdata/src/complex/nest/vendor/v1
		/tmp/workdir/go/pkg/tool/plan9_386/compile -o $WORK/b003/_pkg_.a -trimpath $WORK/b003 -p complex/nest/vendor/v1 -complete -buildid jaGFPcmwwMb5cp00t5X1/jaGFPcmwwMb5cp00t5X1 -dwarf=false -D "" -importcfg $WORK/b003/importcfg -pack ./v1.go
		/tmp/workdir/go/pkg/tool/plan9_386/buildid -w $WORK/b003/_pkg_.a # internal
		mkdir -p $WORK/b008/
		cat >$WORK/b008/importcfg << 'EOF' # internal
		packagefile runtime=/tmp/workdir/go/pkg/plan9_386/runtime.a
		EOF
		cd /tmp/workdir/go/src/cmd/go/testdata/src/complex/nest/sub/vendor/v2
		/tmp/workdir/go/pkg/tool/plan9_386/compile -o $WORK/b008/_pkg_.a -trimpath $WORK/b008 -p complex/nest/sub/vendor/v2 -complete -buildid PSBsUvtTsfbxL1w4FjPP/PSBsUvtTsfbxL1w4FjPP -dwarf=false -D "" -importcfg $WORK/b008/importcfg -pack ./v2.go
		/tmp/workdir/go/pkg/tool/plan9_386/buildid -w $WORK/b008/_pkg_.a # internal
		mkdir -p $WORK/b002/
		cat >$WORK/b002/importcfg << 'EOF' # internal
		importmap v1=complex/nest/vendor/v1
		importmap v2=complex/nest/sub/vendor/v2
		packagefile complex/nest/vendor/v1=$WORK/b003/_pkg_.a
		packagefile complex/nest/sub/vendor/v2=$WORK/b008/_pkg_.a
		packagefile runtime=/tmp/workdir/go/pkg/plan9_386/runtime.a
		EOF
		cd /tmp/workdir/go/src/cmd/go/testdata/src/complex/nest/sub/test12
		/tmp/workdir/go/pkg/tool/plan9_386/compile -o $WORK/b002/_pkg_.a -trimpath $WORK/b002 -p complex/nest/sub/test12 -complete -buildid gXe9Y0IrN2jwtoaajcub/gXe9Y0IrN2jwtoaajcub -dwarf=false -D "" -importcfg $WORK/b002/importcfg -pack ./p.go
		/tmp/workdir/go/pkg/tool/plan9_386/buildid -w $WORK/b002/_pkg_.a # internal
		mkdir -p $WORK/b010/
		cat >$WORK/b010/importcfg << 'EOF' # internal
		packagefile runtime=/tmp/workdir/go/pkg/plan9_386/runtime.a
		EOF
		cd /tmp/workdir/go/src/cmd/go/testdata/src/complex/nest/vendor/v3
		/tmp/workdir/go/pkg/tool/plan9_386/compile -o $WORK/b010/_pkg_.a -trimpath $WORK/b010 -p complex/nest/vendor/v3 -complete -buildid b1tnFJErFPgHdg_tMtzI/b1tnFJErFPgHdg_tMtzI -dwarf=false -D "" -importcfg $WORK/b010/importcfg -pack ./v3.go
		/tmp/workdir/go/pkg/tool/plan9_386/buildid -w $WORK/b010/_pkg_.a # internal
		mkdir -p $WORK/b009/
		cat >$WORK/b009/importcfg << 'EOF' # internal
		importmap v2=complex/nest/sub/vendor/v2
		importmap v3=complex/nest/vendor/v3
		packagefile complex/nest/sub/vendor/v2=$WORK/b008/_pkg_.a
		packagefile complex/nest/vendor/v3=$WORK/b010/_pkg_.a
		packagefile runtime=/tmp/workdir/go/pkg/plan9_386/runtime.a
		EOF
		cd /tmp/workdir/go/src/cmd/go/testdata/src/complex/nest/sub/test23
		/tmp/workdir/go/pkg/tool/plan9_386/compile -o $WORK/b009/_pkg_.a -trimpath $WORK/b009 -p complex/nest/sub/test23 -complete -buildid ckj1RVv_AzUCz6RFyA0R/ckj1RVv_AzUCz6RFyA0R -dwarf=false -D "" -importcfg $WORK/b009/importcfg -pack ./p.go
		/tmp/workdir/go/pkg/tool/plan9_386/buildid -w $WORK/b009/_pkg_.a # internal
		mkdir -p $WORK/b012/
		cat >$WORK/b012/importcfg << 'EOF' # internal
		packagefile runtime=/tmp/workdir/go/pkg/plan9_386/runtime.a
		EOF
		cd /tmp/workdir/go/src/cmd/go/testdata/src/complex/vendor/v
		/tmp/workdir/go/pkg/tool/plan9_386/compile -o $WORK/b012/_pkg_.a -trimpath $WORK/b012 -p complex/vendor/v -complete -buildid nq61HlFmn2FmkvzXMXCu/nq61HlFmn2FmkvzXMXCu -dwarf=false -D "" -importcfg $WORK/b012/importcfg -pack ./v.go
		/tmp/workdir/go/pkg/tool/plan9_386/buildid -w $WORK/b012/_pkg_.a # internal
		mkdir -p $WORK/b001/
		cat >$WORK/b001/importcfg << 'EOF' # internal
		importmap v=complex/vendor/v
		packagefile complex/nest/sub/test12=$WORK/b002/_pkg_.a
		packagefile complex/nest/sub/test23=$WORK/b009/_pkg_.a
		packagefile complex/w=/tmp/gotest243486064/b4/b45c6bf5f2ca094df80c6946b0da8c248f8b74746c128f43949443a2961a134c-d
		packagefile complex/vendor/v=$WORK/b012/_pkg_.a
		packagefile runtime=/tmp/workdir/go/pkg/plan9_386/runtime.a
		EOF
		cd /tmp/workdir/go/src/cmd/go/testdata/src/complex
		/tmp/workdir/go/pkg/tool/plan9_386/compile -o $WORK/b001/_pkg_.a -trimpath $WORK/b001 -p main -complete -buildid 53v12b2i83wlo8skxF2N/53v12b2i83wlo8skxF2N -dwarf=false -D "" -importcfg $WORK/b001/importcfg -pack ./main.go
		/tmp/workdir/go/pkg/tool/plan9_386/buildid -w $WORK/b001/_pkg_.a # internal
		cat >$WORK/b001/importcfg.link << 'EOF' # internal
		packagefile complex=$WORK/b001/_pkg_.a
		packagefile complex/nest/sub/test12=$WORK/b002/_pkg_.a
		packagefile complex/nest/sub/test23=$WORK/b009/_pkg_.a
		packagefile complex/w=/tmp/gotest243486064/b4/b45c6bf5f2ca094df80c6946b0da8c248f8b74746c128f43949443a2961a134c-d
		packagefile complex/vendor/v=$WORK/b012/_pkg_.a
		packagefile runtime=/tmp/workdir/go/pkg/plan9_386/runtime.a
		packagefile complex/nest/vendor/v1=$WORK/b003/_pkg_.a
		packagefile complex/nest/sub/vendor/v2=$WORK/b008/_pkg_.a
		packagefile complex/nest/vendor/v3=$WORK/b010/_pkg_.a
		packagefile runtime/internal/atomic=/tmp/workdir/go/pkg/plan9_386/runtime/internal/atomic.a
		packagefile runtime/internal/sys=/tmp/workdir/go/pkg/plan9_386/runtime/internal/sys.a
		EOF
		mkdir -p $WORK/b001/exe/
		cd .
		/tmp/workdir/go/pkg/tool/plan9_386/link -o $WORK/b001/exe/a.out -importcfg $WORK/b001/importcfg.link -buildmode=exe -buildid=_pS5NLbUfx3IVQFFOTA-/53v12b2i83wlo8skxF2N/wfagzuhH-MMc7fRNMExt/_pS5NLbUfx3IVQFFOTA- -extld=8c $WORK/b001/_pkg_.a
		/tmp/workdir/go/pkg/tool/plan9_386/buildid -w $WORK/b001/exe/a.out # internal
		
	go_test.go:4730: running testgo [build -o /dev/null -x complex]
	go_test.go:4730: standard error:
	go_test.go:4730: WORK=/tmp/go-build555282900
		mkdir -p $WORK/b001/
		cat >$WORK/b001/importcfg.link << 'EOF' # internal
		packagefile complex=/tmp/gotest243486064/1c/1c8c3414aafb228a08ed53d79082d81fc4f67231cbc29883f56ef9a1e86efa21-d
		packagefile complex/nest/sub/test12=/tmp/gotest243486064/e4/e4bbe4d8d88ed120426b0b2ba0f7c0e002baf6af77527e0fd96f8a486e5b033d-d
		packagefile complex/nest/sub/test23=/tmp/gotest243486064/cc/ccf8e65d86c08c10fdb4ab6fee3ce87c23fd859ceaad43ca82072fbe882b6d38-d
		packagefile complex/w=/tmp/gotest243486064/b4/b45c6bf5f2ca094df80c6946b0da8c248f8b74746c128f43949443a2961a134c-d
		packagefile complex/vendor/v=/tmp/gotest243486064/57/57f0cecba8471d893e0e4b69117e909afe0c93235198e1c6f2acb6776eb3d231-d
		packagefile runtime=/tmp/workdir/go/pkg/plan9_386/runtime.a
		packagefile complex/nest/vendor/v1=/tmp/gotest243486064/de/ded9f3f2e2c5411c29117e9a63d6b4abc6a243a5fb728dd6974f4570d8ff03f5-d
		packagefile complex/nest/sub/vendor/v2=/tmp/gotest243486064/38/38e1170ac8c4d21ee935d8f6f95ad57f39b2e43d3caac4319fdc45e2cc9a7969-d
		packagefile complex/nest/vendor/v3=/tmp/gotest243486064/c8/c81f77e1791c122e24b283b9cef87731eda491dbd3a9a76b8076ebd5abd63765-d
		packagefile runtime/internal/atomic=/tmp/workdir/go/pkg/plan9_386/runtime/internal/atomic.a
		packagefile runtime/internal/sys=/tmp/workdir/go/pkg/plan9_386/runtime/internal/sys.a
		EOF
		mkdir -p $WORK/b001/exe/
		cd .
		/tmp/workdir/go/pkg/tool/plan9_386/link -o $WORK/b001/exe/a.out -importcfg $WORK/b001/importcfg.link -buildmode=exe -buildid=_pS5NLbUfx3IVQFFOTA-/53v12b2i83wlo8skxF2N/wfagzuhH-MMc7fRNMExt/_pS5NLbUfx3IVQFFOTA- -extld=8c /tmp/gotest243486064/1c/1c8c3414aafb228a08ed53d79082d81fc4f67231cbc29883f56ef9a1e86efa21-d
		/tmp/workdir/go/pkg/tool/plan9_386/buildid -w $WORK/b001/exe/a.out # internal
		
	go_test.go:4733: remove /tmp/gotest243486064: directory not empty
FAIL
FAIL	cmd/go	213.356s

See https://build.golang.org/log/5d60306dba3a6090b2f1f630f562ebf1ae872645
@0intro 0intro added the OS-Plan9 label Nov 3, 2017
@0intro 0intro added this to the Go1.10 milestone Nov 3, 2017
@0intro 0intro self-assigned this Nov 3, 2017
@0intro
Copy link
Member Author

0intro commented Nov 3, 2017

This issue only happens on the plan9/386 builder running on GCE when the /tmp directory is served by ramfs.

@0intro
Copy link
Member Author

0intro commented Nov 3, 2017 •
edited 

go_test.go:4733: remove /tmp/gotest243486064: directory not empty

The issue happens because the temporary directory contains a lot of directories and is 15489 bytes.
This size is larger than the largest response that the ramfs server can send, which is the size of a 9P message (8 KB).

Initially, I wasn't able to reproduce this issue on my local machines, because I've increased the maximum size of 9P messages to 16 KB a while ago, mainly for performance reasons.

I will send a new plan9/386 builder image for GCE, with the increased 9P message size.

@gopherbot
Copy link

Change https://golang.org/cl/75990 mentions this issue: env: update Plan 9 image

@0intro
Copy link
Member Author

0intro commented Nov 3, 2017

@bradfitz I've uploaded a new Plan 9 disk image for GCE. Could you deploy it?

http://9legacy.org/download/go/2017-11-03/plan9-386-gce.tar.gz

It should bring the plan9/386 builder back to "ok".

Thanks.

@0intro
Copy link
Member Author

0intro commented Nov 4, 2017

I took another look at this issue.

The issue is related to how read offset is handled in ramfs when reading directories.

The os.RemoveAll function calls fd.Readdirnames to read the 100 first file entries of the directory, then delete the files, then read the 100 following file entries and so on.

However, after the 100 first file entries have been removed, the read offset is not valid anymore when reading the directory. When reading the 100 following files, the offset is applied to the remaining files, so a lot of files are skipped. Consequently, the content of the directory is only partially deleted, and os.RemoveAll returns the directory not empty error when removing the base directory.

This issue doesn't happen on the Fossil file system, because Fossil ignores the read offset, except for the zero value. Fossil handles its own offset internally, and use the zero value to reset its buffer.

The issue also doesn't happen when increasing the maximum size of 9P messages to 16 KB, because 16 KB is enough to hold the full content of the directory, so the subsequent calls to fd.Readdirnames read the directory from the cache.

I'm not sure how to change ramfs to fix this issue yet. Also, other file servers might have the same behavior as ramfs.

Instead, I'll propose a change to call fd.Seek(0, 0) before calling fd.Readdirnames in os.RemoveAll, so the read offset will always be reset after removing the directory entries.

@gopherbot
Copy link

Change https://golang.org/cl/75974 mentions this issue: os: fix RemoveAll on large directories on Plan 9

@0intro
Copy link
Member Author

0intro commented Nov 5, 2017 •
edited

After adding TestRemoveAllLarge to try RemoveAll on large directories in CL 75974, I've noticed that the same issue appears on nacl.

--- FAIL: TestRemoveAllLarge (0.02s)
	path_test.go:192: RemoveAll "/tmp/_TestRemoveAllLarge_" (first): remove /tmp/_TestRemoveAllLarge_: Directory not empty
FAIL
FAIL	os	0.538s

See https://storage.googleapis.com/go-build-log/7e56de12/nacl-amd64p32_c76d189d.log

It seems calling fd.Seek(0, 0) also fixes the issue on nacl.

@0intro 0intro mentioned this issue Nov 13, 2017
Closed
@golang golang locked and limited conversation to collaborators Nov 6, 2018
@rsc rsc unassigned 0intro Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge OS-Plan9
Projects
None yet
Milestone
Go1.10
Development

No branches or pull requests
2 participants
@0intro @gopherbot