The test seems to also fail with the same failure mode on plan9-amd64-0intro from time to time, but less frequently. 🤔

This may be similar to #57747 — is os.Rename atomic on Plan 9?

This may be similar to #57747 — is os.Rename atomic on Plan 9?

No, it doesn't look like it. This is rename in GOROOT/src/os/file_plan9.go, and it does Stat and Remove before updating the directory:

	var d syscall.Dir
	d.Null()
	d.Name = newname

	buf := make([]byte, syscall.STATFIXLEN+len(d.Name))
	n, err := d.Marshal(buf[:])
	if err != nil {
		return &LinkError{"rename", oldname, newname, err}
	}

	// If newname already exists and is not a directory, rename replaces it.
	f, err := Stat(dirname + newname)
	if err == nil && !f.IsDir() {
		Remove(dirname + newname)
	}

	if err = syscall.Wstat(oldname, buf[:n]); err != nil { ... }

The locking fix (pending) should fix this too. Care to take a look at that CL? :)

is os.Rename atomic on Plan 9?

No, it doesn't look like it. This is rename in GOROOT/src/os/file_plan9.go, and it does Stat and Remove before updating the directory:

Suppose instead it calls Wstat first to rename the file, and only if that fails with "new file already exists", it then does the Remove and goes back to retry the Wstat (repeating until success) ... would that make it acceptably atomic?

That wouldn't make it as atomic as POSIX, which I think is what the test expects today.

The POSIX invariant is:

If the link named by the new argument exists, […] a link named new shall remain visible to other threads throughout the renaming operation and refer either to the file referred to by new or old before the operation began.

That is: a rename onto an existing path must not cause the path to even temporarily fail to exist (platform bugs like #33041 notwithstanding).

I'll review Alan's file-locking CL, which should hopefully fix both Windows and Plan 9.

That is: a rename onto an existing path must not cause the path to even temporarily fail to exist

I don't believe that's compatible with Plan 9 semantics; man 5 wstat says:

it is an error to change the name to that of an existing file

The Posix definition seems to be implicitly dependent on rename manipulating links; whereas Plan 9 is not Posix and has no links (hard or symbolic) - a file has one name, which would be part of the inode if Plan 9 had inodes ...

It would be possible to mimic Posix semantics by serialising rename operations with respect to file operations in other goroutines, but the invariant still wouldn't hold across separate simultaneous go programs.

I'll review Alan's file-locking CL, which should hopefully fix both Windows and Plan 9.

Plan 9 doesn't have file locking either. It does implement internal/lockedfile.Open by setting the file's mode to io/fs.ModeExclusive, but that's a bit shaky because it means all subsequent opens of that file are implicitly locking. I've been expecting that to cause problems with go.mod, which is opened sometimes with locking and sometimes without.

The test seems to also fail with the same failure mode on plan9-amd64-0intro from time to time, but less frequently.

I believe plan9-arm is the only Plan 9 builder running on a multiprocessor, so the others are less likely to reveal concurrency bugs. @0intro will correct me if that's not right.

This should be fixed by https://go.dev/cl/461800.