Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: copy/delete instead of rename existing binaries on windows #21997

Closed
as opened this issue Sep 24, 2017 · 8 comments
Closed

cmd/go: copy/delete instead of rename existing binaries on windows #21997

as opened this issue Sep 24, 2017 · 8 comments

Comments

@as
Copy link
Contributor

as commented Sep 24, 2017

A surprising behavior on Windows allows executable on disk associated with running processes to be renamed without error. This behavior only occurs for the rename or 'move' operation, and not 'delete' or 'write'.

Currently, go build will move an old binary like main.exe out of the way by renaming it to main.exe~. However, main.exe may still be associated with a running process. Windows thinks this is OK, so it allows the rename operation to happen without error.

This raises two issues:

1.) It is confusing when I terminate the original process (long after a build is finished), run the associated program again on disk, only to see the program is different in some way, because it has been replaced between executions.

2.) The call to os.Executable is wrong after a rename or move operation on Windows. This might be a separate issue, but its worth mentioning, because it may be unfixable. The userspace process environment block is not updated to reflect the change of the rename operation.

My request is that go build instead copies main.exe -> main.exe~ and then attempts to delete the original main.exe. If that operation fails it can then delete main.exe~. To my knowledge this is compatible with the current behavior of go build on Windows systems.

What version of Go are you using (go version)?

go version go1.9 windows/amd64

Does this issue reproduce with the latest release?

Yes

What did you do?

One line reproduction for Windows

mkdir goissue21997 & cd goissue21997 && echo package main;import "time";import "os";import "fmt"; func main(){println("1");for { time.Sleep(100e6); fmt.Println(os.Executable());};};|gofmt > main.go && go build main.go && start main.exe && echo package main;import "time";import "os";import "fmt"; func main(){println("2");for { time.Sleep(100e6); fmt.Println(os.Executable());};};|gofmt > main.go && go build main.go && start main.exe && dir

What did you expect to see?

Copy main.exe -> ~main.exe
Delete main.exe
Resume build

What did you see instead?

Move main.exe -> ~main.exe
Resume build

What operating system and processor architecture are you using (go env)?

set GOARCH=amd64
set GOBIN=C:\gotools
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\g
set GORACE=
set GOROOT=C:\Go
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0
set CXX=g++
set CGO_ENABLED=1
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config

@as as changed the title go/build: don't move existing binaries on windows go/build: copy/delete instead of rename existing binaries on windows Sep 24, 2017
@ghost
Copy link

ghost commented Sep 24, 2017

I sent an e-mail to an MS employee about this.

Considering backup files, I note that running Emacs' eshell to overwrite a text file (on Windows) creates a backup.

@ianlancetaylor ianlancetaylor changed the title go/build: copy/delete instead of rename existing binaries on windows cmd/go: copy/delete instead of rename existing binaries on windows Sep 24, 2017
@ianlancetaylor
Copy link
Contributor

I don't actually understand what the problem is. In this scenario, the go tool has been asked to create an executable. It is possible that that executable is running which the go tool is also running. This typically happens when using go install to update a binary that happens to be running already. On Unix, this is straightforward: we simply remove the executable and then open the pathname for writing. This does not work on Windows, because Windows does not permit removing a running executable. So on Windows, as a workaround, we rename the executable, because Windows does permit that. Then we proceed as normal. The code in question can be found in cmd/go/internal/work/build.go:

	mayberemovefile(dst)
	df, err := os.OpenFile(dst, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, perm)
	if err != nil && base.ToolIsWindows {
		// Windows does not allow deletion of a binary file
		// while it is executing. Try to move it out of the way.
		// If the move fails, which is likely, we'll try again the
		// next time we do an install of this binary.
		if err := os.Rename(dst, dst+"~"); err == nil {
			os.Remove(dst + "~")
		}
		df, err = os.OpenFile(dst, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, perm)
	}

Your proposal for what we should do instead will not work. We have already tried and failed to remove the executable. It won't help to copy the file first.

We could say that using go install to update a running binary simply fails on Windows, but that seems like a regression from past releases.

@as
Copy link
Contributor Author

as commented Sep 24, 2017

I primarily use UNIX based systems, but a quick peer survey returned the conclusion that this behavior was confusing on Windows.

From the perspective of a UNIX user, there is no problem because this approximates the behavior of a UNIX system. But Windows is not a UNIX system and doesn't normally allows files associated with a running process to be altered.

  • If I try to delete a running main.exe on Windows, it will fail. This is the expected behavior on a Windows box.
  • If I do a go build, that executable is successfully replaced. This is unexpected. You can't uninstall a program, delete it, or write to it, but you can go build it.

It should be the user's responsibility to satisfy the conditions of altering a file on their system. Windows users expect files associated with running processes to be immutable, therefore go build should fail if that precondition is not met.

The community can decide whether this is a compelling issue or not. The goal was to get the data on the issue tracker for those who search for it later.

@ianlancetaylor
Copy link
Contributor

OK. But the suggestion of copy/delete does not make sense. It sounds like you are suggesting that if go install is used to replace a binary that is currently running, we should simply let the build fail.

This would essentially reverse https://golang.org/cl/5502054 . CC @alexbrainman .

It's worth noting that this change would almost certainly break the special case of go install cmd/go.

@as
Copy link
Contributor Author

as commented Sep 25, 2017

we should simply let the build fail.

Yes, that would be the idealistic behavior on a Windows system. Reading the cl gave me some perspective on how long this behavior has been in place, and it does indeed suggest that a change would break things.

Perhaps there is a way to inform the user that a file associated with a running process is being moved, at least then the element of surprise is gone.

Another odd behavior occurs when the backup file main.exe~ already exists. In this case go build main.go fails, saying: go install command-line-arguments: open main.exe: The process cannot access the file because it is being used by another process.

mkdir goissue21997 & cd goissue21997 && echo package main;import "time";import "os";import "fmt"; func main(){println("1");for { time.Sleep(100e6); fmt.Println(os.Executable());};};|gofmt > main.go && go build main.go && start main.exe && echo package main;import "time";import "os";import "fmt"; func main(){println("2");for { time.Sleep(100e6); fmt.Println(os.Executable());};};|gofmt > main.go && go build main.go && start main.exe && go build main.go

@alexbrainman
Copy link
Member

we should simply let the build fail.

Yes, that would be the idealistic behavior on a Windows system.

I am not sure I agree that this is ideal. At this moment "go build" and "go install" just silently replaces executable file whether it is running or not - the user does not need to worry if executable is running or not. What you are proposing will make "go build" and "go install" fail occasionally with "executable file is used by another process". People will come here and complain about "go build" is broken sometimes, and we would have to explain the details.

I can see that it confuses knowledgeable users like yourself. But I think this is small price to pay - you could always look at the code or ask here.

Perhaps there is a way to inform the user that a file associated with a running process is being moved, at least then the element of surprise is gone.

How do you propose to do that? We don't want warnings that user should not act upon. And these will only happens occasionally - because there is no reliable way to determine if process is exited or not. I also wouldn't be surprised if we would get many triggers from anti-virus software opening executable file during or after its run.

Another odd behavior occurs when the backup file main.exe~ already exists. In this case go build main.go fails, saying: go install command-line-arguments: open main.exe: The process cannot access the file because it is being used by another process.

Yes that happens when you are trying to actually run new copy of main.exe while previous main.exe process is still running, and we tried our trick of renaming main.exe into main.exe~ and that failed (so we are out of all options). I think the error message is clear about what has happened. So the user has to terminate previous process before trying to build new executable.

I think what we have now is as good as it can be. But if you think you can improve the situation, go ahead and propose you changes. Do not forget, you need to address Ian's comment of:

It's worth noting that this change would almost certainly break the special case of go install cmd/go.

I suggest you find solution to that problem first.

Alex

@as
Copy link
Contributor Author

as commented Sep 25, 2017

I suggest you find solution to that problem first.

Given that the behavior existed for 5+ years and this seems to be the only issue logged against it, I don't think the improvement is worth pursuing. Nobody else seems to have an opinion on it except @forskning either, so it's probably best we just leave things as they are in a working state.

@alexbrainman
Copy link
Member

@as I will close this as "working as expected". But feel free to reopen if you have some new suggestions.

Alex

@golang golang locked and limited conversation to collaborators Sep 25, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants