Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: 'go build' fails when current directory and output directory are symlinks on Windows #27515

Closed
ankeesler opened this issue Sep 5, 2018 · 11 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. OS-Windows
Milestone

Comments

@ankeesler
Copy link

What version of Go are you using (go version)?

go version go1.11 windows/amd64

Does this issue reproduce with the latest release?

Yes. This is a regression from go1.10.3.

What operating system and processor architecture are you using (go env)?

Windows Server 1709 (OS Version 10.0.16299) and Windows Server 1803 (OS Version 10.0.17134).

set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Administrator\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\Administrator\go
set GOPROXY=
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g`<no output>` -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\Administrator\AppData\Local\Temp\go-build075376310=/tmp/go-build -gno-record-gcc-switches

What did you do?

On Windows, create this directory structure.

C:\some-directory\
  hello\
  src\
  src2 [-> symlinked to C:\some-directory\src]
    hello [-> symlinked to C:\some-directory\hello]
    hello.go

From the directory C:\some-directory\src2\, run go build -o c:\some-directory\src2\hello\hello.exe .\hello.go.

What did you expect to see?

The command succeeds and builds hello.exe in the directory C:\some-directory\hello\.

What did you see instead?

go build command-line-arguments: mkdir C:\some-directory\src2\hello\: Cannot create a file when that file already exists.
ankeesler pushed a commit to cloudfoundry/garden-windows-ci that referenced this issue Sep 5, 2018
See golang/go#27515 for context

[#160287101]

Signed-off-by: Andrew Keesler <akeesler@pivotal.io>
@andybons andybons added OS-Windows NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Sep 5, 2018
@andybons
Copy link
Member

andybons commented Sep 5, 2018

@bcmills @rsc

@andybons andybons added this to the Unplanned milestone Sep 5, 2018
arjun024 added a commit to cloudfoundry/winc-release that referenced this issue Sep 6, 2018
This reverts commit 020a875.

We are holding off on upgrading to Go 1.11 until the following symlink
issue is fixed: golang/go#27515.

Signed-off-by: Andrew Keesler <akeesler@pivotal.io>
@alexbrainman
Copy link
Member

@ankeesler I can reproduce this here, thank you very much.. It is broken on the tip too.

Alex

@andybons andybons added the NeedsFix The path to resolution is known, but the work has not been done. label Sep 10, 2018
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 10, 2018
@andybons andybons modified the milestones: Unplanned, Go1.11.1 Sep 10, 2018
@ankeesler
Copy link
Author

Thanks @andybons + @alexbrainman. Let me know if you need anything else from me, happy to help.

sunjayBhatia added a commit to cloudfoundry/diego-release that referenced this issue Sep 19, 2018
Defensively checks if $BOSH_INSTALL_TARGET is a link, if so it installs
binaries in the link's target

See golang/go#27515 for more context

This can be reverted once we have a go version with the fix

[#160103190](https://www.pivotaltracker.com/story/show/160103190)
@rsc
Copy link
Contributor

rsc commented Sep 20, 2018

I'm sorry this broke but we just fundamentally do not support magic tricks with symlinks. The answer is almost certainly to stop trying to trick the go command using symlinks.

@ankeesler
Copy link
Author

@rsc thanks for taking a look here. We were hoping that since this works on Linux and it has worked in previous versions on Windows, then it would continue to function the same way going forward. Windows symlinks can certainly by annoying at times (most times). All this being said, is there any chance that this will be fixed in a future go version?

We are working with an interface that expects the directory hierarchy outlined in my initial comment. Since this works on Linux, it will likely be difficult, if not impossible, for us to change the hierarchy. This interface is used in our automated deployment tooling for Kubernetes.

@alexbrainman
Copy link
Member

I have used https://play.golang.org/p/xYnM-XHKDve to find the commit that broke the test:

e83601b4356f92f2f4d05f302d5654754ff05a6d is the first bad commit
commit e83601b4356f92f2f4d05f302d5654754ff05a6d
Author: Alex Brainman <alex.brainman@gmail.com>
Date:   Sun Jan 7 12:12:25 2018 +1100

    os: use WIN32_FIND_DATA.Reserved0 to identify symlinks

    os.Stat implementation uses instructions described at
    https://blogs.msdn.microsoft.com/oldnewthing/20100212-00/?p=14963/
    to distinguish symlinks. In particular, it calls
    GetFileAttributesEx or FindFirstFile and checks
    either WIN32_FILE_ATTRIBUTE_DATA.dwFileAttributes
    or WIN32_FIND_DATA.dwFileAttributes to see if
    FILE_ATTRIBUTES_REPARSE_POINT flag is set.
    And that seems to worked fine so far.

    But now we discovered that OneDrive root folder
    is determined as directory:

    c:\>dir C:\Users\Alex | grep OneDrive
    30/11/2017  07:25 PM    <DIR>          OneDrive
    c:\>

    while Go identified it as symlink.

    But we did not follow Microsoft's advice to the letter - we never
    checked WIN32_FIND_DATA.Reserved0. And adding that extra check
    makes Go treat OneDrive as symlink. So use FindFirstFile and
    WIN32_FIND_DATA.Reserved0 to determine symlinks.

    Fixes #22579

    Change-Id: I0cb88929eb8b47b1d24efaf1907ad5a0e20de83f
    Reviewed-on: https://go-review.googlesource.com/86556
    Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
    Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
    TryBot-Result: Gobot Gobot <gobot@golang.org>

:040000 040000 6e4ea25b8e56d55d3119d258431eb49b66ac19fc a6eac27eee057861c2fa8254
dc627fa181fd6ae5 M      src

It looks like cmd/go fails when it calls os.MkdirAll, which calls os.Stat of C:\Users\Alex\AppData\Local\Temp\TestAlex496017355\src2\hello\ - note the backslash at the end of the path, and C:\Users\Alex\AppData\Local\Temp\TestAlex496017355\src2 is the symlink. os.Stat of such directory succeeds on linux, and returns directory, but fails on windows.

So this looks like a dup of #27225 or very similar.

If someone wants to have these 2 issues fixed, please, review https://golang.org/cl/134195 .

Alex

yulianedyalkova added a commit to cloudfoundry/garden-runc-release that referenced this issue Sep 28, 2018
Temporary workaround for golang/go#27515

Co-authored-by: Julia Nedialkova <julianedialkova@hotmail.com>
davewalter added a commit to cloudfoundry/smb-volume-release that referenced this issue Oct 1, 2018
- defensively checks if $BOSH_INSTALL_TARGET is a link
- if so it installs binaries in the link's target

See golang/go#27515 for more context

This can be reverted once we have a go version with the fix

[#160669222](https://www.pivotaltracker.com/story/show/160669222)
davewalter added a commit to cloudfoundry-attic/local-volume-release that referenced this issue Oct 1, 2018
- defensively checks if $BOSH_INSTALL_TARGET is a link
- if so it installs binaries in the link's target

See golang/go#27515 for more context

This can be reverted once we have a go version with the fix

[#160669222](https://www.pivotaltracker.com/story/show/160669222)
micahyoung pushed a commit to cloudfoundry-attic/windows2016fs-online-release that referenced this issue Oct 3, 2018
Defensively checks if $BOSH_INSTALL_TARGET is a link, if so it installs
binaries in the link's target

See golang/go#27515 for more context

This can be reverted once we have a go version with the fix

Signed-off-by: Amin Jamali <ajamali@pivotal.io>
Signed-off-by: Yael Harel <yharel@pivotal.io>
micahyoung pushed a commit to cloudfoundry/winc-release that referenced this issue Oct 3, 2018
Defensively checks if $BOSH_INSTALL_TARGET is a link, if so it installs
binaries in the link's target

See golang/go#27515 for more context

This can be reverted once we have a go version with the fix

Signed-off-by: Amin Jamali <ajamali@pivotal.io>
Signed-off-by: Micah Young <myoung@pivotal.io>
@gopherbot
Copy link

Change https://golang.org/cl/143578 mentions this issue: os: use CreateFile for Stat of symlinks

@alexbrainman
Copy link
Member

@ankeesler please try https://go-review.googlesource.com/c/go/+/143578

Does it fixes your problem?

Thank you.

Alex

@ankeesler
Copy link
Author

@alexbrainman thank you for your work on this. We will try this out and let you know.

@ankeesler
Copy link
Author

@alexbrainman we verified that this change does indeed fix the issue. Thank you.

@alexbrainman
Copy link
Member

@alexbrainman we verified that this change does indeed fix the issue

Thank you for checking. I will have it reviewed and submitted.

Alex

@golang golang locked and limited conversation to collaborators Nov 2, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. OS-Windows
Projects
None yet
Development

No branches or pull requests

5 participants