/cc @alexbrainman

Thank you for the report. I will investigate this when I have time.

Alex

Thanks @alexbrainman let me know if I can provide any additional info.

I did a little more investigating. This following program also fails:

package main

import "os"
import "fmt"

func main() {
    path, err := os.Readlink("C:\\host")
    if err != nil {
        panic(fmt.Errorf("os.Readlink failed: %v", err))
    }

    println("readlink path", path)

    fi, err := os.Open(path)
    if err != nil {
        panic(fmt.Errorf("os.Open failed: %v", err))
    }

    defer fi.Close()

    println("file info", fi.Name())
}

readlink path \ContainerMappedDirectories\FF9714DB-37DA-4741-AC60-876DD8CFDFF2
panic: os.Open failed: open \ContainerMappedDirectories\FF9714DB-37DA-4741-AC60-876DD8CFDFF2: The system cannot find the path specified.

However if I prepend \\? to the path returned by os.Readlink, os.Open succeeds:

package main

import "os"
import "fmt"

func main() {
    path, err := os.Readlink("C:\\host")
    if err != nil {
        panic(fmt.Errorf("os.Readlink failed: %v", err))
    }

    println("readlink path", path)

    fi, err := os.Open("\\\\?" + path)
    if err != nil {
        panic(fmt.Errorf("os.Open failed: %v", err))
    }

    defer fi.Close()

    println("file info", fi.Name())
}

readlink path \ContainerMappedDirectories\846DD71E-ABA5-4F3B-93EB-0276148F0F98
file info \\?\ContainerMappedDirectories\846DD71E-ABA5-4F3B-93EB-0276148F0F98

Is is possible that os.Readlink on Windows doesn't handle the \\?\ path style?

let me know if I can provide any additional info.

I just need to be able to reproduce your problem, and I will need Docker installed for that. What is the easiest way to install Docker if I have Windows 10?

You can also try and fix this yourself. If you try, you need to be aware about forthcoming change https://go-review.googlesource.com/#/c/go/+/86556/ that might affect your problem. Also Windows paths starting with \?\ are not handled very well in many Go standard library parts. For example path/filepath does not handle these paths at all. Also see issue #22230.

Is is possible that os.Readlink on Windows doesn't handle the \?\ path style?

os.Readlink is broken on Windows. os.Readlink current implementation does not work in many situations, and I do not know of a way to implement it properly. But lets not discuss os.Readlink here. Create new issue for os.Readlink, if you think it is important.

Alex

@alexbrainman you can install Docker for Windows 10 by following the instructions here. You will want to make sure that you switch to Windows Containers (the second step).

I'll see if I can spend some time looking at this as well.

@alexbrainman you can install Docker for Windows 10 by following the instructions here. You will want to make sure that you switch to Windows Containers (the second step).

When I run "Docker for Windows" app, I get this message

but I do not want to proceed, because I do use VirtualBox. I do not want to loose ability to run VirtualBox. What should I do?

Thank you.

Alex

It is specifically the hardware virtualization extensions that will not work; Hyper-V acquires them before Windows boots and Virtualbox can't use them anymore. This can be restored by disabling the hyper-v service in services.msc.

I happen to have Docker Enterprise on a W10 system and can attempt to reproduce it if that workaround isn't possible

It is specifically the hardware virtualization extensions that will not work; Hyper-V acquires them before Windows boots and Virtualbox can't use them anymore.

I discovered that much myself.

I happen to have Docker Enterprise on a W10 system and can attempt to reproduce it if that workaround isn't possible

I will try https://cloud.google.com/compute/docs/containers/#docker_on_windows when I have time.

Alex

@alexbrainman the link above should work on Windows Server 2016, without the need to disable Virtualbox.

@alexbrainman the link above should work on Windows Server 2016, without the need to disable Virtualbox.

If you are referring to https://docs.microsoft.com/en-us/virtualization/windowscontainers/quick-start/quick-start-windows-10 then it does not help me, because I only have Windows 10 computer, not Windows Server 2016,

If you are referring to https://cloud.google.com/compute/docs/containers/#docker_on_windows then yes, it should work. But I would need to create computer from scratch and install all the tools I need. So I will need some free time.

Alex

@tescherm can you please re-check that the issue was fixed in Go 1.11 (e83601b)?

@kolyshkin I can confirm that this is still an issue. Running docker run --rm -v c:\:c:\host golang:1.11-nanoserver-sac2016 go run c:\host\eval_symlinks.go with the program above yields:

@tescherm I am not surprised that os.Readlink is broken on windows. But maybe path/filepath.EvalSymlinks is OK. Please, try this program instead

package main

import (
	"fmt"
	"path/filepath"
)

func main() {
	path, err := filepath.EvalSymlinks("c:\\host")
	if err != nil {
		panic(err)
	}

	fmt.Printf("eval path: %s\n", path)
}

If it still fails, someone would have to investigate where it fails. I still do not have Windows Docker installed on my computer - I use VirtualBox, and, apparently, you cannot have both. So I would need to configure new PC somewhere (probably in the cloud), and have Go development environment installed on it before I can even start investigating this. I have very little free time nowadays, but I will do that, if nothing else.

Thank you.

Alex

@alexbrainman I tried it out, unfortunately the program also fails on 1.11:

PS C:\> docker run --rm -v c:\:c:\host golang:1.10-nanoserver-sac2016 go run c:\host\eval_symlinks.go
panic: GetFileAttributesEx \ContainerMappedDirectories: The system cannot find the file specified.

goroutine 1 [running]:
main.main()
        c:/host/eval_symlinks.go:11 +0xf0
exit status 2

PS C:\> docker run --rm -v c:\:c:\host golang:1.11-nanoserver-sac2016 go run c:\host\eval_symlinks.go
panic: FindFirstFile \ContainerMappedDirectories: The system cannot find the file specified.

goroutine 1 [running]:
main.main()
        c:/host/eval_symlinks.go:11 +0xf0
exit status 2

@tescherm thanks for checking. I would have to debug this myself when I have time.

Alex

@tescherm

I managed to setup appropriate environment for your test. I build this program

package main

import (
	"fmt"
	"path/filepath"
)

func main() {
	path, err := filepath.EvalSymlinks("c:\\host")
	if err != nil {
		panic(err)
	}

	fmt.Printf("eval path: %s\n", path)
}

and copied resulting exe into c:\test.exe, and then run this command

docker run --rm -v c:\:c:\host microsoft/nanoserver c:\host\test.exe

I can reproduce your error now.

Looking at the code, we call Windows API GetFinalPathNameByHandle with c:\host and VOLUME_NAME_DOS inside the container, and that call fails with ERROR_FILE_NOT_FOUND. I tried all VOLUME_NAME_DOS alternatives, and only VOLUME_NAME_NT works - it converts c:\host into \\Device\\HarddiskVolume1\\.

If we adjust this code to return \\Device\\HarddiskVolume1\\ in your situation, would that help you? How? Please explain in details, what it would give you.

path/filepath.EvalSymlinks is started as code to follow symlinks on Unix. I am not sure what we have here is covered by that description.

Thank you.

Alex

@alexbrainman great, yes, I believe that returning \\Device\\HarddiskVolume1\\ in the example above would help. The goal would be that this path resolves with other fs system calls. For example:

package main

import "os"
import "fmt"

func main() {
    path, err := filepath.EvalSymlinks("c:\\host")
    if err != nil {
        panic(err)
    }

    fmt.Printf("eval path: %s\n", path)

    // open resolves correctly
    fi, err := os.Open(path)
    if err != nil {
        panic(fmt.Errorf("os.Open failed: %v", err))
    }

    defer fi.Close()

    println("file info", fi.Name())
}

Thanks again for drilling in.

I believe that returning \\Device\\HarddiskVolume1\\ in the example above would help. The goal would be that this path resolves with other fs system calls.

If we return \Device\HarddiskVolume1\, that name would refer to
directory inside you container. For example, if your current drive is
C:, then \Device\HarddiskVolume1\ would mean
C:\Device\HarddiskVolume1\. I don't think that is what you want. You
want C:\ outside of container.

The \Device\HarddiskVolume1\ returned by calling
GetFinalPathNameByHandle with VOLUME_NAME_NT, is not proper Windows
path. Search for "NT Namespaces" in
https://docs.microsoft.com/en-us/windows/desktop/fileio/naming-a-file
to understand what it is. I doubt that Windows CreateFile API (and other APIs that Go uses) can access "NT Namespaces" paths.

And, in #23512 (comment) , you did not explain why you need to to call filepath.EvalSymlinks? Why cannot you call os.Open("c:\host") instead?

Alex

Note that filepath.EvalSymlinks has been rewritten completely for 1.12 in https://golang.org/cl/121676 for #23444. It may not affect this case, though.

It may not affect this case, though.

I agree. https://golang.org/cl/121676 is unrelated to this issue. It is still not clear to me what filepath.EvalSymlink should do to path inside container that points to volume outside container. Maybe it should do nothing.

Alex