Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

os: SameFile doesn't handle invalid file IDs on Windows #62042

Open
qmuntal opened this issue Aug 15, 2023 · 4 comments
Open

os: SameFile doesn't handle invalid file IDs on Windows #62042

qmuntal opened this issue Aug 15, 2023 · 4 comments
Labels
NeedsFix The path to resolution is known, but the work has not been done. OS-Windows
Milestone
Go1.23

Comments

@qmuntal
Copy link
Contributor

qmuntal commented Aug 15, 2023

What version of Go are you using (go version)?

$ go version
go version go1.20.7 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\qmuntaldiaz\AppData\Local\go-build
set GOENV=C:\Users\qmuntaldiaz\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\qmuntaldiaz\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\qmuntaldiaz\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Users\qmuntaldiaz\sdk\go1.20.7
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Users\qmuntaldiaz\sdk\go1.20.7\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.20.7
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\Users\qmuntaldiaz\code\gotest\go.mod
set GOWORK=
set CGO_CFLAGS=-O2 -g
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-O2 -g
set CGO_FFLAGS=-O2 -g
set CGO_LDFLAGS=-O2 -g
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=C:\Users\QMUNTA~1\AppData\Local\Temp\go-build3933964541=/tmp/go-build -gno-record-gcc-switches

What did you do?

package main

import "os"

func main() {
	fi1, err := os.Stat(`\\.\pipe\`)
	if err != nil {
		panic(err)
	}
	fi2, err := os.Stat("NUL")
	if err != nil {
		panic(err)
	}
	if os.SameFile(fi1, fi2) {
		panic(`os.SameFile(fi, fi2) = true`)
	}
}

What did you expect to see?

os.SameFile does not report \.\pipe\ and NUL as the same file

What did you see instead?

os.SameFile reports \.\pipe\ and NUL as the same file

@golang/windows
@qmuntal qmuntal added OS-Windows NeedsFix The path to resolution is known, but the work has not been done. labels Aug 15, 2023
@qmuntal qmuntal added this to the Go1.22 milestone Aug 15, 2023
@qmuntal
Copy link
Contributor Author

qmuntal commented Aug 15, 2023 •
edited

The root cause is the following: even if the file system supports file IDs, there are some files which, for whatever reason, don't have one assigned. MS-FSCC 2.1.7 specify how to handle this cases:

For file systems that do not support a 64-bit file ID, this field MUST be set to 0, and MUST be ignored.

For files for which a unique 64-bit file ID cannot be established, this field MUST be set to 0xffffffffffffffff, and MUST be ignored.

os.SameFile is not following this part of the spec. As \.\pipe\ and NUL file ID is 0, then it things they both are the same file.

Fixing this is not as straightforward as it seems, because we still want, for example, that os.SameFile("NUL", "NUL") returns true. I'm afraid a foolproof solution doesn't exist, as we have to fallback to comparing paths, which can led to false negatives in the presence of hard links. This is the best guidance I could find: https://devblogs.microsoft.com/oldnewthing/20220128-00/?p=106201.

If the volume doesn’t support 64-bit identifiers either, then the situation is getting kind of desperate. My colleague Malcolm Smith suggests calling Get­Final­Path­Name­By­Handle with FILE_NAME_NORMALIZED to try to get the file names into some sort of sane state, and then comparing the names.

@gopherbot
Copy link

Change https://go.dev/cl/543815 mentions this issue: os: fix SameFile()

@gopherbot
Copy link

Change https://go.dev/cl/543835 mentions this issue: os: fix SameFile()

@qmuntal qmuntal modified the milestones: Go1.22, Go1.23 Dec 27, 2023
@gopherbot
Copy link

Change https://go.dev/cl/574155 mentions this issue: os: make readdir more robust on Windows

gopherbot pushed a commit that referenced this issue Mar 27, 2024
@qmuntal
os: make readdir more robust on Windows
2860e01 
On Windows, File.readdir currently fails if the volume information
can't be retrieved via GetVolumeInformationByHandle and if the
directory path is relative and can't be converted to an absolute
path.

This change makes readdir more robust by not failing in these cases,
as these steps are just necessary to support a potential call to
os.SameFile, but not for the actual readdir operation. os.SameFile
will still fail in these cases, but that's a separate issue tracked
in #62042.

Change-Id: I8d98d8379bdac4b2832fa433432a5f027756abaa
Reviewed-on: https://go-review.googlesource.com/c/go/+/574155
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Mchnan pushed a commit to Mchnan/go-sylixos that referenced this issue Mar 28, 2024
@qmuntal
os: make readdir more robust on Windows
4d6c5e9 
On Windows, File.readdir currently fails if the volume information
can't be retrieved via GetVolumeInformationByHandle and if the
directory path is relative and can't be converted to an absolute
path.

This change makes readdir more robust by not failing in these cases,
as these steps are just necessary to support a potential call to
os.SameFile, but not for the actual readdir operation. os.SameFile
will still fail in these cases, but that's a separate issue tracked
in golang#62042.

Change-Id: I8d98d8379bdac4b2832fa433432a5f027756abaa
Reviewed-on: https://go-review.googlesource.com/c/go/+/574155
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsFix The path to resolution is known, but the work has not been done. OS-Windows
Projects
None yet
Milestone
Go1.23
Development

No branches or pull requests
2 participants
@qmuntal @gopherbot