Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

archive/tar: TestSparseFile fails #21958

Closed
hirochachacha opened this issue Sep 21, 2017 · 9 comments
Closed

archive/tar: TestSparseFile fails #21958

hirochachacha opened this issue Sep 21, 2017 · 9 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Go1.10

Comments

@hirochachacha
Copy link
Contributor

Please answer these questions before submitting your issue. Thanks!

What did you do?

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

./all.bash

What did you expect to see?

success

What did you see instead?

--- FAIL: TestSparseFiles (1.80s)
    --- FAIL: TestSparseFiles/EmptyFile (0.00s)
    	tar_test.go:887: unexpected DetectSparseHoles error: seek /var/folders/wq/dwn8hs0x7njbzty9f68y61700000gn/T/028689629: inappropriate ioctl for device
    --- FAIL: TestSparseFiles/BigData (0.01s)
    	tar_test.go:887: unexpected DetectSparseHoles error: seek /var/folders/wq/dwn8hs0x7njbzty9f68y61700000gn/T/492668695: inappropriate ioctl for device
    --- FAIL: TestSparseFiles/BigHole (0.00s)
    	tar_test.go:887: unexpected DetectSparseHoles error: seek /var/folders/wq/dwn8hs0x7njbzty9f68y61700000gn/T/643686241: inappropriate ioctl for device
    --- FAIL: TestSparseFiles/DataFront (0.00s)
    	tar_test.go:887: unexpected DetectSparseHoles error: seek /var/folders/wq/dwn8hs0x7njbzty9f68y61700000gn/T/361682491: inappropriate ioctl for device
    --- FAIL: TestSparseFiles/HoleFront (0.00s)
    	tar_test.go:887: unexpected DetectSparseHoles error: seek /var/folders/wq/dwn8hs0x7njbzty9f68y61700000gn/T/025217573: inappropriate ioctl for device
    --- FAIL: TestSparseFiles/DataMiddle (0.00s)
    	tar_test.go:887: unexpected DetectSparseHoles error: seek /var/folders/wq/dwn8hs0x7njbzty9f68y61700000gn/T/420669343: inappropriate ioctl for device
    --- FAIL: TestSparseFiles/HoleMiddle (0.00s)
    	tar_test.go:887: unexpected DetectSparseHoles error: seek /var/folders/wq/dwn8hs0x7njbzty9f68y61700000gn/T/479895849: inappropriate ioctl for device
    --- FAIL: TestSparseFiles/Multiple (1.76s)
    	tar_test.go:887: unexpected DetectSparseHoles error: seek /var/folders/wq/dwn8hs0x7njbzty9f68y61700000gn/T/275436867: inappropriate ioctl for device
2017/09/21 12:07:55 seek /var/folders/wq/dwn8hs0x7njbzty9f68y61700000gn/T/sparse.db734036422: inappropriate ioctl for device
FAIL	archive/tar	1.934s

Does this issue reproduce with the latest release (go1.9)?

No

System details

go version devel +589ea93678 Thu Sep 21 01:01:44 2017 +0000 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/hiro/.go"
GORACE=""
GOROOT="/Users/hiro/go"
GOTOOLDIR="/Users/hiro/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/wq/dwn8hs0x7njbzty9f68y61700000gn/T/go-build263216468=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOROOT/bin/go version: go version devel +589ea93678 Thu Sep 21 01:01:44 2017 +0000 darwin/amd64
GOROOT/bin/go tool compile -V: compile version devel +589ea93678 Thu Sep 21 01:01:44 2017 +0000
uname -v: Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.12.6
BuildVersion:	16G29
lldb --version: lldb-900.0.45
  Swift-4.0
gdb --version: GNU gdb (GDB) 8.0

It happens after 1eacf78
@hirochachacha
Copy link
Contributor Author

/CC @dsnet

@ianlancetaylor ianlancetaylor added this to the Go1.10 milestone Sep 21, 2017
@ianlancetaylor ianlancetaylor added the NeedsFix The path to resolution is known, but the work has not been done. label Sep 21, 2017
@dsnet
Copy link
Member

dsnet commented Sep 21, 2017 •
edited

Can you do show me the contents of the mount command? I'm interested in knowing what the underlying filesystem of /var/folders is.

For example, on mine:

rawr@khaydarin: ~ $ mount | grep " / "
/dev/disk1 on / (hfs, local, journaled)

(of course /var/folders might not be on the root filesystem like it is on my machine)

@hirochachacha
Copy link
Contributor Author 

$ mount
/dev/disk1 on / (hfs, local, journaled)
devfs on /dev (devfs, local, nobrowse)
map -hosts on /net (autofs, nosuid, automounted, nobrowse)
map auto_home on /home (autofs, automounted, nobrowse)

It looks like nothing special.

@dsnet
Copy link
Member

dsnet commented Sep 21, 2017 •
edited

The odd part is that I'm unable to reproduce this. I am on Darwin 16.4.0, but I'm hoping that doesn't make much of a difference.

The problem is that my check in sparse_unix.go:26:

// Check for seekData/seekHole support.
if _, err := f.Seek(0, seekHole); errno(err) == syscall.EINVAL {
    return nil, nil // Either old kernel or FS does not support this
}

is clearly not triggering for some reason on your system.

The man page on lseek makes no mention about ENOTTY being returned, but does say that EINVAL should be returned for an invalid whence, for which this should be the case.

A part of me is tempted to catch ENOTTY as well. This mailing list (not OSX specific) seems to suggest that ENOTTY is a possible return value:

What should happen if a related filesystem implementation does not support SEEK_HOLE/SEEK_DATA? Note that modern OS implement loadable drivers and thus a binary only filesystem drivers may return e.g. ENOTTY (illegal ioctl) in such a case.

However, the source code for BSD tar just does a very liberal check where it assumes there is no support if any error is returned for lseek(..., 0, SEEK_HOLE). Similarly, GNU tar also does a blanket check where it assumes any error implies no support is provided.

@hirochachacha, can you test the following variation of the check on sparse_unix.go:26 and tell me if it fixes the issue?

if _, err := f.Seek(0, seekHole); errno(err) == syscall.EINVAL || errno(err) == syscall.ENOTTY {
    return nil, nil
}

I'm leaning towards just treating any error as having no support, but I want to make sure I'm on the right track.

@gopherbot
Copy link

Change https://golang.org/cl/65191 mentions this issue: archive/tar: make check for hole detection support more liberal

@hirochachacha
Copy link
Contributor Author

@hirochachacha, can you test the following variation of the check on sparse_unix.go:26 and tell me if it fixes the issue?

It works fine! I remembered that my SSD is not made by apple, I replaced it before.
Perhaps, that is related?

@dsnet
Copy link
Member

dsnet commented Sep 21, 2017

Quite possibly, seems to match the statement:

Note that modern OS implement loadable drivers and thus a binary only filesystem drivers may return e.g. ENOTTY (illegal ioctl) in such a case.

Either way. Rather than playing whack-a-mole with every possible errno representing "not supported", I'm going to take the GNU and BSD tar approach and just consider any error as meaning "non supported".

@hirochachacha
Copy link
Contributor Author

Fair enough. Thank you for your fix.

@dsnet
Copy link
Member

dsnet commented Sep 21, 2017

Thank you for the bug report!

@mundaym mundaym mentioned this issue Sep 21, 2017
Closed
@golang golang locked and limited conversation to collaborators Sep 21, 2018
@rsc rsc unassigned dsnet Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Milestone
Go1.10
Development

No branches or pull requests
4 participants
@hirochachacha @ianlancetaylor @dsnet @gopherbot