Skip to content

archive/tar: Writer produces bad tar files on Windows #34839

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
DeedleFake opened this issue Oct 11, 2019 · 4 comments
Closed

archive/tar: Writer produces bad tar files on Windows #34839

DeedleFake opened this issue Oct 11, 2019 · 4 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.

Comments

@DeedleFake
Copy link

DeedleFake commented Oct 11, 2019 •
edited
Loading

What version of Go are you using (go version)?

$ go version
go version go1.13.1 windows/amd64

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=$HOME\AppData\Local\go-build
set GOENV=$APPDATA\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=$HOME\Devel\Go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=$HOME\scoop\apps\go\current
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=$HOME\scoop\apps\go\current\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=$TEMP\tmp.a13440\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=$TEMP\go-build061456054=/tmp/go-build -gno-record-gcc-switches

What did you do?

https://play.golang.org/p/dmkEunM6jGX

Running this on Linux or Darwin produces a tar file containing two files, Test and Other, as expected. Running this on Windows, however, produces a file which the file command identifies as data, not a tar file, and running any tar command against it results in errors. Running tar -t with the GNU tar command yields

tar: This does not look like a tar archive
tar: Skipping to next header
tar: A lone zero block at 4
tar: Exiting with failure status due to previous errors

while bsdtar, which the tar.exe that comes with Windows is a port of, yields

tar : tar.exe: Error opening archive: Unrecognized archive format
@DeedleFake DeedleFake mentioned this issue Oct 11, 2019
Closed
@ianlancetaylor
Copy link
Member

CC @dsnet

@mattn
Copy link
Member

mattn commented Oct 11, 2019

I confirmed generated files on Linux/Windows are both same. The problem is that the generated tar file seems not friendly for Windows. Using tar.exe bundled in msys2/mingw

image

Using bsdtar.exe bundled in msys2/mingw

image

@smasher164 smasher164 added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Oct 11, 2019
@DeedleFake
Copy link
Author

DeedleFake commented Oct 11, 2019 •
edited
Loading

The same GNU tar installation on the same Linux that could read the ones produced on Linux for be was unable to read the files produced on Windows.

I haven't done a binary comparison between the two. Since file reports different things for each I just assumed it was different. I'll try that when I get a chance.

@DeedleFake
Copy link
Author

DeedleFake commented Oct 11, 2019 •
edited
Loading

It was different, but...

False alarm. Sorry about that. It looks like this is a problem with PowerShell. The | and > operators appear to be corrupting the data somehow, so go run test.go > test.tar was creating a bad tar file. It works fine in the CMD, though.

Edit: If anyone stumbles across this at some point, see https://brianreiter.org/2010/01/29/powershells-object-pipeline-corrupts-piped-binary-data/. Apparently it converts everything to UTF-16 by default instead of just piping it through. There doesn't seem to be a generic way around it, unfortunately.

@golang golang locked and limited conversation to collaborators Oct 10, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@mattn @DeedleFake @ianlancetaylor @gopherbot @smasher164